Quick overview
Job description
Salvo Software is a global technology company specializing in custom software development and advanced engineering solutions. With distributed teams across the US, LATAM, and India, we partner with clients to build high-performance, scalable systems that solve complex technical challenges. Our culture values innovation, ownership, and engineering excellence. We’re growing our AI department and are looking for a Senior MLOps Engineer to build the platform that takes our models and LLM systems reliably into production.
Role Overview
We are looking for a Senior MLOps Engineer to design, build, and own the infrastructure and workflows behind our machine learning and LLM-powered systems. Your core work will be creating a reliable path from experimentation to production: training and fine-tuning pipelines, model registries, automated deployment, scalable inference, and monitoring for models running in cloud, on-prem, and air-gapped environments.
This is a senior, hands-on role. You’ll define MLOps standards and platform architecture, make key decisions on reliability, scalability, and cost, and provide technical guidance to AI Developers and RAG Engineers on the team. You’ll work closely with engineering, product, and client stakeholders to make sure AI features are reproducible, observable, secure, and ready to scale.
Key Responsibilities
MLOps Platform & Engineering (core focus)
- Design, deploy, and operate MLOps platform infrastructure, cloud services (AWS, Azure, GCP), and LLM inference pipelines (Ollama, vLLM).
- Build automated CI/CD pipelines, IaC, and monitoring/alerting systems to support RAG applications, AI agents, and production deployments.
- Manage Linux servers, networking, security endpoints, and troubleshoot production issues alongside AI engineering teams.
Technical Leadership
- Establish department-wide MLOps/DevOps standards, best practices, and reusable infrastructure templates.
- Partner with AI Developers and RAG Engineers to guide architecture, deployment strategies, tool selection, and production readiness.
- Translate feature requirements into reliable, secure, cost-effective infrastructure and maintain comprehensive system documentation.
Infrastructure & Operations
- Provision containerized workloads, Linux servers, and infrastructure using IaC (Terraform, Ansible) across cloud and on-prem environments.
- Manage CI/CD (GitHub Actions, Azure DevOps), observability tools (Prometheus, Grafana), secure networking, and GPU-enabled hardware.
- Troubleshoot platform incidents, perform root cause analysis, and maintain operational resilience.
- 5+ years of senior-level experience in DevOps/SRE/platform engineering with at least 1 year deploying ML models and LLM inference engines (Ollama, vLLM) in production.
- Proficiency with Linux, Docker, Kubernetes, virtualization (Proxmox, VMware), GPU infrastructure (CUDA/NVIDIA), and major cloud platforms (AWS, Azure, GCP).
- Hands-on expertise with Infrastructure as Code (Terraform, Ansible), CI/CD platforms (GitHub Actions, Azure DevOps), Bash, and Python scripting.
- Solid background in core networking (DNS, TLS, firewalls, VPNs), monitoring/logging, and security/authentication (OAuth2/OIDC, API gateways, Zero Trust/SOC 2).
- Familiarity with ML tooling (MLflow, Kubeflow), vector DBs, RAG/agent frameworks (MCP), workflow orchestration (Airflow/Kafka), and model optimization/quantization.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Similar jobs
- TT
Senior AI/ML EngineerRecently posted Auto Apply
True Talent Solutions Partners · San Francisco · On-site · Posted 17 Nov 2026 🇺🇸
DockerGCPLLM+7Technology$180k - $275k/yr - RG
Machine Learning EngineerRecently posted Auto Apply
RZR Global Inc. · San Francisco · Hybrid · Posted 17 Nov 2026 🇺🇸
RustSQLFlink+10TechnologySalary not listed - SI

Principal AI/ML Engineer - VC Backed StartupsRecently posted Auto Apply
Signalfire · Remote - No Preference · Remote · Posted 17 Nov 2026 🇺🇸
Talent AcquisitionDockerGCP+23TechnologySalary not listed - SI

Principal AI/ML Engineer - VC Backed StartupsRecently posted Auto Apply
Signalfire · NYC - Hybrid · Hybrid · Posted 17 Nov 2026 🇺🇸
Talent AcquisitionDockerGCP+23TechnologySalary not listed - SI

Principal AI/ML Engineer - VC Backed StartupsRecently posted Auto Apply
Signalfire · Bay Area - Hybrid · Hybrid · Posted 17 Nov 2026 🇺🇸
Talent AcquisitionDockerGCP+23TechnologySalary not listed - SI

Head of AI/ML (Director/VP) - VC Backed StartupsRecently posted Auto Apply
Signalfire · Remote - No Preference · Remote · Posted 17 Nov 2026 🇺🇸
RecruitingTalent AcquisitionWorkforce Planning+22TechnologySalary not listed - SI

Head of AI/ML (Director/VP) - VC Backed StartupsRecently posted Auto Apply
Signalfire · NYC - Hybrid · Hybrid · Posted 17 Nov 2026 🇺🇸
RecruitingTalent AcquisitionWorkforce Planning+22TechnologySalary not listed - SI

Head of AI/ML (Director/VP) - VC Backed StartupsRecently posted Auto Apply
Signalfire · Bay Area - Hybrid · Hybrid · Posted 17 Nov 2026 🇺🇸
RecruitingTalent AcquisitionWorkforce Planning+22TechnologySalary not listed - PH

ML Infra Engineer, ComputeRecently posted Auto Apply
Physicalintelligence · San Francisco · On-site · Posted 16 Nov 2026 🇺🇸
GCPCapacity PlanningPyTorch+4Salary not listed - TR
Automated Driving Advanced Development Intern, Machine Learning Research Auto Apply
Tri · Los Altos · Hybrid · Posted 9 Nov 2026 🇺🇸
RoboticsAutonomous DrivingPyTorch+3Technology$65/hr - TT
Member of Technical Staff, Applied MLRecently posted Auto Apply
True Talent Solutions Partners · New York · On-site · Posted 17 Nov 2026 🇺🇸
Continuous ImprovementLLMPython+1$175k - $350k/yr - TT
Member of Technical Staff, ML SystemsRecently posted Auto Apply
True Talent Solutions Partners · Menlo Park · On-site · Posted 17 Nov 2026 🇺🇸
CUDADeep LearningLLM+2$180k - $260k/yr
