Quick overview
Job description
About the Role
You will build the platform our ML team works on. Training runs, datasets, experiments, compute, and the path from a checkpoint to a model running on a robot all sit on infrastructure someone has to own.
The measure of this role is iteration speed. When an ML engineer has an idea, how long until they know whether it worked.
What You'll Do
Own the data platform: storage, versioning, and lineage for large multimodal datasets, so any model can be traced back to the data that produced it.
Build dataset assembly and serving, including the interfaces ML engineers use to slice, filter, and load training data.
Own our ML compute, on-premise and cloud: cluster provisioning, GPU node health, drivers, networking, and storage.
Own scheduling and quota across the team, so compute is allocated sensibly and utilization stays high.
Keep the cluster reliable, including monitoring, failure recovery, and being the person who figures out why a multi-day run died.
Own training orchestration: job scheduling, multi-node runs, checkpointing, and recovery.
Build experiment tracking and comparison, so results are reproducible and a run from six months ago can be rerun.
Own the model registry and the release path onto robots, including versioning, rollout, and rollback.
Own CI and test infrastructure for the ML stack.
Work with ML engineers as internal customers, and treat their iteration speed as the metric you are accountable for.
What We're Looking For
Strong software engineering with depth in distributed systems and data infrastructure. Python, plus Go, Rust, or C++.
Experience building ML platforms that other engineers depended on daily.
Hands-on GPU cluster operations: provisioning, drivers, interconnect, and diagnosing hardware and network failures under load.
Experience with large-scale data infrastructure: object storage, columnar formats, and pipelines handling terabytes or more.
Familiarity with cluster scheduling: Kubernetes, Slurm, Ray, or equivalent.
Track record of reducing the time between an idea and a result.
Pragmatism about building versus adopting, and the judgment to know when a tool is good enough.
Bonus Points
Experience running on-premise GPU clusters, including procurement, datacenter logistics, and capacity planning.
Experience with multimodal data at scale: video, sensor, and time-series.
Experience deploying models to edge or embedded targets, including versioning and rollback on devices in the field.
Experience with data versioning and lineage tooling.
Robotics or autonomous vehicle infrastructure experience.
Compensation: Competitive base + equity
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- OV

Model Behavior EngineerRecently posted Auto Apply
Overtone · New York City · On-site · Posted 20 Nov 2026 🇺🇸
LLMPrompt EngineeringEngineering$160k - $190k/yr - CI

Forward Deployed AI EngineerRecently posted Auto Apply
Circadiahealth · Salt Lake City · Hybrid · Posted 20 Nov 2026 🇺🇸
OnboardingGCPSQL+9Technology$150k - $250k/yr - CE

AI EngineerRecently posted Auto Apply
CelLink Corporation · Georgetown · Hybrid · Posted 20 Nov 2026 🇺🇸
SQLMLOpsScikit-learn+8TechnologySalary not listed - TA

AI Delivery LeadRecently posted Auto Apply
Tribe Ai · United States · Remote · Posted 19 Nov 2026 🇺🇸
KanbanGenerative AIProject ManagementOperations & Project Management$175k - $260k/yr - HR
Lead AI Engineer, LLM SystemsRecently posted Auto Apply
hermeneutic research · United States · Remote · Posted 19 Nov 2026 🇺🇸
Capacity PlanningLLMPython+1TechnologySalary not listed - FO

Principal, AI Transformation – Revenue & GTMRecently posted Auto Apply
Formlabs · Somerville · Hybrid · Posted 19 Nov 2026 🇺🇸
SQLAirflowB2B Sales+16$125k/yr - FO

GTM AI Lead (Revenue & Go-To-Market)Recently posted Auto Apply
Formlabs · Somerville · Hybrid · Posted 19 Nov 2026 🇺🇸
SQLAirflowB2B Sales+16$125k/yr - FO

Head of AI Transformation, Revenue & Go-To-Market (GTM)Recently posted Auto Apply
Formlabs · Somerville · Hybrid · Posted 19 Nov 2026 🇺🇸
SQLAirflowB2B Sales+16$125k/yr - BE

AI Specialist & Executive Program LeadRecently posted Auto Apply
Backblaze External Website · United States · Remote · Posted 19 Nov 2026 🇺🇸
AWSRoboticsSketch+6$108.5k - $147k/yr - 3U
Senior AI Tools EngineerRecently posted Auto Apply
31st Union · San Mateo · Hybrid · Posted 18 Nov 2026 🇺🇸
C++LLMPython+1$112.7k/yr - 2K

Senior AI Tools EngineerRecently posted Auto Apply
2K · San Mateo · Hybrid · Posted 18 Nov 2026 🇺🇸
C++LLMPython+1$112.7k/yr - DA

Staff AI Acceleration EngineerRecently posted Auto Apply
Dave · United States · Remote · Posted 19 Nov 2026 🇺🇸
SnowflakeAirflowCompliance+6Salary not listed