Quick overview
Job description
About the Role
We're looking for an engineer to own how we know whether our robots are actually getting better. As we move from research to production, we need evaluation that's rigorous enough to trust and fast enough to keep up. You'll build the harnesses, task suites, and regression infrastructure that turn "the demo looked good" into a number we can act on.
What You'll Do
Own the evaluation strategy for our policies in simulation and on real hardware.
Make evaluation results trustworthy and reproducible; the same evaluation, run by different people on different days, gives the same answer.
Ensure a checkpoint that improves one behavior can't silently break another, and that releases are gated on it.
Establish sim-to-real correlation, so we know which simulation results we can trust.
Own the internal benchmark our models are measured against, and keep results comparable as it evolves.
Know where we stand against the state of the art, benchmarked against external baselines and published results.
Make evaluation cheap enough to run continuously, without burning robot hours on every checkpoint.
Give the company clear answers to "is this checkpoint better than the last one" and "what is it failing at."
What We're Looking For
Built evaluation, validation, or benchmarking systems that a team made real decisions from.
Fluency with model training and evaluation workflows: comfortable reading a training curve, reasoning about generalization, and telling a real improvement from an overfit one.
Strong software engineering and programming skills.
Practical statistics for experiment design: sample sizing, significance, and knowing when a result is noise.
Demonstrated skepticism: a track record of finding the flaw in a result that everyone else was ready to accept.
Bonus Points
Experience in robotics: manipulation, locomotion, or mobile autonomy.
Experience with simulation environments (Isaac, MuJoCo, Gazebo) and sim-to-real transfer.
Experience evaluating imitation-learned or RL policies specifically.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- TR

Robotics Intern, Generalist Robot Learning (GRL)Recently posted Auto Apply
Tri · Los Altos · Hybrid · Posted 18 Nov 2026 🇺🇸
RoboticsComputer VisionDeep Learning+4$45 - $65/hr - TT
Senior AI/ML EngineerRecently posted Auto Apply
True Talent Solutions Partners · San Francisco · On-site · Posted 18 Nov 2026 🇺🇸
ScalaC++Java+4Technology$180k - $275k/yr - TT
Founding Research Engineer, Robot LearningRecently posted Auto Apply
True Talent Solutions Partners · San Francisco · On-site · Posted 18 Nov 2026 🇺🇸
RoboticsPyTorchPython+1Engineering$260k - $300k/yr - RE

Machine Learning EngineerRecently posted Auto Apply
Reddit · United States · Remote · Posted 18 Nov 2026 🇺🇸
AirflowBigQueryDeep Learning+11Technology$185.8k - $260.1k/yr - AB

Member of Technical Staff, Machine LearningRecently posted Auto Apply
Abridge · SF Office · Hybrid · Posted 18 Nov 2026 🇺🇸
NLPDeep LearningPyTorch+3TechnologySalary not listed - AI

Machine Learning Engineer, Assistance EngineeringRecently posted Auto Apply
Airbnb · United States · Remote · Posted 18 Nov 2026 🇺🇸
MLOpsLLMReact+4Technology$162k - $186k/yr - AI

Senior Machine Learning Engineer, Assistance EngineeringRecently posted Auto Apply
Airbnb · United States · Remote · Posted 18 Nov 2026 🇺🇸
MLOpsLLMReact+4Technology$196k - $227k/yr - TT
Founding ML Research EngineerRecently posted Auto Apply
True Talent Solutions Partners · San Francisco · On-site · Posted 18 Nov 2026 🇺🇸
Computer VisionPyTorchPython+1$200k - $260k/yr - CL

Founding Machine Learning EngineerRecently posted Auto Apply
Clera · San Francisco · On-site · Posted 18 Nov 2026 🇺🇸
SQLScikit-learnForecasting+3Technology$170k - $200k/yr - AT

Senior Manager, Machine Learning & AI – FinanceRecently posted Auto Apply
Atoms · San Francisco · Hybrid · Posted 18 Nov 2026 🇺🇸
DockerFastAPISQL+12Technology$220k - $250k/yr - 1A

Senior/Staff AI/ML Engineer, Plant IntelligenceRecently posted Auto Apply
1 Arc Power · Mountain View · Hybrid · Posted 18 Nov 2026 🇺🇸
SQLLinearPython+2Technology$237k - $257.5k/yr - TT
Member of Technical Staff, ML SystemsRecently posted Auto Apply
True Talent Solutions Partners · Menlo Park · On-site · Posted 18 Nov 2026 🇺🇸
CUDADeep LearningLLM+1$180k - $260k/yr