Machine Learning Engineer - Distributed Training & Research Infrastructure
Quick overview
Job description
We're partnering with a leading global trading firm to hire a Machine Learning Engineer to build the systems their researchers use to train models at scale. You'll own the infrastructure that turns a research idea into a trained, tested model, and you'll be measured on how fast and how cheaply that happens.
What you'll do
- Build and scale distributed training pipelines for large models and complex, high-volume datasets
- Design the parallelism strategy, data loading and checkpointing for multi-node GPU training
- Automate the research loop with quantitative researchers: experiment orchestration, hyperparameter tuning and model retraining
- Work with HPC specialists to raise training throughput and GPU utilization, and to cut compute cost
- Extend the internals of PyTorch and other open-source ML tools when off-the-shelf isn't fast enough
- Evaluate third-party tools and decide when to build versus buy
What you'll bring
- 3+ years in machine learning, focused on training systems
- Hands-on experience scaling training across many GPUs and nodes with PyTorch, JAX or TensorFlow (e.g. FSDP, DeepSpeed, Megatron, DDP)
- A strong grasp of ML fundamentals, including transformer and attention internals, beyond high-level APIs
- Production-quality Python, with testing discipline and ownership of the systems you ship
- A habit of measuring before optimizing: profiling, benchmarking and finding the real bottleneck
- Clear communication, so you can explain your technical decisions to researchers and adapt when the evidence says to
Nice to have
- Experience with HPC schedulers (Slurm) and cloud or orchestration platforms (Kubernetes, Ray)
- Contributions to open-source ML or distributed-systems projects
- Familiarity with NCCL, collective communication or interconnect tuning
No trading experience is required. What matters is depth in large-scale training systems and the ability to move fast among very smart people.
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- TE
Machine Learning EngineerRecently posted
TechDoQuest · Florham Park, New Jersey · Hybrid · Posted 21 Nov 2026 🇺🇸
GCPGoogle CloudPython+3TechnologySalary not listed - FP
Machine Learning Engineer - GPU Performance & Low-Latency InferenceRecently posted
Fintal Partners · New York · Hybrid · Posted 21 Nov 2026 🇺🇸
CUDAC++PyTorch+2TechnologySalary not listed - CO

Sr. Staff ML Infra Engineer - Offline Infra & TrainingRecently posted Auto Apply
Coupang · Mountain View · Hybrid · Posted 21 Nov 2026 🇺🇸
GCPScalaAWS+12Salary not listed - PA

Robotics Machine Learning EngineerRecently posted Auto Apply
Pantograph · San Francisco · On-site · Posted 21 Nov 2026 🇺🇸
RoboticsTechnology$150k - $260k/yr - AS
ML EngineerRecently posted
Apex Systems · Cleveland, OH · Hybrid · Posted 19 Nov 2026 🇺🇸
MLOpsHugging FacePyTorch+5TechnologySalary not listed - AS

Sr Staff Machine Learning Engineer, Adobe Firefly ServicesRecently posted
Adobe Systems · Seattle, WA · Hybrid · Posted 20 Nov 2026 🇺🇸
MLOpsCUDAPyTorch+5Technology$238.7k - $345.6k/yr - BF
AI/ML Engineer/Data Scientist with Security ClearanceRecently posted
BlueWater Federal Solutions, Inc. · Montgomery, AL · Hybrid · Posted 20 Nov 2026 🇺🇸
NLPPythonMachine Learning+1TechnologySalary not listed - AE

Applied Machine Learning EngineerRecently posted
Aerospace Corporation · Chantilly, VA · On-site · Posted 21 Nov 2026 🇺🇸
DockerMicroservicesSQL+17Technology$110.4k - $193.5k/yr - PS
Senior Machine Learning EngineerRecently posted
Precision System Design Inc. · Cleveland, OH · Remote · Posted 20 Nov 2026 🇺🇸
DockerSQLAWS+8TechnologySalary not listed - BA

Data Scientist and AI/ML Engineer with Security ClearanceRecently posted
Booz Allen Hamilton · Fort Belvoir, VA · On-site · Posted 20 Nov 2026 🇺🇸
MySQLSQLGit+11Technology$99k - $225k/yr - CM

Senior Machine Learning Engineer - Secure AI LabRecently posted
Carnegie Mellon University · Pittsburgh, PA · On-site · Posted 20 Nov 2026 🇺🇸
ETLComputer VisionC+++5TechnologySalary not listed - RI

Engineering Intern - ML Compilers (Spring 2027 Co-op)Recently posted
Rivian · Palo Alto, CA · On-site · Posted 21 Nov 2026 🇺🇸
RecruitingSpringBackground Checks+8$45 - $51/hr