Why this role stands out
This Machine Learning Engineer role at Together AI offers an exciting opportunity to work with cutting-edge AI inference systems and contribute to the development of high-performance AI solutions, ideal for mid-senior engineers passionate about PyTorch and scaling large language models. You'll thrive here if you enjoy building robust production systems and collaborating with a talented team to shape the future of AI. Apply now to make a significant impact in a rapidly evolving field.
Quick overview
Job description
About the Role
Together AI is seeking a Machine Learning Engineer to join our Inference Engine team, focusing on optimizing and enhancing the performance of our AI inference systems. This role involves working with state-of-the-art large language models models and ensuring they run efficiently and effectively at scale. If you are passionate about AI inference, PyTorch, and developing high-performance systems, we want to hear from you. This position offers the chance to collaborate closely with AI researchers and engineers to create cutting-edge AI solutions. Join us in shaping the future at Together AI!
Responsibilities
- Design and build the production systems that power the Together AI inference engine, enabling reliability and performance at scale.
- Develop and optimize runtime inference services for large-scale AI applications.
- Collaborate with researchers, engineers, product managers, and designers to bring new features and research capabilities to the world.
- Conduct design and code reviews to ensure high standards of quality.
- Create services, tools, and developer documentation to support the inference engine.
- Implement robust and fault-tolerant systems for data ingestion and processing.
Requirements
- 3+ years of experience writing high-performance, well-tested, production-quality code.
- Proficiency with Python and PyTorch.
- Demonstrated experience in building high performance libraries and tooling.
- Excellent understanding of low-level operating systems concepts including multi-threading, memory management, networking, storage, performance, and scale.
- Preferred: Knowledge of existing AI inference systems such as TGI, vLLM, TensorRT-LLM, Optimum
- Preferred: Knowledge of AI inference techniques such as speculative decoding.
- Preferred: Knowledge of CUDA/Triton programming.
- Nice to have: Knowledge of Rust, Cython and compilers.
About Together AI
Together AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.
Compensation
We offer competitive compensation, startup equity, health insurance, and other competitive benefits. The US base salary range for this full-time position is $200,000 - $300,000 + equity + benefits. Our salary ranges are determined by location, level, and role. Individual compensation will be determined by experience, skills, and job-related knowledge.
Application Policy
Please read each role carefully and apply to the position best suited to you. Candidates are limited to 6 applications within a 90-day period, and applications beyond that limit will be automatically rejected. You may apply to additional roles 90 days after your last application.
Equal Opportunity
Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunities to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.
Please see our privacy policy at https://www.together.ai/privacy
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- TR
Robotics Intern, Generalist Robot Learning (GRL)Recently posted Auto Apply
Tri · Los Altos · Hybrid · Posted 18 Nov 2026 🇺🇸
RoboticsComputer VisionDeep Learning+4$65/hr - TT
Senior AI/ML EngineerRecently posted Auto Apply
True Talent Solutions Partners · San Francisco · On-site · Posted 18 Nov 2026 🇺🇸
ScalaC++Java+4Technology$180k - $275k/yr - TT
Founding Research Engineer, Robot LearningRecently posted Auto Apply
True Talent Solutions Partners · San Francisco · On-site · Posted 18 Nov 2026 🇺🇸
RoboticsPyTorchPython+1Engineering$260k - $300k/yr - RE

Machine Learning EngineerRecently posted Auto Apply
Reddit · Remote - United States · Remote · Posted 18 Nov 2026 🇺🇸
AirflowBigQueryDeep Learning+11Technology$185.8k - $260.1k/yr - AB

Member of Technical Staff, Machine LearningRecently posted Auto Apply
Abridge · SF Office · Hybrid · Posted 18 Nov 2026 🇺🇸
NLPDeep LearningPyTorch+3TechnologySalary not listed - AI

Machine Learning Engineer, Assistance EngineeringRecently posted Auto Apply
Airbnb · Remote-USA · Remote · Posted 18 Nov 2026 🇺🇸
MLOpsLLMReact+4Technology$162k - $186k/yr - AI

Senior Machine Learning Engineer, Assistance EngineeringRecently posted Auto Apply
Airbnb · Remote-USA · Remote · Posted 18 Nov 2026 🇺🇸
MLOpsLLMReact+4Technology$196k - $227k/yr - AD

Senior Machine Learning Engineer, Financial ProductsRecently posted Auto Apply
Adyen · San Francisco · Hybrid · Posted 18 Nov 2026 🇺🇸
DockerSQLMLOps+12Technology$227.5k - $317k/yr - TT
Member of Technical Staff, Applied MLRecently posted Auto Apply
True Talent Solutions Partners · New York · On-site · Posted 18 Nov 2026 🇺🇸
Continuous ImprovementLLMPython+1$175k - $350k/yr - TT
Founding ML Research EngineerRecently posted Auto Apply
True Talent Solutions Partners · San Francisco · On-site · Posted 18 Nov 2026 🇺🇸
Computer VisionPyTorchPython+1$200k - $260k/yr - CL

Founding Machine Learning EngineerRecently posted Auto Apply
Clera · San Francisco · On-site · Posted 18 Nov 2026 🇺🇸
SQLScikit-learnForecasting+3Technology$170k - $200k/yr - AT

Senior Manager, Machine Learning & AI – FinanceRecently posted Auto Apply
Atoms · San Francisco · Hybrid · Posted 18 Nov 2026 🇺🇸
DockerFastAPISQL+12Technology$220k - $250k/yr
