Quick overview
Job description
About the Role
Join a small AI infrastructure team working to make model inference faster and more energy and cost efficient. You will optimize the systems that serve open-source language models across heterogeneous hardware, with ownership spanning kernel performance through production operations.
What You'll Do
Ship support for new open-source models, tuning them for latency and throughput.
Optimize inference serving with techniques such as batching, KV caching, speculative decoding, and quantization.
Write and tune GPU kernels using CUDA, HIP, and Triton for a range of accelerator platforms.
Design, deploy, and operate heterogeneous clusters across hardware vendors.
Run production inference systems, improving reliability, observability, and cost per token.
Work directly with customers as part of solving infrastructure and performance problems.
What We're Looking For
Typically 1 to 6 years of experience in backend, infrastructure, systems, or low-level engineering, including experience at scale or in a fast-moving startup.
Strong understanding of computer architecture, memory hierarchies, and operating system internals.
Experience with production infrastructure such as data centers, clusters, or large GPU fleets.
Familiarity with machine learning inference optimization, including quantization, batching, KV caching, or speculative decoding.
Experience with CUDA, Triton, or other GPU programming, and familiarity with Python, HIP, or accelerator platforms such as TPU or AWS Trainium.
Evidence of strong technical ownership, rapid learning, or exceptional performance in demanding engineering work; strong computer science fundamentals are valued.
Comfort using AI agents and coding tools to improve engineering workflows, and communicating with customers.
Compensation & Benefits
Base salary: $200,000 to $300,000 annually. Visa sponsorship is available.
Location
On-site in San Francisco, California, five days per week.