Backend Software Engineer (ML Infra)

Rockstar is recruiting for a fast-growing AI infrastructure startup building the optimal inference allocation engine—model routing, dedicated inference, and post-training, all working together to ensure every request is served by the best model for it, in real time. The client’s mission is to deliver truly serverless intelligence to the AI world. Their routers and models allocate and serve 100B+ tokens/month for frontier AI companies across industries.

About the Client

The client is building the optimal inference allocation engine—model routing, dedicated inference, and post-training, all working together to ensure every request is served by the best model for it, in real time. Their mission is to deliver truly serverless intelligence to the AI world.

Their routers and models allocate and serve 100B+ tokens/month for frontier AI companies across industries.

The Role

The client is hiring a Software Engineer to help design, build, and scale the core systems that power large-scale routing, model training, and deployment.

The engineer will work on everything from distributed training pipelines, cloud-native infrastructure, and internal developer platforms that support fine-tuning, reinforcement learning, and inference at scale, stretching right up to the frontend. This role sits at the intersection of product engineering and ML systems—the engineer will collaborate closely with ML scientists while owning production-grade infrastructure operating at 99.99% uptime, alongside a team that has shipped hundreds of models at places like TikTok, MongoDB, and Neeva.

This is an ideal role for an early-career engineer who wants to work on real distributed systems, GPU workloads, and modern ML infrastructure, and actually see their work transform the AI capabilities of frontier application layer companies.

What You’ll Do

Build & Scale Core Infrastructure

Design and implement systems that support large-scale ML workloads, including fine-tuning and reinforcement learning.

Build distributed training, inference, and logging pipelines that are efficient, fault-tolerant, and observable.

Cloud & Systems Engineering

Work on cloud-native systems using containers and orchestration (e.g., Kubernetes).

Optimize systems for performance, reliability, and cost efficiency, especially for GPU-heavy workloads.

Implement monitoring, logging, and observability for long-running training jobs and production services.

Bring the Client’s Capabilities to Customers

Talk to customers. Join their calls, find what’s slowing them down, and ship the fix yourself.

Realize the client’s features. Develop the frontend and backend logic that enables customers to access the client’s capabilities.

Who You Are

Required

1–3 years of engineering experience, ideally working on production systems on the backend and infrastructure layer.

Strong fundamentals in distributed systems, networking, and backend architecture.

Comfortable writing code with and without AI and taking accountability for your code.

Excited to work on-site in San Francisco with a fast-moving early-stage team.