Why this role stands out
You'll have the opportunity to build and scale the core infrastructure for cutting-edge AI models at a rapidly growing startup, working on distributed systems and cloud-native technologies. This role is perfect for an early-career engineer eager to dive into impactful ML infrastructure development and shape the future of intelligent products. Apply now to join a dynamic team and accelerate your career in a critical field.
Quick overview
Job description
Rockstar is recruiting for a fast-growing AI infrastructure startup building the optimal inference allocation engine—model routing, dedicated inference, and post-training, all working together to ensure every request is served by the best model for it, in real time. The client’s mission is to deliver truly serverless intelligence to the AI world. Their routers and models allocate and serve 100B+ tokens/month for frontier AI companies across industries.
About the Client
The client is building the optimal inference allocation engine—model routing, dedicated inference, and post-training, all working together to ensure every request is served by the best model for it, in real time. Their mission is to deliver truly serverless intelligence to the AI world.
Their routers and models allocate and serve 100B+ tokens/month for frontier AI companies across industries.
The Role
The client is hiring a Software Engineer to help design, build, and scale the core systems that power large-scale routing, model training, and deployment.
The engineer will work on everything from distributed training pipelines, cloud-native infrastructure, and internal developer platforms that support fine-tuning, reinforcement learning, and inference at scale, stretching right up to the frontend. This role sits at the intersection of product engineering and ML systems—the engineer will collaborate closely with ML scientists while owning production-grade infrastructure operating at 99.99% uptime, alongside a team that has shipped hundreds of models at places like TikTok, MongoDB, and Neeva.
This is an ideal role for an early-career engineer who wants to work on real distributed systems, GPU workloads, and modern ML infrastructure, and actually see their work transform the AI capabilities of frontier application layer companies.
What You’ll Do
Build & Scale Core Infrastructure
Design and implement systems that support large-scale ML workloads, including fine-tuning and reinforcement learning.
Build distributed training, inference, and logging pipelines that are efficient, fault-tolerant, and observable.
Cloud & Systems Engineering
Work on cloud-native systems using containers and orchestration (e.g., Kubernetes).
Optimize systems for performance, reliability, and cost efficiency, especially for GPU-heavy workloads.
Implement monitoring, logging, and observability for long-running training jobs and production services.
Bring the Client’s Capabilities to Customers
Talk to customers. Join their calls, find what’s slowing them down, and ship the fix yourself.
Realize the client’s features. Develop the frontend and backend logic that enables customers to access the client’s capabilities.
Who You Are
Required
1–3 years of engineering experience, ideally working on production systems on the backend and infrastructure layer.
Strong fundamentals in distributed systems, networking, and backend architecture.
Comfortable writing code with and without AI and taking accountability for your code.
Excited to work on-site in San Francisco with a fast-moving early-stage team.