Back to Jobs
Recently postedFull timeOther

Member of Technical Staff

True Talent Solutions PartnersSan Francisco🇺🇸United States

Quick overview

Salary$200k - $300k/yr
SeniorityMid Senior
Employment typeFull Time
Work modeOn Site
LocationSan Francisco, United States
AWSCUDA

Job description

About the Role

Join a small AI infrastructure team building efficient inference systems for open-source language models. You will optimize models and serving infrastructure across diverse hardware, helping improve performance, reliability, and cost per token.

What You'll Do

  • Ship support for new open-source models, tuning them for latency and throughput.

  • Optimize inference serving with techniques such as batching, KV caching, speculative decoding, and quantization.

  • Write and tune kernels using CUDA, HIP, and Triton across a range of accelerator platforms.

  • Design, deploy, and operate heterogeneous clusters across infrastructure vendors.

  • Run production inference across a mixed hardware fleet, improving reliability, observability, and cost efficiency.

  • Work with customers to understand technical needs and solve infrastructure problems.

What We're Looking For

  • Typically 1 to 6 years of experience in backend, infrastructure, systems, or low-level engineering, including experience at a technology company or early-stage startup.

  • Strong computer science fundamentals, including computer architecture, memory hierarchy, and operating system internals.

  • Experience owning a challenging system end to end, notable technical achievements, or evidence of rapid growth.

  • Systems engineering experience beyond application-level or full-stack development, with a non-research-focused background.

  • Familiarity with CUDA, Triton, or GPU programming; experience with HIP, TPU, or AWS Trainium is also relevant.

  • Knowledge of inference optimization techniques, such as quantization, batching, KV caching, and speculative decoding.

  • Experience deploying or operating infrastructure at scale, such as clusters or GPU fleets.

  • Experience using AI agents and coding tools to improve engineering workflows, and comfort communicating with customers.

  • Strong internships or equivalent hands-on experience are welcome, including for recent graduates.

Compensation & Benefits

Base salary: $200,000 to $300,000 annually. Visa sponsorship is available.

Location

On-site in San Francisco, California, five days per week.

Haystack's AI agent can fill in and submit this application for you. How AI auto apply works