Why this role stands out
This role offers a unique opportunity to significantly impact the future of AI inference by optimizing vLLM for AMD GPUs, with a competitive salary range of USD 200,000 - 400,000. You'll thrive here if you possess deep expertise in AMD GPU performance optimization and a passion for pushing the boundaries of ML kernel performance. Apply now to join a mission-driven team at the forefront of AI acceleration.
Quick overview
Job description
Inferact's mission is to grow vLLM as the world's AI inference engine and accelerate AI progress by making inference cheaper and faster. Founded by the creators and core maintainers of vLLM, we sit at the intersection of models and hardware, a position that took years to build.
About the Role
We're looking for an AMD GPU kernel engineer to make vLLM a first-class inference engine across the AMD accelerator ecosystem. A large part of this role is designing, writing, and optimizing kernels for AMD GPUs in HIP, Triton, CK, and AITER, for attention, GEMM, sampling, KV cache, and communication-heavy operations.
You'll work at the boundary of kernels, compilers, inference systems, and hardware architecture. You'll profile production-relevant models on AMD GPUs, find the bottleneck, and write the kernel that removes it, with clear correctness, latency, and throughput benchmarks. Your work will help make AMD GPU support in vLLM usable, fast, benchmarked, and maintainable.
Inferact has three kernel roles. If your kernel work is mostly on Google TPUs (Pallas, JAX), apply to TPU Kernel Engineering. If it is mostly on AMD GPUs (HIP, CK, AITER), apply to AMD GPU Kernel Engineering. For NVIDIA GPUs and cross-platform kernel work, apply to Kernel Engineering. Apply to one; we route you to the best fit.
Skills and Qualifications
Minimum qualifications:
Bachelor's degree or equivalent experience in computer science, engineering, systems, machine learning, or similar.
Hands-on experience designing and implementing GPU kernels in HIP, Triton, CUDA, CK, or a similar kernel language.
Hands-on experience optimizing AMD GPU workloads using ROCm, HIP, Triton, CK, AITER, or similar AMD ecosystem tools.
Deep understanding of AMD GPU execution, memory behavior, toolchains, kernel performance, and backend-specific performance constraints.
Experience optimizing ML kernels such as attention, GEMM, sampling, KV cache, fused kernels, or communication-heavy kernels.
Strong performance profiling and benchmarking skills, with the ability to use measurements, hardware counters, correctness tests, and reproducible benchmarks to guide optimization work.
Preferred qualifications:
Experience with vLLM, SGLang, TensorRT-LLM, ROCm-based serving, or other LLM inference systems.
Familiarity with batching, KV cache, decoding, serving tradeoffs, and backend performance constraints in production inference systems.
Experience with compiler and kernel technologies such as Triton, MLIR, LLVM, CK, AITER, HIP, or other kernel DSLs and backend libraries.
Knowledge of quantization methods such as INT8, FP8, mixed precision, or AMD hardware-specific numeric formats, including accuracy and performance tradeoffs.
Experience building quantitative models of system performance, such as roofline analysis, simulation, or hardware microbenchmarking, and using them to guide kernel work.
Bonus points if you have:
Contributed to vLLM, ROCm, HIP, Triton, CK, AITER, PyTorch, compiler projects, or other open-source ML infrastructure.
Built AMD GPU benchmarking infrastructure or automated performance regression detection for accelerator workloads.
Worked directly with AMD, accelerator platform teams, or early-access programs to ship backend, compiler, or inference performance improvements.
Representative projects
Write a HIP, CK, or Triton attention or MoE kernel that beats the current default on MI300-class GPUs.
Implement low-latency, high-throughput sampling for large language models on AMD GPUs.
Adapt existing models for low-precision (FP8, MXFP4) inference on AMD GPUs.
Debug kernel performance with rocprof and at the ISA level.
Logistics
Location: This role is based in San Francisco, California. Will consider remote in the US for exceptional candidates.
Compensation: Depending on background, skills, and experience, the expected annual salary range for this position is $200,000 - $400,000 USD + equity.
Visa sponsorship: We sponsor visas on a case-by-case basis.
Benefits: Inferact offers generous health, dental, and vision benefits as well as 401(k) company match.
Inferact, Inc. is an E-Verify company, and you may review the Notice of E-Verify Participation and the Right to Work posters in English and Spanish.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Similar jobs
- TT
Founding Senior EngineerRecently posted Auto Apply
True Talent Solutions Partners · New York · On-site · Posted 16 Nov 2026 🇺🇸
LLMPythonTypeScript+1Engineering$175k - $225k/yr - TT
Robotics Software EngineerRecently posted Auto Apply
True Talent Solutions Partners · Watertown · On-site · Posted 16 Nov 2026 🇺🇸
DockerEmbedded SystemsRobotics+6Technology$130k - $180k/yr - TT
Forward Deployed Infrastructure EngineerRecently posted Auto Apply
True Talent Solutions Partners · San Francisco · On-site · Posted 16 Nov 2026 🇺🇸
DockerGCPAWS+6Technology$200k - $300k/yr - TT
Senior/Staff Software EngineerRecently posted Auto Apply
True Talent Solutions Partners · New York · On-site · Posted 16 Nov 2026 🇺🇸
RustAWSERP+5Technology$180k - $270k/yr - TT
Senior Forward Deployed Engineer, OperateRecently posted Auto Apply
True Talent Solutions Partners · New York · On-site · Posted 16 Nov 2026 🇺🇸
GCPNext.jsAWS+15Engineering$170k - $205k/yr - TT
Senior Software Engineer, Distributed Systems (Core)Recently posted Auto Apply
True Talent Solutions Partners · Remote · Remote · Posted 16 Nov 2026 🇺🇸
DockerSQLFlink+2Technology$180k - $320k/yr - TT
Software Engineer, C++ SystemsRecently posted Auto Apply
True Talent Solutions Partners · San Francisco · On-site · Posted 15 Nov 2026 🇺🇸
RustC++Kubernetes+1Technology$250k - $325k/yr - TT
Robotics Infrastructure EngineerRecently posted Auto Apply
True Talent Solutions Partners · Pittsburgh · On-site · Posted 15 Nov 2026 🇺🇸
DockerGCPAWS+5Technology$100k - $300k/yr - TT
Software Engineer, iOSRecently posted Auto Apply
True Talent Solutions Partners · New York · On-site · Posted 15 Nov 2026 🇺🇸
SwiftiOSRESTTechnology$130k - $230k/yr - TT
Founding Product EngineerRecently posted Auto Apply
True Talent Solutions Partners · New York · On-site · Posted 15 Nov 2026 🇺🇸
Next.jsHTMLFigma+4$160k - $220k/yr - TT
Founding EngineerRecently posted Auto Apply
True Talent Solutions Partners · San Francisco · On-site · Posted 15 Nov 2026 🇺🇸
OnboardingAWSLLM+4Engineering$180k - $220k/yr - TT
GTM EngineerRecently posted Auto Apply
True Talent Solutions Partners · San Francisco · On-site · Posted 15 Nov 2026 🇺🇸
Generative AIn8nEngineering$140k - $180k/yr
