Why this role stands out
This role offers a unique opportunity to directly impact AI inference performance across diverse hardware architectures, fostering significant technical growth. You will thrive here if you are passionate about low-level optimization and enjoy solving complex challenges at the intersection of hardware and software. Apply to shape the future of AI infrastructure with a cutting-edge company.
Quick overview
Job description
About us
Gimlet is building the first multi-silicon neocloud designed for fast, efficient AI inference.
We combine large-scale compute infrastructure with an execution platform that partitions AI workloads and maps each stage to the hardware best suited to run it.
We work with foundation labs, hyperscalers, and AI-native companies, giving our team access to technical problems spanning frontier models, production infrastructure, and emerging hardware.
About the role
As a Member of Technical Staff, you will build and optimize the low-level execution primitives that turn accelerator performance into production inference performance.
Rather than optimizing for one hardware architecture, you will work across accelerators with different execution models, memory hierarchies, capabilities, and software stacks. Your work will shape the latency, throughput, and efficiency Gimlet can achieve across established and emerging hardware architectures.
You will work close to the hardware across kernel implementation, memory access, execution behavior, profiling, and performance validation. You will develop optimizations that account for differences between accelerator architectures and partner with compiler, ML systems, and distributed systems engineers to improve performance across the full execution stack.
What success looks like
In the first 12-18 months, you will:
Build and optimize kernels that improve latency, throughput, and hardware utilization for production AI workloads
Develop execution strategies that unlock performance across both established and emerging accelerator architectures
Improve memory efficiency, scheduling behavior, and execution characteristics across the inference stack
Partner with compiler, runtime, and distributed systems engineers to ensure end-to-end performance optimization
Influence how heterogeneous hardware is deployed and utilized within the next generation of AI infrastructure
Help establish performance engineering standards that shape the future of Gimlet's execution platform
You may be a good fit if
Strong software engineering fundamentals
Experience working on performance-critical systems close to hardware
Comfort reasoning about low-level execution behavior, memory hierarchies, and performance tradeoffs
Bachelor's degree in a relevant field, or an equivalent combination of education, training, and professional experience.
Strong candidates may also have
Experience with CUDA, Triton, CUTLASS, or other accelerator programming models
Deep understanding of GPU execution models (warps/wavefronts, blocks, grids)
Experience optimizing memory access patterns (coalescing, shared memory, cache behavior)
Familiarity with occupancy, latency hiding, and instruction-level parallelism
Experience using profiling and performance analysis tools
Familiarity with multi-GPU or distributed execution is a plus
Why join now?
Gimlet is expanding from its core technology into a production neocloud spanning new hardware, customers, and data centers.
Solve hard problems.
Own meaningful work.
Build for production.
Help define what’s next.
Agency Policy: Gimlet Labs does not accept unsolicited resumes from recruitment agencies or search firms. Any unsolicited resumes submitted without a signed agreement will be considered the property of Gimlet Labs, and no fees will be paid.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Similar jobs
- PI

Engineering InternRecently posted Auto Apply
Parabola Io · San Francisco · On-site · Posted 17 Nov 2026 🇺🇸
EngineeringSalary not listed - HE
Associate Engineer (New Grad)Recently posted Auto Apply
Helicon · Los Angeles (Torrance) · On-site · Posted 17 Nov 2026 🇺🇸
OnboardingRoboticsCAD+2EngineeringSalary not listed - HE
Responsible EngineerRecently posted Auto Apply
Helicon · Los Angeles (Torrance) · On-site · Posted 17 Nov 2026 🇺🇸
OnboardingRoboticsCAD+2EngineeringSalary not listed - HE
Founding Engineer, CAD & SimulationRecently posted Auto Apply
Helicon · Los Angeles (Torrance) · On-site · Posted 17 Nov 2026 🇺🇸
RoboticsCADCFD+3EngineeringSalary not listed - HE
Founding Software EngineerRecently posted Auto Apply
Helicon · Los Angeles (Torrance) · On-site · Posted 17 Nov 2026 🇺🇸
RoboticsPythonTypeScriptTechnologySalary not listed - CL

Software Engineer, Admin DashboardRecently posted Auto Apply
Clever · San Francisco · Hybrid · Posted 18 Nov 2026 🇺🇸
DynamoDBMicroservicesMongoDB+6Technology$115k - $135k/yr - CR

Systems Reliability Engineer Auto Apply
CFD Research Corporation · Huntsville · Hybrid · Posted 10 Nov 2026 🇺🇸
AssemblyCFDCompliance+1TechnologySalary not listed - AO
Senior Software Engineer, Vehicle Platform IntegrationsRecently posted Auto Apply
Aurora Operations Inc · Pittsburgh · Hybrid · Posted 17 Nov 2026 🇺🇸
RoboticsC++LinuxTechnology$146k - $234k/yr - TE

Senior Software Engineer, ReliabilityRecently posted Auto Apply
Temporal · United States (Remote) · Remote · Posted 17 Nov 2026 🇺🇸
Capacity PlanningLLMTriage+1TechnologySalary not listed - MG
Senior Java Engineer - Order Management SystemRecently posted Auto Apply
Man Group · Massachusetts · On-site · Posted 17 Nov 2026 🇺🇸
DockerMicroservicesMongoDB+31Technology$150k - $170k/yr - UP

Software Engineer II, Marketplace OptimizationRecently posted Auto Apply
Upstart · United States | Remote · Remote · Posted 17 Nov 2026 🇺🇸
RubyKotlinPython+4Technology$142k/yr - PG

Software Engineer II Platform Data ReliabilityRecently posted Auto Apply
PlayStation Global · United States · Hybrid · Posted 17 Nov 2026 🇺🇸
DynamoDBGCPAWS+7Technology$150.1k - $225.1k/yr
