Quick overview
Job description
About the Role
As a Member of Technical Staff, Model API Infra, you will own the production API and agentic platform for MAI models. Combining strong software engineering with a deep understanding of how models are trained, evaluated, and served, you will build harness infrastructure ensuring training and production parity, automate the model deployment lifecycle, and create cross-stack observability and tooling to run these services reliably at scale. You will collaborate closely with post-training, evaluation, Foundry, Copilot, and product teams, to continuously deliver new model capabilities to production surfaces.
Job Overview
Model API and Agentic Platform
- Design, implement, and operate scalable API services for text, multimodal, and emerging MAI models, with strong compatibility, versioning, reliability, and performance guarantees.
- Build API and systems that maintain consistent model, tool, and harness behavior across training, evaluation, and production serving.
- Partner with Foundry, Copilot platforms, inference, research, eval, and product teams to provide consistent interfaces and enable rapid model iteration and delivery.
- Build business, API, and model system telemetry that enables cross-system debugging, automated investigations, and data-driven insights.
Model Serving Automation
- Build a closed-loop system that automates model deployment and validation.
- Integrate functional, safety, quality, and performance evaluations into deployment decisions; debug eval regressions across the stack, from checkpoint and serving config to API and product.
- Design and build autoscaling for model pools using latency SLOs, utilization, model characteristics, and capacity constraints.
Required Qualifications
- Bachelor's Degree in Computer Science or related technical field AND 5+ years of technical engineering experience with coding in languages including Python, C++, or similar OR equivalent experience.
- Experience designing, building, and operating large-scale production distributed systems, cloud services, or developer-facing APIs.
- Strong systems design and debugging skills, including the ability to investigate failures across service boundaries.
- Experience with deployment automation, orchestration such as Kubernetes, and cloud platforms.
Preferred Qualifications
- Master's Degree AND 5+ years of relevant experience OR equivalent experience.
- Experience building model APIs, AI gateways, agent runtimes, agent frameworks, or tool-calling and long-running execution systems.
- Experience deploying and operating LLMs or multimodal models through platforms such as Microsoft Foundry, Azure Machine Learning, or custom serving infrastructure.
- Familiarity with LLM serving concepts, including streaming inference, token-based capacity, batching, KV cache behavior, model routing, and heterogeneous GPU pools.
- Experience evaluating agents on reasoning, coding, tool use, or long-horizon tasks.
- Familiarity with post-training, evaluation workflows, and open-source ML frameworks
Software Engineering IC4 - The typical base pay range for this role across the U.S. is USD $119,800 - $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $160,200 - $261,000 per year.
Software Engineering IC5 - The typical base pay range for this role across the U.S. is USD $142,800 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- NE

Software Engineer II (Remote in United States) Auto Apply
Nextech · United States · Remote · Posted 23 Oct 2026 🇺🇸
GCPMicroservicesSQL+7TechnologySalary not listed - SL

Founding Engineer (DreamLever) - NYRecently posted Auto Apply
Scope Labs · New York City · Hybrid · Posted 19 Nov 2026 🇺🇸
Engineering$170k - $200k/yr - SE
Software Engineer - Serko.aiRecently posted Auto Apply
Serko Ltd · Seattle, Washington · Remote · Posted 19 Nov 2026 🇺🇸
Next.jsHTMLPython+2Technology$130k - $160k/yr - TT

Systems Reliability EngineerRecently posted Auto Apply
The Trade Desk · Boulder · Hybrid · Posted 18 Nov 2026 🇺🇸
MongoDBRubyRust+11Technology$124.9k/yr - TO
Senior Software Engineer, Care AutomationRecently posted Auto Apply
Toast · Remote · Remote · Posted 18 Nov 2026 🇺🇸
MicroservicesGraphQLJava+2Technology$125k - $254k/yr - RL

Principal Flight Software EngineerRecently posted Auto Apply
Rocket Lab Corporation · Littleton · Hybrid · Posted 19 Nov 2026 🇺🇸
RustEmbedded SystemsBackground Checks+3Technology$150k - $225k/yr - RL

Senior Flight Software Engineer IIRecently posted Auto Apply
Rocket Lab Corporation · Littleton · Hybrid · Posted 19 Nov 2026 🇺🇸
RustEmbedded SystemsBackground Checks+3Technology$131k - $190k/yr - RL

Senior Flight Software Engineer IRecently posted Auto Apply
Rocket Lab Corporation · Littleton · Hybrid · Posted 19 Nov 2026 🇺🇸
RustEmbedded SystemsC+++2Technology$150k - $220k/yr - SM

Staff Software Engineer, BackendRecently posted Auto Apply
Scout Motors · San Ramon · Hybrid · Posted 18 Nov 2026 🇺🇸
MicroservicesComplianceJava+3Technology$165k - $200k/yr - MB

Senior Software Engineer, Lab ApplicationsRecently posted Auto Apply
Manifold Bio · Boston · Hybrid · Posted 18 Nov 2026 🇺🇸
EchoFastAPIAWS+9TechnologySalary not listed - KP

Firmware Development EngineerRecently posted Auto Apply
Kairos Power · Alameda · Hybrid · Posted 19 Nov 2026 🇺🇸
Embedded SystemsFPGAVHDL+1Engineering$92.5k - $108.8k/yr - GU

Staff Software Engineer, Onboarding PlatformRecently posted Auto Apply
Gusto, Inc. · San Francisco · Hybrid · Posted 19 Nov 2026 🇺🇸
OnboardingRubyLLM+4Technology$197k - $247k/yr
