Quick overview
Job description
About the team
At Microsoft AI, our health team is on a mission to help millions of users better understand and proactively manage their health and wellbeing. We’re responsible for ensuring that Microsoft AI’s models and services are useful, trusted and safe across diverse customer health journeys.
We’re assembling a world-class team of builders with backgrounds in healthcare, big tech, and frontier AI research labs. Our goal is to build transformative products for millions of users across copilot.com and the wider Microsoft ecosystem.
We are looking for engineers with experience designing and building LLM-based products, and taking them through to production. You will play a pivotal role in establishing Copilot as the leader in safe, informative, trustworthy and useful health information. We are specifically looking for engineers with experience designing, building and running evaluations for LLMs. You will be expected to build eval pipelines, curate and synthesise datasets, design automated analyses and explain results to internal stakeholders.
Responsibilities
- Be a key cross-functional team member, and close collaborator to product teams. Make significant contributions from translating research into benchmarks to co-authoring product roadmap.
- Design and build evaluation systems that test LLM capabilities in the healthcare domain, and interpret and communicate results.
- Run experiments to determine how different prompting techniques affect results on internal and industry benchmarks.
- Improve the tooling that is used internally to implement, run and analyze evaluations.
- Design internal benchmarking and regression testing capabilities that capture model accuracy, safety and utility.
Qualifications
Required Qualifications
- Bachelor's or Higher Degree in Computer Science, or related technical discipline AND significant Python programming experience / machine learning research.
- Experience building with and around LLMs including prompt / context and harness engineering.
- Deep expertise in evaluating and understanding the performance of agents.
- Experience collaborating in cross functional teams, working through ambiguity to deliver high quality results.
- Have 0 to 1 experience with a bias towards shipping and learning, while balancing a high-quality bar.
- Proven ability to collaborate and contribute to a positive, inclusive work environment, fostering knowledge sharing and growth within the team.
Preferred qualifications
- Experience in healthcare technology, or experience in the health domain.
- Experience with data engineering - handling text dataset sourcing, curation, and processing tasks at scale.
- Demonstrated written and verbal communication skills with the ability to work closely with cross-functional teams, including product managers, designers, and other engineers.
- Passion for learning new technologies and staying up to date with industry trends, best practices, and emerging technologies and patterns in AI.
Software Engineering IC5 - The typical base pay range for this role across United Kingdom is £ 93,500.00 - £ 161,800.00 per year. Certain roles may be eligible for benefits and other compensation.
Find additional benefits and pay information here:
https://careers.microsoft.com/v2/global/en/corporate-pay/united-kingdom-corporate-pay.html
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your CV get past this employer's screening? Check your CV against this job, free
Similar jobs
- RH
Senior AI EngineerRecently posted Auto Apply
Reset Health · London · Hybrid · Posted 19 Nov 2026 🇬🇧
GitClinical ResearchConfluence+4TechnologySalary not listed - RH
AI Platform & Enablement EngineerRecently posted Auto Apply
Reset Health · London · Hybrid · Posted 19 Nov 2026 🇬🇧
GitAWSMLOps+5Salary not listed - ET
Lead Engineer, Agentic SystemsRecently posted Auto Apply
Ellie Technologies Ltd · London · On-site · Posted 19 Nov 2026 🇬🇧
IoTLLMPayroll+3EngineeringSalary not listed - AC
Applied AI Engineer - Operations Auto Apply
Action1 · United Kingdom · Remote · Posted 29 Oct 2026 🇬🇧
SQLMicrosoft OfficeAWS+14TechnologySalary not listed - DA

AI Forward Deployed EngineerRecently posted Auto Apply
Databricks · London · Hybrid · Posted 18 Nov 2026 🇬🇧
GCPMicrosoft ExcelApache Spark+16Salary not listed - SN

AI Deployment StrategistRecently posted Auto Apply
Snowflake · GB-London · Hybrid · Posted 18 Nov 2026 🇬🇧
SnowflakeProcurementSalary not listed - FO

AI Business Partner (C&E) Auto Apply
Foxway · Braintree · On-site 🇬🇧
PlumbingSalary not listed - SN

Senior AI Systems EngineerRecently posted Auto Apply
Snyk · United Kingdom - London Office · Hybrid · Posted 17 Nov 2026 🇬🇧
DockerGCPMicroservices+8TechnologySalary not listed - KA
Senior Engineer II - EngEx AI EngineerRecently posted Auto Apply
Kaluza · London · Hybrid · Posted 17 Nov 2026 🇬🇧
AWSLLMGenerative AITechnologySalary not listed - TE
AI Transformation Lead - EditorialRecently posted Auto Apply
The Economist Group · London - Commercial · Hybrid · Posted 16 Nov 2026 🇬🇧
Business AnalysisEnterprise SalesInstagramSalary not listed - TE
AI Transformation LeadRecently posted Auto Apply
The Economist Group · London - Commercial · Hybrid · Posted 16 Nov 2026 🇬🇧
Business AnalysisEnterprise SalesInstagramSalary not listed - PE

AI Growth Enablement SpecialistRecently posted Auto Apply
Private Equity Insights · London · Hybrid · Posted 16 Nov 2026 🇬🇧
CopywritingSalary not listed
