Why this role stands out
Accelerate your career at the forefront of AI innovation with Glean, where you will join a world-class team building agentic solutions that redefine enterprise productivity. You will thrive in this hybrid role if you are a forward-thinking engineer passionate about refining model quality and post-training excellence within a highly collaborative, fast-growing culture. Apply today to shape the future of Work AI and make a measurable impact on a global scale.
Quick overview
Job description
About the Role:
The bottleneck in building reliable and useful AI agents isn't writing code anymore. It's knowing what good looks like, measuring it, and steadily closing the gap. Glean is building the next generation of enterprise AI assistants and autonomous agents that reason, plan, and act across complex, multi-step workflows. This role works at the intersection of applied research, quality, and product engineering to make those systems better.
The focus is on quality improvements across Glean's models and agent harnesses. That means hill-climbing with evals, evolving evalsets and agentic verifiers, post-training models for use cases like model routing, and training task-specific models for enterprise workflows. It also means bringing emerging research into the product and working closely with product teams and customers, turning learnings from evals and production usage into better outcomes through quality improvements.
You will:
- Use eval results and production traces to improve the entire stack, including the agent harness, model routing, prompts, and retrieval, while weighing quality against latency and cost.
- Work on post-training models with supervised fine-tuning (SFT) and preference- or reward-based optimization (e.g., DPO, PPO, GRPO), including generating the data for training.
- Build RL environments for simulating enterprise workflows, including relevant evalsets, harness support, and LLM-based verifiers for hill-climbing and for internal and external benchmarking.
- Improve the quality of representative evalsets, LLM judges and agentic verifiers, and calibrate them against human graders so the scores can be trusted in release decisions and model training.
- Work with product, platform, and research teams across Glean to land quality improvements in production.
About you:
- 5+ years of industry experience in AI or Machine Learning Engineering.
- Hands-on experience post-training LLMs including adapting open-weight models to specialized tasks.
- Experience building evals for AI assistant and agent systems, including generating evalsets, LLM-as-a-judge verifiers, calibrating offline and online signals coupled with good judgment about when a metric reflects quality improvements.
- Strong coding skills in Python and/or Go (TypeScript is a plus), proficiency in your ML framework of choice, and a track record of shipping production ML systems.
- Product-minded, able to turn a vague problem into a concrete experiment, and thrive in a tight-knit, cross-functional team.
- This role is hybrid (4 days a week in our Mountain View or San Francisco office)
By clicking “Submit Application,” I confirm that I have read the Global Data Privacy Notice and the Applicant Arbitration Agreement, and I agree to the terms.
Similar jobs
- MI

Applied AI Engineer, Internal SystemsNew Auto Apply
Minerva · New York City · On-site 🇺🇸
Applicant Tracking SystemsOnboardingGoogle Workspace+10TechnologySalary not listed5 hours ago - LE

AI Automation & Enablement EngineerNew Auto Apply
Legora · New York City · On-site 🇺🇸
Generative AIMavenSalary not listed5 hours ago - BG
Agentic AI R&D InternNew Auto Apply
Bosch Group · Sunnyvale, CA · On-site 🇺🇸
RoboticsGenerative AIEngineeringSalary not listed7 hours ago - AB

Lead AI Automation Engineer Auto Apply
AbbVie · North Chicago, IL · Hybrid 🇺🇸
ComplianceGraphQLLLM+7Technology$109.5k - $208.5k/yr4 days ago - AB

Head of AI Transformation & Analytics, Medical Affairs + Health Impact Auto Apply
AbbVie · Mettawa, IL · On-site 🇺🇸
Generative AIHealthcare$182k - $346k/yr4 weeks ago - AB

Director, Data & AI Architect, Operations and Finance Auto Apply
AbbVie · North Chicago, IL · Hybrid 🇺🇸
SourcingMicroservicesMLOps+6Technology$160.5k - $305k/yr4 weeks ago - AB

Senior Business Systems Engineer, AI Capability Lead - MHI Auto Apply
AbbVie · Mettawa, IL · Hybrid 🇺🇸
ComplianceGenerative AIMachine Learning+4Technology$124.5k - $236.5k/yr4 weeks ago - AB

Principal AI Engineer II Auto Apply
AbbVie · North Chicago, IL · Hybrid 🇺🇸
MicroservicesAWSMLOps+7Technology$141.5k - $268.5k/yr1 month ago - AB

Semantic AI Engineer Auto Apply
AbbVie · North Chicago, IL · Hybrid 🇺🇸
Stakeholder ManagementGenerative AITechnology$78.5k - $141k/yr2 months ago - WO

Senior AI Engineer Auto Apply
Workstream · Lehi · Hybrid 🇺🇸
RecruitingCompliancePayrollTechnologySalary not listed1 week ago - WO

Senior AI Engineer Auto Apply
Workstream · Virginia · Hybrid 🇺🇸
RecruitingCompliancePayrollTechnologySalary not listed1 week ago - TA

Forward Deployed AI EngineerNew Auto Apply
Talkdesk · United States Remote · Remote 🇺🇸
ERPPythonJavaScript+3TechnologySalary not listedYesterday
