Quick overview
Job description
About Vmax
Vmax is an applied research lab developing AI capable of open-ended learning. We are building systems to exceed humans in all capacities by optimising beyond the local maxima of learning from human expertise.
About the role
RL has become the de-facto method of post-training LLMs. We are limited by the sample efficiency of the current policy gradient algorithms in use today, and are looking for a talented researcher to weave together pre-LLM and post-LLM approaches to learning from experience.
Responsibilities
- Develop new RL algorithms for post-training language models.
- Adapt ideas from pre-LLM reinforcement learning, such as model-based RL, temporal abstraction, and value-based learning, to modern LLM and agentic settings.
- Establish empirical baselines and evaluation protocols for measuring sample efficiency, robustness, generalization, and reward exploitation in LLM RL.
- Analyze failure modes of RL-trained models, including reward hacking, mode collapse, over-optimization, exploration failures, and distribution shift.
- Collaborate with researchers working on environments, evals, interpretability, reward modeling, and infrastructure to turn algorithmic ideas into reliable training systems.
- Own and develop a research agenda within Vmax, from identifying promising directions to executing experiments and communicating results.
Minimum Requirements
- PhD or equivalent experience in machine learning, reinforcement learning, or a closely related field.
- Track record of research excellence, as demonstrated by publications, open source work, deployed AI systems, or other substantial technical contributions.
- Deep understanding of modern machine learning, especially reinforcement learning, representation learning, and large language models.
- Strong familiarity with LLM post-training methods.
- Experience designing and running rigorous ML experiments, including ablations, baselines, evaluation design, and failure analysis.
- Experience with large-scale ML infrastructure, distributed training, experiment tracking, data pipelines, and debugging unstable training runs.
- Expertise with Python and at least one major ML framework such as PyTorch or JAX.
- Ability to work independently on open-ended research problems and turn ambiguous ideas into concrete experimental programs.
Nice to have
- Experience developing new RL algorithms or improving existing ones in domains such as robotics, games, simulated control, language models, or agents.
- Experience with LLM pre-training.
- Strong understanding of reward modeling, verifiers, process supervision, outcome supervision, or automated evaluation systems.
- Demonstrated software engineering ability
- Strong communication skills, especially the ability to explain algorithmic ideas, empirical results, and research implications to both technical and non-technical audiences
Role specific location policy
- This role is based in our San Francisco office; for exceptional candidates we are willing to consider a hybrid arrangement
Compensation
The expected salary range for this position is $300,000 - $500,000 USD
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- WA
PhD Drug Discovery Scientist (Pharma / Biotech), AI ExpertRecently posted Auto Apply
Weekday AI · United States · Remote · Posted 19 Nov 2026 🇺🇸
ArticulateGenerative AIHealthcare$70 - $100/hr - HR
Lead Quant Researcher, Event Driven SignalsRecently posted Auto Apply
hermeneutic research · United States · Remote · Posted 19 Nov 2026 🇺🇸
SQLBootstrapDerivatives+4Salary not listed - ON

Applied Scientist, Foundation ModelsRecently posted Auto Apply
Oneapp · United States (Remote) · Remote · Posted 19 Nov 2026 🇺🇸
Talent AcquisitionSalary not listed - AP

Associate Scientist III, BioassaysRecently posted Auto Apply
Arrowhead Pharmaceuticals · Madison · Hybrid · Posted 19 Nov 2026 🇺🇸
$80k - $90k/yr - AP

Scientist I, Quality Control (Validation)Recently posted Auto Apply
Arrowhead Pharmaceuticals · Verona · Hybrid · Posted 19 Nov 2026 🇺🇸
$90k - $105k/yr - AP

Associate Scientist I, Mechanistic PharmacologyRecently posted Auto Apply
Arrowhead Pharmaceuticals · Madison · Hybrid · Posted 18 Nov 2026 🇺🇸
Healthcare$55k - $68k/yr - TT
Physics Expert, Quantum Optics and Atomic PhysicsRecently posted Auto Apply
True Talent Solutions Partners · Remote · Remote · Posted 19 Nov 2026 🇺🇸
Auditing$100 - $170/hr - LA

Principal Scientist - Analytical Development & Quality ControlRecently posted Auto Apply
Landmarkbio · Watertown · On-site · Posted 18 Nov 2026 🇺🇸
ComplianceContinuous Improvement$150k - $180k/yr - MA

Senior / Principal BioProcess ScientistRecently posted Auto Apply
Maverickx · Austin · On-site · Posted 18 Nov 2026 🇺🇸
Root Cause AnalysisSalary not listed - MG
Junior Quant - MBSRecently posted Auto Apply
Man Group · Boston, New York · On-site · Posted 18 Nov 2026 🇺🇸
Onboarding$110k - $130k/yr - SA

Research Scientist - Frontier BenchmarksRecently posted Auto Apply
Snorkel AI · New York City · Remote · Posted 18 Nov 2026 🇺🇸
NLPArticulateLLM+1$200k - $350k/yr - IN
Senior Epidemiologist, Real-World EvidenceRecently posted Auto Apply
Intuitive · Sunnyvale, CA · On-site · Posted 18 Nov 2026 🇺🇸
Data AnalysisLogisticsSalary not listed
