Quick overview
Job description
About the Role
Build reinforcement learning environments and evaluations for clinical reasoning models at a healthtech company developing decision-support tools for clinicians. You will own the work from task design through delivery to model-training teams, helping make model outputs more reliable in healthcare.
What You'll Do
Design multi-step clinical reasoning tasks with tool use, state transitions, and measurable success criteria.
Create and implement RL environments, including task data, reward functions, grading rubrics, and LLM judges.
Test evaluations for reward hacking, misleading scores, and other failure modes.
Run post-training and agent evaluation experiments, then use observed failures to improve environments.
Partner with clinicians and research colleagues to validate model outputs and deliver environments to training teams.
What We're Looking For
A PhD in computer science, machine learning, or a related field, or at least 3 years of full-time RL research or post-training experience; candidates with 3 or more years in research or industry, or who are in the final year of a PhD or a postdoc, are also considered.
At least a bachelor's degree in computer science, machine learning, mathematics, or a related technical field.
Hands-on experience building RL environments for post-training or agent evaluation, along with experience owning reward functions, grading rubrics, or LLM-judge calibration.
Production-level Python, PyTorch, and Transformers ecosystem skills, with experience in LLMs, reinforcement learning, post-training, evaluation, or reward modeling.
Experience in a full-time role at a leading AI research lab or advanced AI company.
Clear technical writing, comfort with uncertainty and cross-disciplinary work, and a habit of validating plausible model outputs.
Clinical AI experience is useful, as is knowledge of information retrieval, constraint solving, SAT/SMT, or domain-specific languages.
Authorization to work remotely in the United States or Canada, and willingness to attend in-person offsites several times per year.
Compensation & Benefits
Base salary: $250,000 to $400,000 annually. Visa sponsorship is not available.
Location
Remote, with a primary location in Montreal, Quebec, Canada.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Similar jobs
- NB
Senior AI Programmer | Programmeur en IA seniorRecently posted Auto Apply
NBCUniversal · Montreal, QUEBEC · Hybrid · Posted 13 Nov 2026 🇨🇦
RecruitingGitC+++2Salary not listed - PF
Scientifique IA/ML | Scientist, AI/MLRecently posted Auto Apply
Power Factors · Montreal, Quebec · Hybrid · Posted 13 Nov 2026 🇨🇦
DroneSCADALLM+5TechnologySalary not listed - HE

Staff Machine Learning Engineer, AdTechRecently posted Auto Apply
HelloFresh · Toronto · Hybrid · Posted 11 Nov 2026 🇨🇦
DockerFastAPISQL+16TechnologySalary not listed - NA

Director, Machine LearningRecently posted Auto Apply
Narvar · Remote - Canada · Remote · Posted 10 Nov 2026 🇨🇦
GCPAirflowCAD+4TechnologyC$240k/yr - UR
AI/ML Engineer Auto Apply
Urban Ridge Supplies · Toronto · On-site 🇨🇦
DockerAWSETL+12TechnologySalary not listed - UR
Forward Deployed Engineer (Data, ML & AI) Auto Apply
Urban Ridge Supplies · Toronto · On-site 🇨🇦
DockerRustSQL+29C$160k - C$180k/yr - UR
AI Inference Engineer Auto Apply
Urban Ridge Supplies · Toronto · On-site 🇨🇦
DockerContinuous ImprovementCustomer Success+3C$165k - C$330k/yr - UR
Lead, AI/Machine Learning Engineer Auto Apply
Urban Ridge Supplies · Toronto · On-site 🇨🇦
RecruitingWorkdayDocker+21TechnologyC$86k - C$130k/yr - UR
Machine Learning Engineer, Lyft Business & Ads Auto Apply
Urban Ridge Supplies · Toronto · On-site 🇨🇦
AWSCADLLM+7TechnologyC$148.5k/yr - UR
Machine Learning Engineer Auto Apply
Urban Ridge Supplies · Toronto · On-site 🇨🇦
GitMLOpsMLflow+6TechnologyC$210k - C$224k/yr - WS
Senior Machine Learning Engineer Auto Apply
Wiser Solutions · Toronto, ON · Remote · Posted 6 Nov 2026 🇨🇦
CADGenerative AITechnologyC$160k - C$200k/yr - SA

Staff Machine Learning Engineer - Edge AI Auto Apply
Samsara · Remote - Canada · Remote · Posted 6 Nov 2026 🇨🇦
RustCADComputer Vision+7TechnologyC$170.4k/yr