Quick overview
Job description
About the Role
We are hiring a Research Engineer to help train our next generation of large language models. You will design, run, and scale training experiments on thousands of GPUs, and turn research ideas into production-grade models.
You will join a small team of researchers and engineers who own the full model lifecycle: data, pre-training, post-training, and evaluation. Research and engineering are not separate here: you will write the code, run the experiments, and shape the conclusions.
Key Responsibilities – What You'll Do
-
Design and run large-scale pre-training and fine-tuning experiments, from small ablations to full training runs.
-
Study scaling laws to choose model size, data mix, and hyperparameters under a fixed compute budget.
-
Improve training efficiency through parallelism strategies (data, tensor, pipeline, FSDP), mixed precision, and custom CUDA or Triton kernels.
-
Diagnose and fix training instabilities such as loss spikes, divergence, and hardware failures.
-
Build and improve data pipelines, including filtering, deduplication, quality scoring, and dataset mixing.
-
Implement ideas from recent research papers and test whether they hold at scale.
-
Build evaluations that measure real model capabilities and track results rigorously.
-
Work closely with the alignment, infrastructure, and product teams, and share findings through clear write-ups.
Required Qualifications – What We're Looking For
-
3+ years of hands-on experience training deep learning models, or a PhD in a related field (computer science, mathematics, or physics).
-
Strong Python skills and deep expertise in PyTorch or JAX.
-
Practical experience with Transformer architectures, including attention, tokenization, embeddings, and positional encodings.
-
Experience with distributed training on multi-node GPU clusters (FSDP, DeepSpeed, Megatron-LM, or similar).
-
Solid foundations in linear algebra, probability, statistics, and optimization.
-
Ability to read a research paper, implement it from scratch, and judge whether it works.
-
Rigorous experimental habits, including clean baselines, ablations, reproducible results, and experiment tracking.
-
Clear written and verbal communication skills, and comfort working in a fast-moving, ambiguous environment.
Preferred Qualifications – Nice to Have
-
Publications at top venues (NeurIPS, ICML, ICLR, ACL) or well-known open-source contributions.
-
Experience pre-training an LLM from scratch, at any scale.
-
Post-training experience, including supervised fine-tuning, RLHF, DPO, or other preference-optimization methods.
-
GPU performance work involving CUDA, Triton, FlashAttention-style kernels, and profiling.
-
Experience with efficient inference, including quantization, distillation, and optimized model serving.
-
Cluster and infrastructure experience with Slurm, Kubernetes, or cloud GPU platforms.
-
Large-scale data engineering experience (Spark, Ray, dataset curation for multilingual or domain-specific corpora).
Please apply only if your experience and technical background align with the required qualifications for this position.
Thank you for your interest!
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- VE
Ingénieur(e) de projets / Project EngineerRecently posted Auto Apply
Veolia Environnement SA · Montréal, QC · Hybrid · Posted 20 Nov 2026 🇨🇦
CADMechanical DesignProject ManagementEngineeringSalary not listed - CI

Senior Transportation EngineerRecently posted Auto Apply
CIMA+ · Calgary, Alberta · Hybrid · Posted 19 Nov 2026 🇨🇦
AutoCADCivil 3DProject Management+1EngineeringSalary not listed - CI

Chargée ou Chargé de projet - Infrastructures municipalesRecently posted Auto Apply
CIMA+ · Québec, Quebec · Hybrid · Posted 19 Nov 2026 🇨🇦
EngineeringSalary not listed - CI

Directrice ou directeur, études de réseaux électriques et interconnexions - SherbrookeRecently posted Auto Apply
CIMA+ · Sherbrooke, Quebec · Hybrid · Posted 18 Nov 2026 🇨🇦
EngineeringSalary not listed - SP
Engineering Manager, Systems EngineeringRecently posted Auto Apply
Sony Pictures Imageworks · Vancouver · Hybrid · Posted 21 Nov 2026 🇨🇦
DockerCADCapacity Planning+7EngineeringC$154.5k - C$193.1k/yr - KE

Founding Engineer Auto Apply
Keplar · Toronto (Canada) · Remote 🇨🇦
Generative AIEngineeringSalary not listed - QU

Senior Manager, Analytics EngineeringRecently posted Auto Apply
Questrade · Toronto · Hybrid · Posted 19 Nov 2026 🇨🇦
Applicant Tracking SystemsSQLGit+9EngineeringC$170k - C$200k/yr - QU

Marketing EngineerRecently posted Auto Apply
Questrade · Toronto · Hybrid · Posted 19 Nov 2026 🇨🇦
Applicant Tracking SystemsGCPMicroservices+18EngineeringC$90k - C$120k/yr - QU

Design EngineerRecently posted Auto Apply
Questrade · Toronto · Hybrid · Posted 19 Nov 2026 🇨🇦
Applicant Tracking SystemsGitHTML+14EngineeringC$90k - C$120k/yr - GD
Lead Electrical EngineerRecently posted Auto Apply
GDKN Corp. · Mississauga, ON · On-site · Posted 20 Nov 2026 🇨🇦
Electrical DesignEngineeringSalary not listed - GD
Sr Project EngineerRecently posted Auto Apply
GDKN Corp. · Mississauga, ON · On-site · Posted 20 Nov 2026 🇨🇦
PMPAgileProject ManagementEngineeringSalary not listed - AE
Senior Geotechnical Engineer - TailingsRecently posted Auto Apply
AECOM · Sudbury, ON · Hybrid · Posted 20 Nov 2026 🇨🇦
EngineeringC$120k - C$200k/yr
