Haystack
← Back to Jobs
Other
IN

Senior or Staff Applied Scientist

Incedo IncNew York, NY🇺🇸United StatesPosted 21 Aug 2026

Quick Overview

Seniority
Mid Senior
Work mode
Hybrid
Location
New York, NY, United States
Posted
1 week ago
MLOpsMLflowNLPA/B TestingDeep LearningForecastingHugging FaceLLMPyTorchPythonTensorFlow

Job Description

Senior / Staff Applied Scientist (ML)

About the Role

We're looking for a Senior or Staff Applied Scientist to lead the modeling problems that off-the-shelf LLMs can't solve on their own. You'll set the technical direction for custom modeling, fine-tune and adapt foundation models, and define the evaluation methods that tell us whether our AI is actually working driving research and experimentation all the way through to models that ship at scale.

This is a role for a senior scientist who is equally comfortable reading a paper, running rigorous experiments, and shaping production systems. You'll own our hardest, most ambiguous modeling problems end to end, set the scientific bar for the team, and influence the AI strategy. At the Staff level, you'll also drive cross-team technical direction and mentor other scientists and engineers.

What You'll Do

  • Set the technical approach for and lead development of custom models for problems beyond prompting off-the-shelf LLMs personalization, recommendation, ranking, forecasting, classification, and domain-specific tasks.
  • Lead fine-tuning and adaptation of foundation models (full fine-tuning, LoRA/PEFT, instruction tuning, preference optimization/RLHF/DPO, distillation).
  • Define rigorous evaluation research and frameworks offline metrics, human evals, LLM-as-judge, benchmarks, and A/B testing and establish them as org-wide standards for quality, robustness, and bias.
  • Frame ambiguous, high-stakes business problems as tractable modeling problems and choose the right technique for each.
  • Set the standard for principled experimentation: hypotheses, baselines, ablations, and error analysis.
  • Drive the path from prototype to production, partnering with ML/AI engineers to deploy reliably at scale.
  • Investigate model failure modes and lead improvements to accuracy, robustness, calibration, and fairness.
  • Track the research literature, decide what's worth adopting, and translate it into practical, high-impact applications.
  • Influence the AI roadmap and communicate findings and trade-offs to technical leaders and executives.
  • Mentor scientists and engineers, review methodology, and raise the scientific bar across the org. (Staff: drive multi-team technical direction.)

What You'll Bring

Required

  • MS or PhD in CS, ML, statistics, or a related quantitative field or equivalent applied research experience.
  • 6+ years (Senior) / 8+ years (Staff) applying ML to real problems across research and production, or a PhD with substantial applied impact.
  • Track record of owning hard modeling problems end to end and setting scientific direction or methodology adopted by others.
  • Deep understanding of modern ML: deep learning, transformers, and the foundations behind LLMs.
  • Extensive hands-on experience fine-tuning and adapting models (LoRA/PEFT, instruction/preference tuning, distillation).
  • Strong record designing evaluation methodology and reasoning rigorously about metrics, bias, and statistical significance.
  • Expert Python and deep learning frameworks (PyTorch and/or JAX/TensorFlow); strong command of the Hugging Face ecosystem.
  • Exceptional experimentation discipline baselines, ablations, error analysis, reproducibility.
  • Proven ability to translate research into production-grade models with engineering partners.
  • (Staff) Demonstrated technical leadership and influence across multiple teams.

Nice to Have

  • Publications, patents, or notable open-source contributions in ML/NLP.
  • Experience with recommendation, ranking, personalization, or forecasting systems at scale.
  • RLHF/DPO, reward modeling, or alignment research.
  • Distributed/large-scale training and GPU optimization.
  • Experiment tracking and eval tooling (Weights & Biases, MLflow, LangSmith, Ragas).
  • Familiarity with MLOps/LLMOps and model serving.

Similar jobs