Quick overview
Job description
This is a hands-on engineering role for someone who wants to build real GenAI systems — not prototypes. You'll work across the full lifecycle, from experimentation and prompt/RAG design through production deployment, observability, and iteration, partnering closely with senior engineers, product owners, and technical leadership.
- Build features and components for production-grade GenAI products, with a focus on quality, reliability, and performance.
- Implement and iterate on LLM-based solutions, including retrieval-augmented generation (RAG), prompt engineering, context Engineering, Loop Engineering and agentic workflows.
- Write clean, well-tested, maintainable Python code and contribute to the broader codebase and infrastructure.
- Help build evaluation and monitoring pipelines to measure output quality, groundedness, and reliability of GenAI features.
- Work with foundation models and GenAI frameworks/tooling, evaluating tradeoffs between approaches under guidance from senior engineers.
- Collaborate with Product Owners and cross-functional stakeholders to translate requirements into working technical solutions.
- Participate in code reviews, design discussions, and engineering best practices across the GenAI development lifecycle.
- Debug and resolve technical issues across the stack, escalating and collaborating with senior team members on complex problems.
- (SDE II) Take ownership of medium-complexity features end-to-end, mentor SDE I engineers, and contribute to architecture and design decisions for GenAI components.
- 3+ years of industry experience in software engineering, data science, or machine learning.
- 1+ years of hands-on experience building GenAI/LLM-based features or applications, ideally in a production setting.
- Strong Python skills and experience with modern AI/ML and GenAI frameworks and tooling.
- Working understanding of LLMs, embeddings, RAG, prompt engineering, context Engineering, and GenAI architecture patterns.
- Experience with cloud-native architectures and deploying scalable applications to production.
- Understanding of GenAI evaluation, testing, monitoring, and responsible AI practices.
- Ability to work with moderate autonomy, break down ambiguous problems, and communicate technical tradeoffs clearly.
- Genuine curiosity and enthusiasm for learning new technologies, tools, and techniques in a fast-moving field.
- Bachelor's degree in Computer Science, Software Engineering, Machine Learning, Data Science, or a related technical field.
Additional Qualifications
- knowledge on fine-tuning or adapting LLMs (e.g., SFT, LoRA/PEFT, instruction tuning), including preparing training datasets or evaluating fine-tuned models against baselines is a plus.
- Experience with agentic AI patterns or multi-step LLM workflows is a plus.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- SA

Staff Software EngineerRecently posted Auto Apply
Samsara · Remote - US · Remote · Posted 19 Nov 2026 🇺🇸
GraphQLIoTLLM+2Technology$154.7k/yr - NR

Associate Software Engineer - Java (Distributed Systems and Databases)Recently posted Auto Apply
New Relic · Portland · Hybrid · Posted 19 Nov 2026 🇺🇸
DockerDynamoDBGCP+9Technology$98k - $123k/yr - KO

Software Engineer, Controls (C++)Recently posted Auto Apply
Kodiak · Mountain View · Hybrid · Posted 20 Nov 2026 🇺🇸
MATLABSpringAWS+4Technology$145k/yr - IN

Senior Software EngineerRecently posted Auto Apply
Instawork · San Francisco · Hybrid · Posted 19 Nov 2026 🇺🇸
DjangoRoboticsAngular+8Technology$170k - $200k/yr - ME

Product Engineer, Talent ExperienceRecently posted Auto Apply
Mercor · San Francisco · On-site · Posted 20 Nov 2026 🇺🇸
GitLLMMental Health+3Salary not listed - PA

Product EngineerRecently posted Auto Apply
Paradigmai · San Francisco · On-site · Posted 19 Nov 2026 🇺🇸
ZustandReactTypeScriptSalary not listed - UP

Senior Software EngineerRecently posted Auto Apply
Upstart · United States | Remote · Remote · Posted 19 Nov 2026 🇺🇸
Next.jsPrismaREST+3Technology$164.8k/yr - TO
Senior Software Engineer, Vision AIRecently posted Auto Apply
Toast · Remote · Remote · Posted 19 Nov 2026 🇺🇸
AWSComputer VisionJava+6Technology$125k - $254k/yr - NA

Sr. Software Engineer, BackendRecently posted Auto Apply
Navan · New York · Hybrid · Posted 19 Nov 2026 🇺🇸
MicroservicesJavaKafka+4Technology$113.4k - $252k/yr - RO

Software Engineer, Enterprise BuildRecently posted Auto Apply
Robinhood · Menlo Park · Hybrid · Posted 19 Nov 2026 🇺🇸
GCPSSOLLM+1Technology$166k - $195k/yr - PI

Software Engineer Intern 2027 (USA) Auto Apply
Pinterest · San Francisco · Remote · Posted 6 Nov 2026 🇺🇸
GitC++Java+4Technology$8.3k - $11k/mo - EV

Technical Services Engineer, Portworx (Shift: Monday - Friday from 9:00 AM - 6:00 PM EST)Recently posted Auto Apply
Everpure · Raleigh · Hybrid · Posted 19 Nov 2026 🇺🇸
Engineering$122k/yr