Quick overview
Job description
About the Role
Join a fast-moving, pre-seed-backed AI startup building the next generation of agentic systems that automate complex, multi-step workflows across regulated and enterprise domains — including healthcare, legal, fintech, logistics, and compliance. As a mid-level AI Engineer on the core product team, you'll own production LLM-based services end-to-end, collaborate closely with founders and product, and ship features that deliver measurable impact for real users.
What You'll Do
Design, build, and maintain agentic systems that automate complex, multi-step workflows across regulated industries.
Own production retrieval-augmented generation (RAG) pipelines and retrieval infrastructure, including vector databases, embeddings, and indexing for domain-specific search at scale.
Implement multi-agent orchestration, tool-calling, memory, and reasoning components to deliver robust AI-driven user experiences.
Develop evaluation and safety infrastructure to measure model performance, surface regressions, and enforce enterprise-level trust and reliability.
Ship full-stack AI products from MVP to enterprise-grade — designing APIs and data models, writing frontend and backend code, and operating production systems with CI/CD, monitoring, and testing.
Collaborate with founders, product, and design to prioritize work, define success metrics, and iterate based on user feedback and telemetry.
What We're Looking For
Must-haves:
2–8 years of software engineering experience with demonstrated delivery of shipped user-facing or backend products.
Practical experience deploying LLMs or LLM-based services in production, including prompt design, orchestration, and tool integration.
Proficiency across the stack: Python plus TypeScript/React (or equivalent), cloud platforms (AWS or GCP), and relational or NoSQL databases.
Working knowledge of RAG patterns, vector databases, embeddings, and retrieval pipelines, with sound judgment on when to apply each approach.
Experience building automated tests, evaluations, and monitoring for AI systems to ensure reliability beyond demos.
Experience designing API-driven, high-throughput systems and real-time product features.
Nice-to-haves:
Experience with agent or workflow frameworks (e.g., LangGraph, CrewAI) and orchestration tools (e.g., Temporal, Trigger).
Familiarity with fine-tuning, parameter-efficient tuning, or multi-modal model integration.
Background building multi-tenant or enterprise-ready systems, or prior experience in regulated industries such as healthcare, fintech, or legal.
Compensation & Benefits
Salary: $180,000 – $400,000 USD annually (reflecting a wide band across base and equity depending on experience).
Early-stage equity in a venture-backed startup.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Similar jobs
- AR

Staff AI Systems EngineerRecently posted Auto Apply
Archer · San Jose · Hybrid · Posted 13 Nov 2026 🇺🇸
DockerSQLAWS+5Technology$190k - $237k/yr - SO

Senior AI Quality & Evaluation EngineerRecently posted Auto Apply
Solvd · United States · Remote · Posted 13 Nov 2026 🇺🇸
AWSSOC 2LLM+3Salary not listed - PA

AI Enablement ManagerRecently posted Auto Apply
Patreon · San Francisco · Hybrid · Posted 13 Nov 2026 🇺🇸
WorkdaySQLGoogle Workspace+10Salary not listed - ES

Applied AI EngineerRecently posted Auto Apply
Esrtreit · New York · Hybrid · Posted 12 Nov 2026 🇺🇸
GCPGitAWS+15Technology$135k - $145k/yr - SY

Applied AI EngineerRecently posted Auto Apply
Symmetry · Seattle HQ · On-site · Posted 13 Nov 2026 🇺🇸
FastAPILLMPython+2TechnologySalary not listed - PA

Large Language Model (LLM) Systems ScientistRecently posted Auto Apply
PassiveLogic · Holladay · Hybrid · Posted 12 Nov 2026 🇺🇸
SQLSwiftGit+9Salary not listed - TT
AI Forward Deployed EngineerRecently posted Auto Apply
True Talent Solutions Partners · Washington · On-site · Posted 13 Nov 2026 🇺🇸
LLMPythonTypeScript+2$160k - $250k/yr - GE

Clinical AI Specialist (Remote)Recently posted Auto Apply
Generatorhealth · Remote · Remote · Posted 13 Nov 2026 🇺🇸
AuditingCompliancePharmacy+1Salary not listed - ET

AI Deployment StrategistRecently posted Auto Apply
Eleve Talent · New York City · Hybrid · Posted 12 Nov 2026 🇺🇸
Prompt EngineeringSalary not listed - AP
Associate Director, Nonclinical Writing AI InnovationRecently posted Auto Apply
Alnylam Pharmaceuticals · Cambridge · Hybrid · Posted 10 Nov 2026 🇺🇸
ComplianceLLMProcurement+1$156.5k/yr - AL
SG&A AI Implementation Intern Auto Apply
Astera Labs Early Career - North America 2027 · San Jose · Hybrid · Posted 30 Oct 2026 🇺🇸
ConcreteData PrivacyProcurement+3Technology$45 - $57/hr - AL
Applied AI Engineer Intern (Silicon Engineering) Auto Apply
Astera Labs Early Career - North America 2027 · San Jose · Hybrid · Posted 28 Oct 2026 🇺🇸
CADDeep LearningPython+4Technology$35 - $55/hr
