Back to Jobs
Recently postedFull-timeTechnology

Full-Stack Engineer, Agent Evaluation & Reliability

True Talent Solutions PartnersSan Francisco🇺🇸United States

Quick overview

Salary$120k - $180k/yr
Employment typeFull-time
Work modeOn-site
LocationSan Francisco, United States
OAuthLLMNode.jsPostgreSQLTypeScript

Job description

About the Role

Join an AI team building the backend infrastructure and product experiences behind agent evaluation and reliability tools. This backend-leaning full-stack role offers ownership of architecture and product decisions while turning complex systems into useful capabilities for enterprise customers.

What You'll Do

  • Build and maintain backend systems for agent evaluation and continuous improvement.

  • Design data models for evaluation datasets, experiments, agent traces, knowledge versions, findings, and customer-defined quality criteria.

  • Develop and operate enterprise integrations, including authentication, OAuth, webhooks, incremental syncs, backfills, rate-limit handling, retries, and monitoring.

  • Take features from technical design and implementation through deployment, monitoring, customer feedback, and iteration.

  • Work with customers to understand their systems, diagnose issues, and turn specific needs into reusable product capabilities.

  • Ship full-stack product features that make underlying systems accessible and useful.

What We're Looking For

  • At least 2 years of experience as a full-stack or backend engineer building and shipping production systems.

  • Production experience with TypeScript and Node.js or Bun, PostgreSQL, and job queues or worker systems.

  • Experience at a fast-paced startup, and ownership of projects from technical design through deployment and iteration.

  • Experience building enterprise integrations, including OAuth flows, webhooks, incremental syncs, backfills, and rate-limit handling.

  • Hands-on experience with agent infrastructure, LLM evaluation, tracing, or agent memory systems is valuable.

  • Early-stage engineering experience, including owning backend systems end-to-end, and AI side projects, open-source contributions, or hackathon participation are also relevant.

Compensation & Benefits

Base salary: $120,000 to $180,000 annually. Visa sponsorship is available.

Location

On-site in San Francisco, California.

Haystack's AI agent can fill in and submit this application for you. How AI auto apply works

Will your resume get past this employer's screening? Check your resume against this job, free