Quick Overview
Job Description
Software or data engineering, including 3+ years building LLM-based systems
Prior experience building Agent Registry or Catalog.
Production experience with agent-generated code that executes sandboxed execution, automated validation and testing of generated artifacts, and engineer review-and-approve workflows gating deployment.
Built and operated agent evaluation harnesses in production: offline eval suites, regression testing for prompt and model changes, and measurable quality gates that block release on failure.
Operated LLM observability in production: per-run tracing of agent decisions and tool calls, token and cost monitoring, and hands-on triage of agent failures and incidents.
Implemented guardrails and human-in-the-loop controls in a governed environment: approval gates, permission-scoped tool access for agents, audit logging, and rollback procedures.
Has built or deployed MCP servers/clients or OpenAI-compatible tool interfaces in a production system — not just consumed a vendor API.
Strong Python and SQL; CI/CD for data platforms; SSO (SAML/OIDC) and RBAC design.
Designing and operating agentic AI systems in production — systems serving live business users or workloads. Prototypes, pilots, internal demos, and RAG chatbots do not meet this bar.
Served as the lead architect of at least one multi-agent system that has run in production for 12+ months, with direct ownership of supervisory/planner–worker orchestration, tool calling, state and memory management, and workflow error recovery.
Similar jobs
- IC
Principal Agentic AI Architect- 15+ yrs- New York, United States- Onsite
NewiMedhas Consulting Services
New York, NY🇺🇸Hybrid18 hours agoLLMRESTTechnology - IN
AI Engineer with Google''s Agent Development Kit ADK
NewInfiCare
Richardson, TX🇺🇸On-site18 hours agoNestJSNext.jsNode.js+7Technology - CO
Senior AI Engineer
NewCollibra
New York🇺🇸8 hours ago401kSQLMLOps+4Technology - CH
Director, GTM Systems & Applied AI
NewCheckr
San Francisco🇺🇸22 hours agoSalesforceBusiness DevelopmentCRM+6 - CA
Forward Deployed AI Lead
NewCaylent
United States🇺🇸23 hours ago401kAWSBusiness Development+2 - PR
AI Architect
NewProhires
Charlotte, NC🇺🇸Hybrid18 hours agoDockerMicroservicesSQL+13Technology