Quick overview
SeniorityMid Senior
Work modeHybrid
LocationUnited States
DockerFastAPISQLGitAWSOAuthGitHub ActionsGoogle CloudGrafanaHelmJWTJavaJenkinsKafkaLLMPrometheusPythonRedisTerraformVaultVercelgRPCGenerative AIRESTAzureReactPostgreSQLKubernetesCI/CDLangChainLangGraphRAG
Job description
We're building an enterprise platform for AI agents. As the Technical Lead, you'll lead development of the platform end to end, guide a team of engineers, and stay hands-on in the code.
You'll work hands-on across every part of the platform:
- Agent harness and SDK: the frameworks and tools engineers use to build and run agents.
- AI infrastructure and governance: secure, policy-controlled access to models, tools, and data.
- Context and memory: getting agents the right information at the right time.
- Evaluation and observability: measuring agent quality, cost, and behavior before and after release.
- AI user experiences: how agents appear in applications, assistants, and partner integrations.
- Applied agents: building production agents on the platform alongside domain teams.
What you'll do
- Own the design, and delivery of the platform, from first design through production operation.
- Write code, review designs and pull requests, and take on the hardest problems.
- Build reusable platform capabilities that other teams can depend on, with clear interfaces, versioning, and documentation.
- Set and uphold engineering standards for quality, security, observability, and evaluation.
- Improve latency, reliability, and cost for production AI workloads.
- Run what you build: define SLOs, take part in on-call, and lead incident reviews.
- Partner with architects, product managers, security, and domain teams.
Minimum qualifications
- Bachelor's degree in computer science or a related field, or equivalent practical experience.
- 8+ years building and running production backend or distributed systems, including 2+ years as a technical lead.
- Hands-on experience shipping LLM-powered applications or agents to production, and handling failure.
- Strong Python , and experience designing frameworks, SDKs, or platform APIs that other engineers build on.
- Solid distributed systems fundamentals: API design, asynchronous and event-driven processing, streaming, caching, rate limiting, and fault tolerance.
- Clear written communication, including design docs that explain trade-offs to both engineers and leadership.
Technical skills
We're looking for experience in most of these and depth in several:- Agent development: tool and function calling, multi-step and multi-agent orchestration, task delegation, durable execution, and human-in-the-loop approvals.
- Agent interoperability: exposing agents and tools to other applications and agents through open protocols.
- Context engineering: retrieval-augmented generation, vector and hybrid search, short- and long-term memory, summarization, and tool selection across large tool catalogs.
- LLM evaluation: eval datasets and rubrics, LLM-as-judge, regression testing in CI, and online evaluation of production traffic.
- LLM infrastructure: routing across model providers, token and cost management, response caching, and latency optimization.
- AI safety and governance: guardrails, prompt-injection defense, PII detection and redaction, policy engines, and sandboxed code execution.
- Security: authentication and authorization, service-to-service identity, least-privilege access, tenant isolation, and audit logging in multi-tenant systems.
- Observability: distributed tracing, metrics, logging, SLOs, and LLM observability.
- AI user experiences: streaming agent output to web clients, and generative or dynamic UI.
- Cloud-native infrastructure: containers, container orchestration, CI/CD, and infrastructure as code.
Languages, frameworks, and technologies
Python is a must. For the rest, we're looking for hands-on experience across most of these categories.- Languages: expert Python; working knowledge of SQL. Java is a plus.
- Agent frameworks: LangGraph and LangChain, or comparable frameworks such as the OpenAI Agents SDK, Google ADK, or Pydantic AI.
- Agent protocols: MCP, A2A, A2UI.
- Models and providers: APIs from major model providers and cloud AI platforms, such as OpenAI, Anthropic, Google Gemini, Amazon Bedrock, and Azure OpenAI; serving open-weight models with vLLM; LLM gateways such as LiteLLM.
- Backend and APIs: FastAPI, asyncio, and Pydantic; REST, gRPC, Server-Sent Events, and WebSockets; event streaming with Kafka or similar.
- Data and retrieval: PostgreSQL with pgvector; search and vector engines such as OpenSearch, Elasticsearch, or Qdrant; Redis; embedding and reranking models.
- Evaluation and observability: OpenTelemetry, including its GenAI semantic conventions; LLM tracing and eval platforms such as LangSmith; eval libraries such as Ragas or DeepEval; Prometheus and Grafana.
- Security and governance: OAuth 2.0, OIDC, and JWT; workload identity such as SPIFFE/SPIRE; policy as code with Open Policy Agent or Cedar; secrets management such as HashiCorp Vault; guardrail and PII tooling such as NeMo Guardrails, Llama Guard, or Presidio.
- Cloud and infrastructure: Docker, Kubernetes, and Helm; Terraform; at least one major cloud (AWS, Azure, or Google Cloud Platform); CI/CD such as GitHub Actions or Jenkins; sandboxed execution with gVisor or Firecracker.
- Front end: React; streaming chat and agent interfaces; generative UI toolkits and specs such as the Vercel AI SDK, A2UI, or MCP-UI.
Preferred qualifications
- Experience building an internal platform, SDK, or framework that multiple engineering teams adopted.
- Experience running AI systems for external enterprise customers.
- Contributions to open-source agent frameworks or protocols.
Similar jobs
- RT
Enterprise Strategy & Operational Resilience ConsultantRecently posted
Random Trees · New York, NY · Hybrid · Posted 18 Nov 2026 🇺🇸
Stakeholder ManagementComplianceRegulatory Compliance+3Salary not listed - NE

Distinguished Engineer- AI (SAN JOSE)Recently posted
NetApp · San Jose, CA · Hybrid · Posted 18 Nov 2026 🇺🇸
MicroservicesMicrosoft ExcelAWS+12$266.1k - $396k/yr - NT
Senior Relationship Advisor
Northern Trust · Atlanta, GA · Hybrid 🇺🇸
ComplianceRisk ManagementBusiness DevelopmentSalary not listed - CF

Lead Product CounselRecently posted Auto Apply
Chime Financial, Inc · San Francisco · Hybrid · Posted 18 Nov 2026 🇺🇸
ComplianceGenerative AI$177k/yr - BR
Maintenance PlannerRecently posted Auto Apply
Braskem · La Porte · Hybrid · Posted 18 Nov 2026 🇺🇸
Talent AcquisitionEstimatingProcurement+1Salary not listed - BR
Data Entry Admin - Learning & DevelopmentRecently posted Auto Apply
Braskem · La Porte · Hybrid · Posted 18 Nov 2026 🇺🇸
Talent AcquisitionMicrosoft ExcelMicrosoft Office+3Salary not listed - MA

Network / Communications Engineer with Security ClearanceRecently posted
MAG Aerospace · Fairfax, VA · Hybrid · Posted 13 Nov 2026 🇺🇸
Edge ComputingEncryptionTCP/IP+4$86.3k - $143.8k/yr - PA

QUALITY SUPPLIER TECHNICIAN III with Security Clearance
Platform Aerospace · Hollywood, MD · On-site 🇺🇸
FiberComplianceProcurement+1Salary not listed - NO

Senior Field MarketerRecently posted Auto Apply
Notion · San Francisco · Hybrid · Posted 18 Nov 2026 🇺🇸
SourcingMarketoNotion+2$135k - $160k/yr - AX

Senior Specialist, Communications & PRRecently posted Auto Apply
Axon · Scottsdale · Hybrid · Posted 18 Nov 2026 🇺🇸
RoboticsBackground ChecksOutreach+1Salary not listed - MA

Lab TechnicianRecently posted Auto Apply
Marianaminerals · San Francisco HQ · On-site · Posted 18 Nov 2026 🇺🇸
SourcingSalary not listed - MC
Co-Founder, Hull.AIRecently posted Auto Apply
Montauk Capital · Remote · Hybrid · Posted 18 Nov 2026 🇺🇸
ComplianceSalary not listed
