Quick overview
Job description
About the Company
ThirdLaw is building the control layer for AI in the enterprise. As companies rush to adopt LLMs and AI agents, they face new safety, compliance, and operational risks that traditional observability tools were never designed to detect. Metrics like latency or cost don’t capture when a model makes a bad decision, leaks sensitive data, or behaves unpredictably.
We help IT and Security teams answer the foundational question: "Is this OK?"—and take real-time action when it’s not.
Backed by top-tier venture firms and trusted by forward-looking enterprise design partners, we’re building the infrastructure to monitor, evaluate, and control AI behavior in real-world environments—at runtime, where it matters. If you're excited to build systems that help AI work as intended—and stop it when it doesn’t—we’d love to meet you.
About the Role
You won’t just be piping logs or tuning models—you’ll build and scale systems that reconcile latency, correctness, and observability across distributed pipelines. You’ll be designing the nervous system of a new class of software—where AI Agents reason, act, and fail in unpredictable ways. If you're excited by AI and want to shape its safe deployment—not just watch from the sidelines—this is your opportunity.
What You’ll Do
Architect scalable, low-latency services for running evaluations in real-time and batch, integrating with streaming data pipelines and trace-based event models.
Design and build the core evaluation engine within ThirdLaw that applies heuristics, semantic models, and foundation model calls to detect violations across LLM inputs and outputs.
Build a runtime intervention layer to determine and execute appropriate enforcement actions—such as block, redact, notify, escalate—based on evaluation results and risk context.
Create reusable frameworks for pluggable evaluators and intervention policies, supporting no-code authoring and automated deployment pipelines.
Configure and operationalize a vector database pipeline for RAG-like use cases.
Build for scale. Mitigate blocking gRPC threads, implement micro-batching & streaming for LLM/embedding calls and add reliability controls such as queue-based back-pressure and graceful degradation paths.
Who We Are Looking For
Required
5+ years of Backend software engineering experience, including designing and shipping production software services
Strong coding proficiency in Python and/or Go
Deep experience with streaming data pipelines (e.g. Kafka, Pulsar, Redis Streams) and batch processing systems
Proven track record scaling high-QPS, low-latency services (p95/p99 ownership a plus).
Familiarity with vector databases (e.g. FAISS, Weaviate, Qdrant, pgvector) and embedding-based matching
Strong grasp of cloud-native infrastructure: containers, Kubernetes, serverless functions, CI/CD pipelines
Exposure to structured observability patterns, including OpenTelemetry (or similar tracing standards)
Comfortable designing—then defending—trade-offs around build vs buy vs OSS.
Nice-to-Have
Experience with modern Python APIs & concurrency, e.g. gRPC, FastAPI, asyncio, multithreading/processes
Familiarity with ClickHouse, Apache Arrow, or fast analytical storage engines
Prior work on agent frameworks (e.g. LangChain, CrewAI, AutoGen) or LLM orchestration
Experience in trust & safety, compliance, or AI safety domains
Hands-on experience with secure enterprise integrations (authorization/authentication, webhooks, SIEM, IAM)
Why Apply?
Our team is small and focused, valuing autonomy and real impact over titles and management. We need strong technical skills, a proactive mindset, and clear written communication, as much of our work is asynchronous. If you're organized, take initiative, and want to work closely with customers to shape our products, you'll fit in well here.
Finally, we pay market cash compensation and generally above-market equity. The compensation package for this role is benchmarked using Carta Total Compensation and reflects real-time market data for our company’s size, this role’s level, and your geographic location. We have well-designed and generous benefits.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- SP

Backend Software Engineer, GNC (Starlink)Recently posted Auto Apply
SpaceX · Redmond · Hybrid · Posted 19 Nov 2026 🇺🇸
DockerCockroachDBCompliance+7Technology$125k - $165k/yr - CL

Founding AI / Backend EngineerRecently posted Auto Apply
Clera · New York · Remote · Posted 19 Nov 2026 🇺🇸
FirebaseGoogle CloudLLM+2Technology$180k/yr - EN

Backend Software Engineer, PaymentsRecently posted Auto Apply
ERGO NEXT Insurance · Boston · Hybrid · Posted 18 Nov 2026 🇺🇸
SQLAWSJava+3Technology$163k/yr - IO

Staff Backend Software Engineer, Cloud PlatformRecently posted Auto Apply
IonQ · Bothell · Hybrid · Posted 18 Nov 2026 🇺🇸
GCPSQLAWS+10Technology$152k - $228k/yr - GL
Senior Backend Software Engineer - Loki | US | RemoteRecently posted Auto Apply
Grafana Labs · United States (Remote) · Remote · Posted 18 Nov 2026 🇺🇸
GCPMicroservicesRust+8Technology$154k - $185k/yr - EL

Senior / Staff Backend Software EngineerRecently posted Auto Apply
Endor Labs · Palo Alto · Hybrid · Posted 18 Nov 2026 🇺🇸
DockerMicroservicesMongoDB+9Technology$200k - $250k/yr - CO

Senior Staff Backend Engineer (Pricing)Recently posted Auto Apply
Coupang · Seattle · Hybrid · Posted 16 Nov 2026 🇺🇸
DockerSpringSpring Boot+6Technology$174k - $299k/yr - CO

Senior Staff Backend Engineer (Pricing)Recently posted Auto Apply
Coupang · Mountain View · Hybrid · Posted 16 Nov 2026 🇺🇸
DockerSpringSpring Boot+6Technology$174k - $299k/yr - SI

Founding Backend Software Engineer (Senior/Staff) - VC Backed StartupsRecently posted Auto Apply
Signalfire · Remote - No Preference · Remote · Posted 17 Nov 2026 🇺🇸
Talent AcquisitionDjangoDocker+31TechnologySalary not listed - SI

Founding Backend Software Engineer (Senior/Staff) - VC Backed StartupsRecently posted Auto Apply
Signalfire · NYC - Hybrid · Hybrid · Posted 17 Nov 2026 🇺🇸
Talent AcquisitionDjangoDocker+31TechnologySalary not listed - SI

Founding Backend Software Engineer (Senior/Staff) - VC Backed StartupsRecently posted Auto Apply
Signalfire · Bay Area - Hybrid · Hybrid · Posted 17 Nov 2026 🇺🇸
Talent AcquisitionDjangoDocker+31TechnologySalary not listed - SI

Staff Backend Software Engineer - VC Backed StartupsRecently posted Auto Apply
Signalfire · Remote - No Preference · Remote · Posted 17 Nov 2026 🇺🇸
Talent AcquisitionDjangoDocker+31TechnologySalary not listed
