Quick overview
Job description
About the Role
Lead platform engineering for a data and infrastructure team building a foundational platform that turns heterogeneous laboratory data into a unified, queryable knowledge base. You will own its architecture, security, and scale, helping scientists and AI agents work with reliable, well-governed data.
What You'll Do
Set the architectural direction and build production ingestion pipelines, APIs, microservices, and storage layers that normalize diverse scientific data while preserving provenance.
Operate knowledge graphs and vector search systems to support hybrid symbolic and semantic retrieval.
Build high-concurrency services in Go and Python, including controlled agent write boundaries and isolated execution environments.
Design service authentication, tenant isolation, secrets management, encryption, and other zero-trust security controls.
Own Kubernetes infrastructure, infrastructure as code, CI/CD, and observability, and establish SLOs and internal developer pathways.
Set engineering standards through architecture, code reviews, mentorship, and support for building the platform team.
What We're Looking For
At least 5 years of platform or infrastructure engineering experience, with a track record building and operating production backend infrastructure, data platforms, or distributed systems.
Strong experience with Kubernetes, Terraform, Ansible, and modern CI/CD systems.
Experience architecting ingestion for messy, multi-source data, including schema harmonization and entity resolution.
Proficiency with Go and Python or FastAPI, plus production data systems such as knowledge graphs or vector search.
Experience designing production security boundaries and improving early-stage infrastructure and engineering practices.
Background in computational biology, biotech data systems, or SOC 2 and similar compliance work is beneficial.
Must be authorized to work in the United States without visa sponsorship or transfer.
Compensation & Benefits
Base salary is $220,000 to $300,000 USD annually. Visa sponsorship is not available.
Location
On-site, five days per week in San Francisco, California. Relocation assistance is available.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works