Full timeTechnology

Software Engineer (SE / Sr SE), Data & ML Platform

Plus 2Santa Clara🇺🇸United States

Quick overview

Salary$135k - $200k/yr

SeniorityMid Senior

Employment typeFull Time

Work modeHybrid

LocationSanta Clara, United States

Apache SparkApacheHelmKubernetesDelta Lake

Job description

All key offline workloads — large-scale data processing, simulation, auto-labeling, scenario mining, and model training — run on the compute platform this role owns. In this role, you will improve the reliability and efficiency of our Kubernetes infrastructure, make workload onboarding simpler and more self-service, and build reusable batch and workflow capabilities for petabyte-scale processing. We are looking for strong Kubernetes and platform-engineering fundamentals, depth in at least one adjacent area—distributed data processing, ML/GPU infrastructure, or multi-tenant compute systems—and the curiosity and ownership to grow across the others.

We are open to candidates at either the Software Engineer or Senior Software Engineer level. Level will be determined by experience, technical depth, scope of ownership, and demonstrated impact. You do not need experience with every technology in our stack; we value strong fundamentals, ownership, and the ability to learn.

Responsibilities:

Operate and evolve our production Kubernetes clusters end to end: bare-metal provisioning automation, highly available control planes, node lifecycle, GPU container runtime, networking, and storage
Build safe, repeatable GitOps-based delivery for platform services and user applications using tools such as Argo CD, Helm, and Kustomize
Develop shared multi-tenant platform capabilities for scheduling, resource isolation, storage, networking, access control, secrets, and observability while improving CPU/GPU utilization and cost efficiency
Build and improve reusable distributed batch and workflow platforms for Spark data processing and GPU-based replay and simulation
Ensure that your work is performed in accordance with the company’s Quality Management System (QMS) requirements and contribute to continuous improvement efforts

Required Skills:

BS, MS, or PhD in Computer Science or a related technical field, or equivalent practical experience

Hands-on experience operating production Kubernetes clusters — node lifecycle, upgrades, troubleshooting — plus GitOps and infrastructure-as-code experience

Experience with GPU or ML workload scheduling, queueing and priorities, fractional GPU sharing, autoscaling, or multi-tenant resource management

Self-driven with a strong sense of ownership: a quick learner who is eager to take responsibility and drive projects forward end to end

Preferred Skills:

Experience with Ray or Kubeflow

Experience with lakehouse technologies such as Delta Lake or Apache Iceberg

Experience operating large-scale distributed data-processing and workflow systems, with hands-on depth in a system such as Apache Spark and working knowledge of Argo Workflows or an equivalent orchestrator

Haystack's AI agent can fill in and submit this application for you. How AI auto apply works