Quick overview
Job description
In this role, you will:
-
Build and maintain the Behavior Evaluation data pipelines (PySpark/zspark medallion architecture on Databricks, EMR Serverless and Airflow) that produce weekly per-geofence estimates.
-
Turn the statistical method (sampling, triage weights, mileage sharing, projections) into tested, reproducible code with checks that fail loudly when the data is bad.
-
Build tools that cut the manual work in each weekly run.
-
Work with cross functional stakeholders so the methodology is applied consistently in code and in clearance reports.
-
Help adapt to new release methodologies and new geofences.
Qualifications:
-
7 - 9 Years of experience or 5+ years with a strong data-platform track record
-
Expert Python, with production experience in PySpark/Spark and Pandas
-
Experience designing scalable batch data pipelines: orchestration, Delta/lakehouse tables, idempotent backfills
-
Solid grounding in applied statistics: sampling, weighting, estimators, confidence intervals
-
Strong testing and CI habits for data code
Bonus Qualifications:
-
Experience with autonomous-vehicle simulation, safety metrics or triage workflows
-
Experience putting ML models into production in a data pipeline
-
Experience with Databricks and AWS (EMR Serverless, S3)
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Similar jobs
- KC
Software Engineer Co-op (Spring 2027)Recently posted Auto Apply
Klaviyo Campus · Boston · Hybrid · Posted 12 Nov 2026 🇺🇸
DjangoMySQLSpring+9TechnologySalary not listed - SE

Senior Software Engineer, Developer Infrastructure Auto Apply
Sentry · San Francisco · Hybrid 🇺🇸
DockerGCPAWS+3Technology$190k - $400k/yr - AC

Sr. Specialist, Systems Engineer (Test Equip)Recently posted Auto Apply
Acronaviation · Phoenix · On-site · Posted 12 Nov 2026 🇺🇸
MATLABAssemblyBackground Checks+2TechnologySalary not listed - RD
Forward Deployed Engineer (NYC)Recently posted Auto Apply
Rox Data Corp · New York · Hybrid · Posted 12 Nov 2026 🇺🇸
OnboardingSQLAWS+6Engineering$2k/mo - AC

Sr. Specialist, Systems Engineer (Environmental Qualification)Recently posted Auto Apply
Acronaviation · Phoenix · On-site · Posted 12 Nov 2026 🇺🇸
Background ChecksComplianceTechnical SupportTechnologySalary not listed - AC

Software Engineer - Observability (Pulse) Auto Apply
Acceldata · Campbell · On-site 🇺🇸
MicroservicesJavaPython+2Technology$100k - $170k/yr - ZO

Contract Student Worker – Firmware Engineer (FW Core System Services)Recently posted Auto Apply
Zoox · Foster City · Hybrid · Posted 11 Nov 2026 🇺🇸
C++PythonLinuxTechnology$30/hr - ZO

Contract Student Worker - Software Engineer (20/hrs week)Recently posted Auto Apply
Zoox · Foster City · On-site · Posted 11 Nov 2026 🇺🇸
GitHTMLCSS+4Technology$30/hr - SO

Senior Software Engineer (Portals)Recently posted Auto Apply
SoundExchange, Inc. · United States · Hybrid · Posted 12 Nov 2026 🇺🇸
DjangoMicroservicesSQL+7Technology$158k - $190k/yr - RE

Software Certification EngineerRecently posted Auto Apply
Regent · North Kingstown · On-site · Posted 12 Nov 2026 🇺🇸
ComplianceAgileTechnologySalary not listed - PA

Senior Software EngineerRecently posted Auto Apply
Podium Automation · New York City · On-site · Posted 12 Nov 2026 🇺🇸
DockerSQLAWS+7TechnologySalary not listed - NU
Systems Engineering, New Grad 2027Recently posted Auto Apply
Nuro · Mountain View · Hybrid · Posted 12 Nov 2026 🇺🇸
RoboticsAutonomous DrivingC+++2Engineering$153k/yr