Why this role stands out
Accelerate your career at a Forbes-recognized top startup by joining AiDASH, where you will shape the future of grid resilience through cutting-edge model evaluation in a flexible hybrid environment. This role is perfect for a skilled SDET who thrives on technical innovation and wants to make a tangible global impact while collaborating with a high-growth, award-winning team. Apply today to contribute your expertise to a mission-driven company that is redefining the industry.
Quick overview
Job description
About AiDASH
AiDASH is leading the PreventionFirst™movement for electric utilities and transforming grid resilience through its pioneering platform that unifies vegetation, asset, storm, and wildfire intelligence. Powered by SatelliteFirst™ Inspection & Monitoring, AiDASH delivers comprehensive visibility across the entire grid at the right frequency and budget, using the right data modality. More than 200 customers trust AiDASH to keep the lights on, spend where it counts, and defend every decision, Securing Tomorrow across every mile of the grid. Learn more at www.aidash.com.
The PreventionFirst movement is growing, and so is the recognition behind it. In 2026, Forbes named AiDASH one of America's Best Startup Employers for the 4th consecutive year, and TIME included AiDASH among America's Top GreenTech Companies for the 3rd year in a row. Deloitte Technology Fast 500™ ranked AiDASH No. 12 in the San Francisco Bay Area, and No. 59 overall in their selection of the top 500 for 2024.
Join us in Securing Tomorrow Together!
The Role
We are looking for a dedicated ML Model Evals & Benchmarking Engineer to bridge the gap between machine learning research and high-performance production systems. In this role, you will own the end-to-end evaluation lifecycle, designing rigorous offline and online evaluation frameworks, defining robust model performance metrics, and architecting the scalable infrastructure needed to benchmark throughput, latency, resource utilization, and statistical accuracy across diverse ML/AI models.
How You’ll Make an Impact:
- Own model quality before production: Establish standard evaluation frameworks using statistical metrics (e.g., Precision, Recall, F1-Score, ROC-AUC) tailored to specific business domains. Build validation frameworks for computer vision models (preferably satellite or geospatial). Catch regressions in statistical metrics before they reach customers.
- Build version-aware model regression: Set up pipelines that compare inference outputs across model versions on curated golden datasets. Every release should come with a clear, data-backed go/no-go signal.
- Make data quality a first-class gate: Validate training and evaluation datasets for label quality, class balance, geographic and seasonal coverage, leakage, and drift. Ensure models are tested on data that reflects real-world conditions.
- Validate inference at scale: Design, build, and scale distributed evaluation pipelines and testing harnesses capable of running large-scale batch and real-time model benchmarks.
- Harden the platform end-to-end: Benchmark model inference servers for throughput, latency (p50, p99), and hardware efficiency (GPU/CPU/Memory footprints) under heavy concurrent loads.
- Software Engineering & Scripting: Advanced proficiency in Python (NumPy, Pandas, Scikit-learn, PyTorch/TensorFlow evaluation utilities) for building custom metric calculators and test harnesses.
- Distributed Compute & Execution: Experience orchestrating heavy evaluation jobs using frameworks like Ray, Apache Spark, or Kubeflow.
- Raise the bar for the team: Strengthen the shared automation frameworks and set standards for maintainable, non-flaky tests. Show how tools like Cursor and Claude can speed up test authoring, triage, and reporting while staying under human review.
- Influence release decisions: This is a high-visibility role. Your quality signals will directly shape what ships across Aidash’s product lines
What we’re looking for:
We are looking for a senior QA engineer with deep backend automation experience who can validate ML models and backend systems end-to-end, and who actively uses agentic AI development tools (e.g., Cursor, Claude) to accelerate test design, automation, debugging, and reporting — without compromising fundamentals, coverage, or engineering quality.
- 8–10 years in software testing and test automation, with strong ownership of backend system quality
- Experience testing ML applications (model/service behavior, data quality, inference pipelines, evaluation/regression of model-backed features)
- Broader stack familiarity: observability (logs/metrics/traces), cloud (AWS), contract testing, data-pipeline validation
- Experience improving shared automation frameworks and raising team standards for maintainable tests
- Solid grasp of testing fundamentals and best practices (test design, risk-based coverage, isolation, maintainability, flaky-test control, clear defect/RCA discipline)
- Strong coding ability in Python, with willingness to learn the other (or additional languages) as needed.
- Good understanding of backend system design and how services interact in distributed systems
- Practical exposure to databases — SQL and/or NoSQL (at least one is mandatory)
- Working knowledge of application deployment and underlying orchestration (e.g., MLflow, Kubeflow, containers, Kubernetes, service rollout/health concepts)
- CI/CD pipelines — must have (Jenkins/GitHub Actions/Bitbucket Pipelines or equivalent): triggering suites, reading failures, improving pipeline reliability
- Performance testing — must have (JMeter/k6/Locust or equivalent): load/benchmark basics, interpreting results, flagging SLA/risk
- Exposure to agentic development tools such as Cursor and Claude (or similar) for coding, test authoring, log/triage assistance, and documentation — used with human review and quality gates
We are proud to be an equal-opportunity employer. We are committed to embracing diversity and inclusion in our hiring practices, and we promote a work environment where everyone, from any race, color, religion, sex, sexual orientation, gender identity, or national origin, can do their best work.
Read our Privacy Policy here: https://www.aidash.com/policy/privacy-policy/
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Similar jobs
- CA

UAT TestingRecently posted Auto Apply
Capco · Mumbai · Hybrid · Posted 15 Nov 2026 🇮🇳
LLMPlaywrightRoot Cause Analysis+3Salary not listed - F7
Systems Test Engineer Auto Apply
F7d8749a 72d6 407c Bf98 292ea630c73a · Bengaluru · Hybrid · Posted 21 Oct 2026 🇮🇳
RoboticsPythonCI/CD+2TechnologySalary not listed - WA
QA EngineerRecently posted Auto Apply
Weekday AI · India · Remote · Posted 14 Nov 2026 🇮🇳
MicroservicesGitCompliance+9TechnologySalary not listed - YA

Senior SDETRecently posted Auto Apply
YipitData (Alternative) · India Remote · Remote · Posted 14 Nov 2026 🇮🇳
DockerSQLSelenium+12TechnologySalary not listed - YI

Senior SDETRecently posted Auto Apply
YipitData · India Remote · Remote · Posted 14 Nov 2026 🇮🇳
DockerSQLSelenium+12TechnologySalary not listed - CO

Principal QE Engineer/Architect - Automation Testing (10+ Years)- 11936 Auto Apply
Coupa · Pune · Hybrid 🇮🇳
Stakeholder ManagementEngineeringSalary not listed - PR
Quality Assurance EngineerRecently posted Auto Apply
Provenirinc · Bengaluru · Remote · Posted 13 Nov 2026 🇮🇳
OnboardingDockerGCP+32TechnologySalary not listed - WE

Associate Testing QA Engineer Auto Apply
Weloglobal · Gurugram · On-site 🇮🇳
PythonVBAVideo EditingTechnologySalary not listed - NI
Associate Software Test Engineer (Playwright, Typescript/ Javascript)Recently posted Auto Apply
NielsenIQ · Pune, MH · On-site · Posted 13 Nov 2026 🇮🇳
ScrumRoot Cause AnalysisGenerative AI+1TechnologySalary not listed - EN
Senior Automation Tester with AI Agent Testing experienceRecently posted Auto Apply
Endava · Bengaluru, Karnataka · Hybrid · Posted 12 Nov 2026 🇮🇳
PlaywrightTechnologySalary not listed - SC

Intern - QA Engineer - Royal Lounge CasinoRecently posted Auto Apply
Scopely · Bengaluru · Hybrid · Posted 13 Nov 2026 🇮🇳
Microsoft ExcelMicrosoft OfficeMicrosoft PowerPoint+1TechnologySalary not listed - PL

QA Engineer - ContractRecently posted Auto Apply
Plume · Hyderabad · Hybrid · Posted 13 Nov 2026 🇮🇳
GitContinuous ImprovementDNS+7TechnologySalary not listed
