Quick overview
Job description
Role Overview
We are looking for a QA Lead who can architect the overall quality strategy for JazzX AI’s platforms and AI-powered products. This role requires a blend of traditional QA expertise, AI/ML awareness, and strong leadership capabilities.
You will lead the quality roadmap for agent workflows, LLM outputs, APIs, and cloud services while managing a high-performing QA team.
Key Responsibilities
- Quality Strategy & Leadership
- Build and own JazzX AI’s end-to-end QA strategy across AI copilots, workflows, and enterprise deployments.
- Lead, mentor, and scale a team of QA engineers and automation specialists.
- Partner closely with Product, ML Engineering, and Platform Engineering to embed quality in early design phases.
- Define and enforce quality gates and release-readiness standards.
- AI/LLM-Specific Quality Assurance
- Develop evaluation systems for LLM accuracy, hallucination detection, factuality, safety, and bias mitigation.
- Design scalable tests for agent reasoning, tool-use correctness, and multi-step workflow validation.
- Create synthetic datasets and scenario-based tests for edge-case prediction and regression testing.
- Validate model performance across fine-tuning, RAG, knowledge-grounding, and inference scenarios.
- Automation Engineering
- Architect and manage automation frameworks using Python/Java/JS, Playwright, Selenium, PyTest, Postman, etc.
- Ensure seamless integration of automated suites into CI/CD pipelines (GitHub Actions/GitLab/Jenkins).
- Automated API, microservices, and backend testing for JazzX AI’s cloud architecture running on AWS/GCP.
- Performance Testing
- Load Testing, Stress Testing, Spike Testing, Endurance/Soak Testing using industry-standard tools like JMeter, LoadRunner, Gatling, k6, or Locust.
- Release, Monitoring & Reliability
- Own the release sign-off process for all AI software components and platform features.
- Conduct RCA, maintain issues of dashboards, and drive continuous improvement in QA lifecycle.
- Collaborate with DevOps/ML Ops to monitor model drift, performance anomalies, and post-production behaviors.
- Documentation & Communication
- Prepare detailed QA documents: strategies, test plans, evaluation reports, and execution metrics.
- Provide clear communication of risks, quality insights, and impact analysis to leadership.
Required Qualifications
- 12+ years of experience in Software QA.
- Strong hands-on experience with test automation frameworks and scripting (Python/Java/JavaScript).
- Experienced in working with microservice-based architecture
- Working knowledge of AI/ML systems, including datasets, model evaluation, vector stores, RAG, and LLM operations.
- Architect and manage automation frameworks using Python/Java/JS, Playwright, Selenium, PyTest, Postman, etc.
- Automated API, microservices, and backend testing for JazzX AI’s cloud architecture running on Azure /AWS.
- Experience testing distributed cloud systems, APIs, and high-performance backend services.
- Strong analytical and problem-solving skills with the ability to drive organization-wide quality initiatives.
Preferred Skills
- Experience testing LLMs, RAG pipelines, or autonomous AI agents.
- Familiarity with AI observability tools, logging/monitoring stacks (ELK, Prometheus, Datadog).
- Exposure security testing, performance/load testing, and compliance (SOC2, GDPR).
- Ability to operate in a fast-paced, zero-to-one startup environment.
Why Join JazzX AI:
What draws people to JazzX AI is the opportunity to build something truly foundational.
We are at the beginning of a new era — one where enterprise systems will no longer be static tools, but dynamic collaborators that learn, reason, and evolve alongside the people who use them. At JazzX AI, you’ll help build a platform that redefines how work happens in the AGI era.
JazzX AI brings together vision, deep technology, and purpose. This is not about experimenting with AI features; it’s about turning the promise of AGI into practical, auditable, and human-aligned systems that enterprises can trust and rely on at scale. The work here directly impacts productivity, efficiency, and decision-making in complex, real-world environments.
If you’re motivated by building systems that matter — systems that combine intelligence with responsibility — JazzX AI offers the chance to do that work at a deeper level, with real ownership and lasting impact.
Headquartered in Los Altos, CA. Backed by SAIGroup.
Learn more about JazzX AI:
Website: https://jazzx.ai
LinkedIn: https://www.linkedin.com/company/jazzx-ai
Learn more about SAIGroup:
Website: https://saigroup.ai
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- IZ

SAP - QA Automation TesterRecently posted Auto Apply
Input Zero Technologies · Pune, Maharashtra · On-site · Posted 19 Nov 2026 🇮🇳
SQLGitGitHub Actions+12TechnologySalary not listed - IZ

SAP - QA Automation TesterRecently posted Auto Apply
Input Zero Technologies · Bengaluru, Karnataka · On-site · Posted 19 Nov 2026 🇮🇳
SQLGitGitHub Actions+12TechnologySalary not listed - TH

Senior QA Engineer (On-site)Recently posted Auto Apply
Tech Holding · Ahmedabad · Hybrid · Posted 19 Nov 2026 🇮🇳
GCPAWSContinuous Improvement+9TechnologySalary not listed - CA

Associate Junior- Testing - Pune- CitiRecently posted Auto Apply
Capco · Pune · Hybrid · Posted 19 Nov 2026 🇮🇳
Stakeholder ManagementSwiftFixed Income+4Salary not listed - NE

QA EngineerRecently posted Auto Apply
Nextiva · Bengaluru · Hybrid · Posted 19 Nov 2026 🇮🇳
GCPGitAWS+10TechnologySalary not listed - QU

QA Engg - (Manual+AI-Automation)Recently posted Auto Apply
QuinStreet · Pune · Hybrid · Posted 19 Nov 2026 🇮🇳
MySQLSQLGit+16Salary not listed - SU

QA Engineer Team LeadRecently posted Auto Apply
SupplyHouse.com · India · Remote · Posted 18 Nov 2026 🇮🇳
ScrumSeleniumCypress+3TechnologySalary not listed - SU

QA Engineering Team LeadRecently posted Auto Apply
SupplyHouse.com · India · Remote · Posted 18 Nov 2026 🇮🇳
ScrumSeleniumBackground Checks+8₹29k - ₹36k/yr - ON

Mobile QA Automation EngineerRecently posted Auto Apply
OneSpan · Noida · Hybrid · Posted 18 Nov 2026 🇮🇳
SwiftGitBash+5TechnologySalary not listed - BG
QA Engineer / Tester – ServiceNow PlatformRecently posted Auto Apply
Bosch Group · bengaluru · On-site · Posted 18 Nov 2026 🇮🇳
GitSeleniumJMeter+12TechnologySalary not listed - BG
QA Automation Specialist – ServiceNow PlatformRecently posted Auto Apply
Bosch Group · bengaluru · On-site · Posted 18 Nov 2026 🇮🇳
GitSeleniumGitHub Actions+16TechnologySalary not listed - CO

Senior Engineer I - QARecently posted Auto Apply
Contentstack · Pune · Hybrid · Posted 18 Nov 2026 🇮🇳
MochaGraphQLJest+9EngineeringSalary not listed
