Haystack
← Back to Jobs
Technology
RI

QA Engineer with AI/ML experience

Rivago infotech incMcLean, VA🇺🇸United StatesPosted 11 Sept 2026

Quick Overview

Seniority
Mid Senior
Work mode
On Site
Location
McLean, VA, United States
Posted
21 hours ago
SQLETLMLOpsMachine LearningSeleniumSnowflakeAzureDatabricksGenerative AIJiraLLMPostmanPythonStakeholder ManagementTestNGpytest

Job Description

Role: AI/ML QA Engineer
Location: McLean, VA 22102 (100% Onsite)
Primary Skills: AI/ML Testing, Python, SQL, ETL Testing, Snowflake, Databricks, Azure, Healthcare Domain, GenAI Testing.
 
Job Summary: We are seeking a skilled AI/ML QA Engineer to ensure the quality, reliability, performance, and governance of AI and Machine Learning solutions in the healthcare domain. The role involves validating AI models, testing data pipelines, evaluating model outputs, identifying bias and performance issues, and supporting the end-to-end quality assurance lifecycle for AI-driven applications.
 
The ideal candidate should have strong QA expertise combined with exposure to AI/ML technologies, data validation, automation testing, and healthcare data environments.
 
Key Responsibilities
  • Develop and execute test strategies for AI/ML applications and data-driven systems.
  • Validate AI model outputs for accuracy, consistency, bias, fairness, and reliability.
  • Design and execute data quality testing for structured and unstructured datasets.
  • Test ETL/ELT pipelines supporting AI model development and deployment.
  • Validate training, testing, and inference datasets used in AI solutions.
  • Perform functional, integration, regression, performance, and user acceptance testing.
  • Work with Data Scientists, Data Engineers, Product Owners, and Business Stakeholders to define AI testing requirements.
  • Create automated test frameworks for AI and analytics solutions.
  • Monitor model performance and identify drift, degradation, and anomalies.
  • Verify compliance with healthcare data quality and governance requirements.
  • Support defect triage, root cause analysis, and resolution tracking.
  • Prepare QA documentation, test cases, test reports, and validation evidence.
 
Required Technical Skills
  • QA & Automation
  • Manual Testing
  • Test Planning & Execution
  • Test Automation (Selenium, API Testing with Postman)
  • JIRA, TestNG / PyTest
  • Data Testing (SQL, ETL Testing, Data Validation)
  • AI/ML Testing (AI Model Validation, LLM Testing, Bias & Fairness Testing)
  • Programming (Python, PySpark preferred)
  • Cloud & Data Platforms (Azure, Databricks, Snowflake)
 
Expertise You'll Bring:
  • Experience testing Generative AI and LLM-based applications.
  • Familiarity with GitHub Copilot, Microsoft 365 Copilot, and enterprise AI tools.
  • Understanding of prompt engineering and prompt validation techniques.
  • Experience evaluating AI output quality, hallucinations, and response consistency.
  • Knowledge of Responsible AI principles, fairness, explainability, and governance.
 
Preferred Qualifications
  • Experience validating AI/ML models in production environments.
  • Exposure to MLOps processes and model deployment frameworks.
  • Knowledge of healthcare compliance and data privacy requirements.
  • Strong analytical and problem-solving skills.
  • Excellent communication and stakeholder management capabilities.
 
Education
  • Bachelor's Degree in Computer Science, Information Technology, Engineering, Data Science, or related field.
 
Nice to Have
  • Healthcare domain experience
  • Snowflake Certification
  • Azure Certification
  • Databricks Certification
  • AI/ML Testing Certification
  • Experience with GenAI and LLM evaluation frameworks

Similar jobs

Browse related jobs

Apply now