QA Engineer for AI Products/Solutions

Job Description

• Design and execute test plans for AI/ML-driven features, including model outputs, prompts, and integrated application behavior.
• Build and maintain automated test suites covering functional, regression, integration, and API testing.
• Evaluate model outputs for accuracy, consistency, bias, hallucination, and edge-case failures.
• Develop evaluation frameworks and golden datasets/test cases to benchmark model performance over time.
• Test prompt engineering changes, model version upgrades, and fine-tuning outputs for regressions.
• Perform adversarial and red-team style testing to surface safety, security, and robustness issues.
• Validate data pipelines feeding into AI models, including data quality, schema validation, and drift detection.
• Collaborate with data scientists and ML engineers to define acceptance criteria and quality metrics for models.
• Test latency, scalability, and reliability of AI services under load.