AI QA Engineer

Job Description

Seeking an AI Transformation Engineer with strong experience in Generative AI, LLMs, RAG and AI-powered application testing. The ideal candidate will drive AI quality initiatives, build AI evaluation frameworks, and ensure the reliability, accuracy, and performance of AI solutions. A solid QA automation background is required.

Key Responsibilities

• Lead AI quality and transformation initiatives for AI-powered applications.
• Design and implement automated validation frameworks for AI/LLM solutions.
• Evaluate AI model performance, accuracy, reliability, and safety.
• Develop AI testing and evaluation pipelines using Python.
• Collaborate with AI/ML teams to improve AI application quality.

Required Skills

• Strong experience in AI Engineering, Generative AI, and LLMs.
• Experience building AI evaluation and validation frameworks.
• Strong understanding of AI/ML concepts, prompt evaluation, and model testing.
• QA automation background with tools such as Playwright and Selenium.
• Proficiency in Python and CI/CD integration.

Note: The responsibilities outlined above are intended to provide a general overview of the role. Actual responsibilities may vary based on business priorities, project needs, and team requirements.