Gen AI Tester

Job Description

We are seeking experienced Gen AI Testers to join our team for a long-term engagement. The ideal candidate will have hands-on experience testing Generative AI applications, validating AI-generated outputs, creating test strategies, and ensuring the quality, reliability, and accuracy of AI-powered solutions.

Key Responsibilities

• Develop and execute test plans, test cases, and test scripts for Generative AI applications.
• Validate AI-generated responses for accuracy, relevance, consistency, and safety.
• Perform functional, regression, integration, and end-to-end testing.
• Identify, document, and track defects through resolution.
• Test AI prompts and evaluate prompt engineering scenarios.
• Verify LLM behavior, response quality, hallucinations, bias, and edge cases.
• Collaborate with developers, data scientists, and product teams to resolve issues.
• Participate in sprint planning, daily stand-ups, and QA reviews.
• Ensure compliance with quality standards and testing best practices.

Required Skills

• 3+ years of QA/Software Testing experience.
• Hands-on experience testing Generative AI/LLM-based applications.
• Strong understanding of AI testing methodologies, prompt validation, and response evaluation.
• Experience with manual and/or automation testing.
• Familiarity with defect tracking tools such as JIRA.
• Experience with API testing tools like Postman is a plus.
• Understanding of Agile/Scrum methodologies.
• Strong analytical, communication, and problem-solving skills.

Preferred Qualifications

• Experience working with OpenAI, Azure OpenAI, Google Gemini, Anthropic Claude, or similar LLM platforms.
• Knowledge of prompt engineering and AI evaluation metrics.
• Experience with test automation frameworks is a plus.
• Exposure to Python or scripting for testing is preferred.