Automation QA Engineer with AI

Automation QA Engineer with AI

Alpharetta, GA

6+ Months

Experience: 10+ years

Tech Stack: Python/JavaScript, Test Automation, Pytest/Selenium, LLM Evaluation, Jira/Test Management

Job Description:

• Design and execute test cases for multi-turn agent conversations.
• Build evaluation frameworks to assess LLM quality.
• Automate API and UI testing.
• Create regression suites for agentic outputs.
• Drive test coverage and quality metrics.
• Develop automated testing frameworks using Python/JavaScript, Pytest, Selenium, or similar technologies.
• Validate AI/LLM responses and agent behavior against defined quality criteria.
• Track defects and test results using Jira or other test-management tools.