Why This Role Stands Out
This role offers a unique opportunity to pioneer AI and LLM testing, building cutting-edge evaluation frameworks and automating complex conversational agents, perfect for experienced QA engineers eager to shape the future of intelligent systems. If you excel in Python/JavaScript, possess strong automation skills, and are excited by the prospect of working with advanced AI technologies, you'll thrive in this impactful position. Apply now to join a forward-thinking team and grow your expertise in a critical emerging field.
Quick Overview
Job Description
Automation QA Engineer with AI/LLM
Alpharetta, GA- Only Locals
6+ Months
Experience: 10+ years
Tech Stack: Python/JavaScript, Test Automation, Pytest/Selenium, LLM Evaluation, Jira/Test Management
Job Description:
- Design and execute test cases for multi-turn agent conversations.
- Build evaluation frameworks to assess LLM quality.
- Automate API and UI testing.
- Create regression suites for agentic outputs.
- Drive test coverage and quality metrics.
- Develop automated testing frameworks using Python/JavaScript, Pytest, Selenium, or similar technologies.
- Validate AI/LLM responses and agent behavior against defined quality criteria.
- Track defects and test results using Jira or other test-management tools.
Similar jobs
- TC
Quality Management Systems Specialist II
TE Connectivity
Mount Joy, PA🇺🇸$74.6k - $93.3k/yrHybrid5 weeks agoComplianceContinuous ImprovementExternal Audit+4 - DO
Quality Assurance Specialist with Security Clearance
NewDepartment of the Army
Chambersburg, PA🇺🇸$85.4k - $111.1k/yrRemote12 hours agoLESSTechnology - WY
Test Engineer 1 with Security Clearance
NewWyetech
Annapolis Junction, MD🇺🇸$45 - $76/hrOn-siteYesterdayTechnology - CI
Quality Assurance Test Automation Manager - Citi
NewCiti
Tampa, FL🇺🇸$97.0k - $145.4k/yrHybrid12 hours agoAWSTechnology - PD
Test & Evaluation Engineer 3 with Security Clearance
NewPDS Defense Inc.
Taft, CA🇺🇸$71 - $76/hrHybridYesterdayEngineering - VE
STS Principal Test Development Engineer - MAT Ramp with Security Clearance
NewVertuCom
Tewksbury, MA🇺🇸Hybrid2 days agoPCB DesignEngineering - AR
Software Quality Assurance and Testing Lead
NewAlexan RPM
United States🇺🇸Hybrid12 hours agoOracleTechnology - CO
Quality Assurance Associate I
NewConduent
Florence, Kentucky🇺🇸On-site5 hours agoData Entry - CO
Quality Assurance Associate I
NewConduent
Cincinnati, Ohio🇺🇸On-site5 hours agoData Entry - CO
Quality Assurance Associate I
NewConduent
Latonia, Kentucky🇺🇸On-site5 hours agoData Entry - CO
Quality Assurance Associate I
NewConduent
Morning View, Kentucky🇺🇸On-site5 hours agoData Entry - CO
Quality Assurance Associate I
NewConduent
Lawrenceburg, Indiana🇺🇸On-site5 hours agoData Entry
