AI Engineer

AI Engineer – GenAI Quality & Evaluation

Locations: Oakdale, MN | Atlanta, GA | Scottsdale, AZ | St. Petersburg, FL | Omaha, NE
Work Arrangement: Hybrid – 3 days onsite
Employment Type: Contract (C2C)
Requirement: Candidates must be local to one of the listed locations.

We are looking for an experienced AI Engineer with a background in software engineering, QA, automation testing, or business systems analysis. The ideal candidate should have at least one year of hands-on experience with Generative AI, LLMs, prompt engineering, conversational AI, or AI testing.

Key Skills

• Generative AI and Large Language Models
• Prompt engineering and prompt optimization
• LLM testing and AI quality assurance
• RAG, vector databases, semantic search, and embeddings
• AI evaluation and observability
• Python and SQL
• REST APIs, JSON, and Postman
• Test automation frameworks
• Azure DevOps, Git, and Azure OpenAI
• Arize AI or similar observability tools

Responsibilities

• Design and execute test plans for GenAI and LLM applications
• Test prompts, agents, APIs, conversational AI, and RAG workflows
• Evaluate AI responses and identify hallucinations or quality issues
• Perform functional, regression, integration, and API testing
• Build or improve automated testing frameworks
• Track defects, prompt versions, experiments, and test results
• Prepare technical documentation and evaluation reports
• Work closely with developers, QA teams, product teams, and business stakeholders

Qualifications

• 5–10 years of relevant technical experience
• 1+ year of hands-on GenAI or LLM experience
• Strong experience with test planning, APIs, JSON, and Agile delivery
• Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related field
• Strong analytical, troubleshooting, communication, and documentation skills