AI Quality Engineer

Role: AI Quality Engineer

Location: 100% Remote – U.S.
Hours: 8:00 AM – 5:00 PM ET
Clearance: Must be eligible to obtain and maintain a Public Trust clearance / favorable suitability determination

Position Overview

We are seeking an experienced AI Quality Engineer to support a federal customer in designing, validating, and implementing AI-enabled modernization solutions.

The AI Quality Engineer will help develop and integrate AI capabilities while ensuring that AI-generated code, prototypes, skills, and agents meet quality, security, reliability, and compliance standards before moving into production.

This role will support broader modernization initiatives focused on automating and streamlining servicing operations, reducing manual processing, improving data quality, aligning solutions with modernization standards, and establishing a roadmap for the eventual retirement of legacy platforms.

The ideal candidate combines strong software QA and test engineering experience with hands-on exposure to AI-assisted development, automated testing, CI/CD, and enterprise technology environments.

Key Responsibilities

• Design and implement AI-enabled solutions supporting customer modernization initiatives.
• Develop and integrate AI capabilities into existing systems, applications, and workflows.
• Establish and enforce QA frameworks, including:
  • Test plans
  • Acceptance criteria
  • Regression test suites
  • Defect management processes
• Validate AI-generated code, skills, agents, and prototypes for accuracy, reliability, security, and compliance.
• Guide the development of reusable AI skills and agents with an emphasis on:
  • Automated testing
  • Security scanning
  • Data sensitivity
  • Scalability
  • Reliability
• Collaborate with DevOps engineers to create and orchestrate long-running, multi-agent workflows.
• Ensure AI workflows provide appropriate reliability, observability, monitoring, and recoverability in production environments.
• Support the development of an innovation pipeline that moves AI-generated or “vibe-coded” prototypes from sandbox environments into secure, enterprise-grade applications.
• Review and quality-check prototype outputs developed by Value Engineers before promotion through the innovation pipeline.
• Continuously monitor AI-assisted development outputs for:
  • Accuracy
  • Reliability
  • Security
  • Compliance
  • Quality drift
• Provide feedback to AI Engineers and Value Engineers to improve prompts, skills, agents, and development practices.
• Collaborate across product, engineering, data, DevOps, security, and business SME teams.

Minimum Qualifications

• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent professional experience.
• Demonstrated experience in software quality assurance, test engineering, or a related discipline.
• Experience designing test plans, defining acceptance criteria, executing regression testing, and managing defects.
• Hands-on experience testing or validating AI-generated code, AI prototypes, or AI-assisted/"vibe-coded" solutions.
• Experience with automated testing tools and frameworks such as Selenium across web and enterprise applications.
• Experience with one or more of the following:
  • Salesforce
  • AWS
  • AWS Lambda
  • AWS Redshift
  • Java
  • PostgreSQL
  • Comparable enterprise technology stacks
• Familiarity with CI/CD pipelines, automated quality gates, and modern software delivery practices.
• Strong analytical and problem-solving skills, including the ability to identify:
  • Security risks
  • Data-sensitivity concerns
  • Compliance issues
  • Reliability and quality risks
• Excellent written and verbal communication skills.
• Ability to collaborate effectively across technical and business stakeholder groups.

Preferred Qualifications

• Direct experience supporting Federal clients.
• Experience with FedRAMP authorization and compliance processes.
• Mortgage, fintech, financial services, or other regulated-industry experience.
• Familiarity with AI safety, responsible AI, governance, risk, and compliance.
• Experience with or ability to teach tools such as:
  • ChatGPT
  • Claude
  • Gemini
  • Notion
  • Airtable
  • Zapier
  • LangChain
  • Similar AI and automation platforms
• Experience working in startups, consulting, internal strategy, L&D, innovation, or AI transformation teams.

What Success Looks Like

• AI-generated code and prototypes are thoroughly tested before entering production environments.
• AI skills and agents meet established quality, security, reliability, and compliance standards.
• Automated testing and quality gates become an integral part of the AI innovation pipeline.
• AI-assisted prototypes can be transitioned safely from experimentation into enterprise-grade applications.
• Quality findings are continuously fed back into AI prompts, skills, agents, and development processes.
• Modernization initiatives achieve improved automation, data quality, reliability, and operational efficiency.