AI Test Lead

AI Test Lead 100% Remote NOTES: Project Overview: TechEvo Application Modernization The Pitch: We are shifting our strategy from massive, slow application re-architectures to a highly targeted, AI-driven platform roadmap. Because bad actors are rapidly scaling AI models to find vulnerabilities, we need to stay ahead of the threat curve and "stop the bleeding." We are building internal AI agents to automate the modernization and security upgrades of our legacy applications.

The Technical Scope The team is focused on two primary modernization tracks, utilizing Claude Code and Java: Track 1: Infrastructure & Core Tech Upgrades Migrating legacy applications running on older Linux platforms and older Java runtimes (e.g., upgrading from JDK 17 to JDK 25). Building AI agents that can scan and understand legacy applications, generate technical specifications, and automatically execute the dependency-by-dependency upgrades.

Track 2: Security & Identity Modernization Transitioning applications from legacy security/authentication models (LDP, ISD) to modern Okta integration. Evaluating applications on a case-by-case basis to determine if they need a tech stack upgrade, a security upgrade, or both.

The Role: Agentic Engineers We are not looking for standard developers to manually rewrite code. We are looking for Agentic Engineers.

The Core Task: Write, instruct, and constantly refine AI agents/skills to handle the heavy lifting of modernizing our infrastructure stack.

The Goal: Engineer the AI agents to successfully complete 90% of the upgrade work autonomously, leaving the remaining 10% for human verification and validation.

The Ideal Candidate Profile Systems Thinkers: Must understand how massive enterprise systems interconnect. If they upgrade a database to a new server, they need to instantly understand the downstream impacts on connected systems. The "Fixers," Not The "Finders": We do not need auditors who simply identify vulnerabilities. We need hands-on engineers who build the automated solutions to fix them.

Battle-Scarred Veterans: We are looking for engineers with real-world scars. A standard Senior Developer with 7 years of isolated coding experience will not survive here; they need a history of navigating complex, high-stakes system failures (P1 incidents). Quick Screening Questions for the Recruiter "Tell me about a time you had to modernize a legacy application.

How did you map out the downstream impacts before making changes?" (Listen for systems thinking and architectural awareness, not just code-level changes). "How many P1 (Priority 1) incidents have you been involved in during your career, and what did they teach you about system design?" (Listen for battle scars and lessons learned). "If I asked you to automate a massive JDK upgrade across dozens of legacy applications today, how would you approach building an AI agent to do 90% of the work?" (Listen for an understanding of agentic engineering, automation, and prompt/skill design). ob Description Hands-on quality engineering lead or Senior AI Tester responsible for reverse-engineering application behavior, establishing missing system integration regression coverage, designing AI assisted test generation workflows and proving that agent driven modernization preserves functional, integration, security and operational behavior.

Required Skills - 7-10 Years of QA/SDET/QE experience with strong automation and AI testing skills. • Good experience with AI Assisted QE, system integration testing.

Strong SDET/quality engineering background with hands on automation of APIs, integrations, UI and end to end enterprise workflows. • Demonstrated use of AI/agentic tools to generate, refine, review, or maintain test scenarios and automation not merely general chatbot usage. • Experience creating regression, system integration coverage for applications with incomplete or outdated test suites. • Strong knowledge of test strategy, risk-based testing, test design, traceability, defect management, root-cause analysis and CI/CD quality gates. • Hands on experience with modern automation tools such as Playwright, Cypress, Selenium, REST/API automation, or equivalent; ability to choose tools based on architecture. • Working knowledge of Java/.NET application behavior, databases/SQL, APIs, middleware, authentication/authorization, and enterprise integration patterns. • Understanding of LDAP and token-based authentication concepts sufficient to design security and role/authorization test scenarios. • Ability to evaluate nondeterministic AI outputs with repeatable acceptance criteria, structured evaluation, and human verification.

Job Responsibilities Job Duties - Assess each application's existing test assets, business-critical flows, integrations, data dependencies, security behavior and regression risk before modernization begins. • Reverse-engineer system behavior with developers and AI agents, then create a risk-based system integration and regression strategy where adequate suites do not already exist. • Design and build executable test suites in enterprise test management/automation tooling (for example Xray-style suites), beyond developer-only unit/JUnit coverage. • Create, refine and govern AI agents/skills that assist with test scenario generation, test data design, coverage analysis, traceability, defect triage and regression maintenance. • Define test coverage for runtime/framework upgrades, dependency changes, OS/middleware changes, database/integration impacts, and LDAP-to-Okta/token-based security changes. • Establish quality gates for agent-generated code and modernization changes, including functional, integration, API, security, negative, compatibility, performance-smoke and deployment validation as appropriate. • Lead testers embedded in delivery pods, assign test work, review automation, coach AI-assisted testing practices and remain hands on for complex flows. • Partner with Development Leads to ensure technical specifications are testable and that acceptance criteria include measurable verification and rollback conditions. • Integrate automated tests into CI/CD where feasible and provide rapid, trustworthy feedback on modernization changes. • Drive defect root cause analysis and distinguish application defects, environment issues, test data issues, dependency incompatibility and AI agent errors. • Track escaped defects, automation coverage, execution reliability, agent generated test quality, rework, and regression effectiveness; use findings to improve skills/agents.

Job Requirements - 7-10 Years of QA/SDET/QE experience with strong automation and AI testing skills. • Good experience with AI Assisted QE, system integration testing.

Role Purpose: Hands-on quality engineering lead or Senior AI Tester responsible for reverse-engineering application behavior, establishing missing system integration regression coverage, designing AI assisted test generation workflows and proving that agent driven modernization preserves functional, integration, security and operational behavior.

Roles and Responsibilities: • 7-10 Years of QA/SDET/QE experience with strong automation and AI testing skills. • Good experience with AI Assisted QE, system integration testing. • Assess each application's existing test assets, business-critical flows, integrations, data dependencies, security behavior and regression risk before modernization begins. • Reverse-engineer system behavior with developers and AI agents, then create a risk-based system integration and regression strategy where adequate suites do not already exist. • Design and build executable test suites in enterprise test management/automation tooling (for example Xray-style suites), beyond developer-only unit/JUnit coverage. • Create, refine and govern AI agents/skills that assist with test scenario generation, test data design, coverage analysis, traceability, defect triage and regression maintenance. • Define test coverage for runtime/framework upgrades, dependency changes, OS/middleware changes, database/integration impacts, and LDAP-to-Okta/token-based security changes. • Establish quality gates for agent-generated code and modernization changes, including functional, integration, API, security, negative, compatibility, performance-smoke and deployment validation as appropriate. • Lead testers embedded in delivery pods, assign test work, review automation, coach AI-assisted testing practices and remain hands on for complex flows. • Partner with Development Leads to ensure technical specifications are testable and that acceptance criteria include measurable verification and rollback conditions. • Integrate automated tests into CI/CD where feasible and provide rapid, trustworthy feedback on modernization changes. • Drive defect root cause analysis and distinguish application defects, environment issues, test data issues, dependency incompatibility and AI agent errors. • Track escaped defects, automation coverage, execution reliability, agent generated test quality, rework, and regression effectiveness; use findings to improve skills/agents.

Required Skills: • Strong SDET/quality engineering background with hands on automation of APIs, integrations, UI and end to end enterprise workflows. • Demonstrated use of AI/agentic tools to generate, refine, review, or maintain test scenarios and automation not merely general chatbot usage. • Experience creating regression, system integration coverage for applications with incomplete or outdated test suites. • Strong knowledge of test strategy, risk-based testing, test design, traceability, defect management, root-cause analysis and CI/CD quality gates. • Hands on experience with modern automation tools such as Playwright, Cypress, Selenium, REST/API automation, or equivalent; ability to choose tools based on architecture. • Working knowledge of Java/.NET application behavior, databases/SQL, APIs . click apply for full job details