Lead Engineer AI Agent Platform - Nashville, TN - 100% Onsite (5 Days Onsite/week) - Contract - CTS/Dollar General

Job Title: Lead Engineer AI Agent Platform

Job Location: Nashville, TN - 100% Onsite (5 Days Onsite/week)

Lead Engineer AI Agent Platform

About the Role:-

We are building a next-generation enterprise AI platform that enables teams to design, deploy, and scale intelligent agents across digital, marketing, and operational workflows.

This platform provides standardized infrastructure, model access, orchestration, evaluation, and governance, enabling teams to rapidly build AI-powered capabilities in a secure, scalable, and production-ready way.

We're looking for Lead Engineers who can both:-

Build high-quality AI agents and workflows

Lead the evolution of the platform powering them at scale

This role sits at the intersection of Agentic AI, platform engineering, and enterprise systems, with strong emphasis on security, CI/CD, observability, and rigorous validation of AI systems.

What You'll Do:-

Build and deploy AI agents and agentic workflows for real-world business use cases.

Design and evolve core platform capabilities for agent development, orchestration, and runtime.

Develop RAG pipelines, tool integrations, and structured reasoning systems.

Enable flexible usage across multiple LLM providers and model libraries.

Implement secure, scalable, and observable architectures for AI systems.

Establish robust CI/CD pipelines for agent lifecycle (build, test, deploy, rollback).

Define and implement agent testing & evaluation strategies, including:

o Automated evaluation pipelines

o Agentic testing models and simulation-based validation

o Continuous quality, accuracy, and regression testing

Build and enforce LLM observability and evaluation frameworks (this is core to the role).

Drive platform standards, reusable components, and engineering best practices.

Partner across teams and external partners to scale adoption of the platform.

What We're Looking For:-

Core Engineering Skills:-

8+ years of software engineering experience.

Strong experience with Google Cloud Platform (Vertex AI, GKE, BigQuery, APIs).

Experience with distributed systems and scalable platform design.

AI / Agentic Systems:-

Hands-on experience with:

o LLMs, embeddings, and RAG architectures

o Agentic workflows and AI-driven systems

Experience working with multiple model providers / model ecosystems.

Strong familiarity with agent frameworks such as LangChain, LangGraph, or similar ecosystems.

Platform, Security & DevOps:-

Strong experience with:

o CI/CD pipelines and automation for production systems

o Secure system design (authentication, authorization, data access controls)

o API-first and microservices architectures

Experience designing highly observable systems (metrics, tracing, logging).

Ability to manage performance, cost, and reliability trade-offs in AI workloads.

Evaluation, Observability & Testing (Must-Have):-

Hands-on experience with LLM evaluation frameworks and observability tooling.

Experience designing:

o Model evaluation and benchmarking pipelines

o Test harnesses for agent workflows

o Continuous validation / regression testing for AI systems

o Simulation or scenario-based testing for agent behavior

Nice to Have:-

Experience with AI governance / Responsible AI practices.

Familiarity with enterprise-scale AI platforms or shared developer platforms.

Retail or domain experience (digital, marketing, operations).

Why This Role:-

Work on a foundational AI platform initiative with enterprise-wide impact.

Define how AI agents are built, governed, and scaled across the organization.

Influence both hands-on engineering and long-term platform direction.