ML Ops Engineer / Overwatch — Observability and Evaluation Engineer

Job Title: Overwatch — Observability and Evaluation Engineer

Location: Charlotte, North Carolina 

Job Type: Contract

Work Model: Onsite

Experience: 10+ years in Software Engineering; 3+ years in AI/ML and Machine Learning Model Operations

Job Overview

 

Our client is seeking a skilled ML Ops Engineer to join their team in Charlotte, NC. In this role, you will be responsible for building, deploying, and monitoring machine learning pipelines while ensuring model governance, observability, and evaluation at scale. You will work closely with cross-functional engineering teams to support high-performance AI/ML systems in production.

Key Responsibilities

 

• Develop and maintain ML pipelines using tools such as MLflow, Kubeflow, or Vertex AI
• Automate model training, testing, deployment, and monitoring across cloud environments (Google Cloud Platform, AWS, Azure)
• Implement CI/CD workflows for model lifecycle management including versioning, monitoring, and retraining
• Monitor model performance using observability tools and ensure compliance with model governance frameworks (MRM, documentation, explainability)
• Provision containerized environments and support model scoring via low-latency APIs
• Leverage AutoML tools (Vertex AI AutoML, H2O Driverless AI) for rapid, low-code model development and deployment
• Implement telemetry, traces, dashboards, SLOs, evaluation suites, and readiness evidence for priority agent releases

Required Skills

 

• Python, Java, SQL
• ML libraries: scikit-learn, XGBoost, TensorFlow, PyTorch
• MLflow, Kubeflow, Vertex AI
• Cloud platforms: Google Cloud Platform, AWS, Azure
• CI/CD, containerization, low-latency API integration
• LLM and agent evaluation, tracing, telemetry, metrics, and dashboards
• SLOs, test automation, prompt and model performance analysis

Preferred Skills

 

• Experience with AutoML platforms (Vertex AI AutoML, H2O Driverless AI)
• Background in model governance and MRM frameworks
• Hands-on production operations experience for AI/ML systems

Location & Work Model

 

Onsite in Charlotte, North Carolina. Local candidates only — in-person interviews required.

Engagement Details

 

Contract engagement. Start date to be confirmed. Apply with your updated resume to be considered.