Lead / Senior Azure DevOps Engineer – LLMOps

Introduction

Our client is seeking a Lead / Senior Azure DevOps Engineer with strong LLMOps experience to join a cloud engineering team supporting secure, large-scale enterprise and banking environments. The ideal candidate will have deep hands-on experience with Azure, Kubernetes, Infrastructure-as-Code, CI/CD, enterprise security, and production AI/LLM deployments. This role will focus heavily on building and operating infrastructure for AI/ML workloads, including model deployment, inference optimization, monitoring, evaluation, and production LLM operations.

Responsibilities

• Design, deploy, and maintain secure Microsoft Azure cloud infrastructure for enterprise and regulated environments.
• Build and manage Azure Kubernetes Service (AKS) and containerized workloads at scale.
• Develop and maintain infrastructure using Terraform and Infrastructure-as-Code (IaC) practices.
• Build, automate, and support CI/CD pipelines using Jenkins, GitLab, Azure DevOps, or similar tools.
• Deploy and operate LLM/AI workloads in production environments.
• Implement LLMOps processes covering model deployment, inference, monitoring, evaluation, optimization, and lifecycle management.
• Support GPU-accelerated AI infrastructure and technologies such as NVIDIA GPNVIDIA NIM.
• Optimize LLM inference performance, scalability, reliability, and cloud infrastructure costs.
• Implement cloud security best practices including Azure IAM/RBAC, networking, secrets management, encryption, and access controls.
• Support hybrid cloud and on-premises infrastructure environments and integrations.
• Implement monitoring, logging, alerting, and operational processes for cloud and AI workloads.
• Ensure infrastructure and AI workloads meet enterprise security, compliance, and regulatory requirements.
• Troubleshoot complex Kubernetes, cloud infrastructure, networking, and production deployment issues.
• Mentor engineers and serve as a technical point of contact for client engagements.

Requirements

Required Skills

• 10+ years of experience in DevOps, Cloud Engineering, or Infrastructure Engineering.
• Strong hands-on Microsoft Azure experience.
• Extensive experience with Azure Kubernetes Service (AKS), Kubernetes, and Docker.
• Strong Terraform / Infrastructure-as-Code experience.
• Hands-on experience building and maintaining CI/CD pipelines using Jenkins, GitLab, Azure DevOps, or similar.
• Strong scripting and automation skills using Python and/or Bash.
• Hands-on experience with LLMOps, MLOps, or production AI/ML infrastructure.
• Experience deploying and operating LLMs/model inference workloads in production.
• Experience with model monitoring, evaluation, inference optimization, and production operations.
• Experience with GPU infrastructure, preferably NVIDIA GPUs and/or NVIDIA NIM.
• Strong understanding of cloud networking, IAM/RBAC, security, secrets management, and encryption.
• Experience working with hybrid/on-premises infrastructure in addition to cloud environments.
• Experience supporting enterprise-scale and security-sensitive environments.
• Strong troubleshooting, communication, leadership, and mentoring skills.

Preferred Qualifications

• Experience in banking, financial services, or other regulated industries.
• Experience with Azure AI, Azure Machine Learning, Azure OpenAI, or similar AI platforms.
• Experience with NVIDIA NIM, NVIDIA Triton, GPU orchestration, or other LLM inference technologies.
• Experience with AKS GPU node pools and Kubernetes-based AI workloads.
• Experience with observability tools such as Prometheus, Grafana, Azure Monitor, or similar.
• Azure certifications such as Azure Solutions Architect Expert, Azure DevOps Engineer Expert, or Azure Administrator Associate.
• Experience with Google Cloud Platform or AWS is a plus, but strong Azure experience is more important.

Location Requirements

Candidates must be US-based and available to work onsite. Preferred locations, in order:

1. New York / New Jersey
2. Lake Mary, Florida
3. Pittsburgh, Pennsylvania