Azure DevOps Engineer Quincy - MA - Massachusetts

Please strictly adhere to the following resume naming convention:

Role: Azure Devops Databricks platform Engineer

Skills: Azure Devops

Exp: 7+ Years

Key Responsibilities

Platform & Infrastructure

Design, deploy, and manage Databricks workspaces across multiple Azure subscriptions and environments (dev, staging, production)

Implement Infrastructure-as-Code (IaC) solutions using Terraform for Databricks resources

Establish and maintain security best practices, including network isolation, RBAC, and data governance

Configure and optimize cluster management, including auto-scaling policies and cost optimization strategies

Manage Databricks workspace connectivity and networking (VNets, private endpoints, firewalls)

DevOps & Automation

Build CICD pipelines using Azure DevOps for deploying Databricks notebooks, jobs, and configurations

Implement automated testing and validation of data pipelines

Create self-service deployment frameworks and templates for development teams

Establish monitoring, alerting, and logging solutions for Databricks environments

Automate workspace provisioning and deprovisioning processes

Collaboration & Enablement

Partner with data engineers and analysts to understand platform requirements

Provide technical guidance and documentation for platform usage

Develop runbooks and operational procedures for incident response

Mentor team members on Databricks best practices and Azure DevOps tools

Governance & Compliance

Implement access controls and audit logging across Databricks workspaces

Ensure compliance with organizational security and data governance policies

Monitor and manage workspace usage, costs, and resource utilization

Document architecture decisions and maintain platform documentation

Required Qualifications

7+ years of experience in platform engineering, infrastructure engineering, or DevOps

2+ years of hands-on experience with Databricks infrastructure setup (or Apache SparkDelta Lake environments)

3+ years of experience with Azure cloud services and infrastructure

2+ years of experience with Azure DevOps (Pipelines, Repos, Artifacts)

Strong proficiency in Infrastructure-as-Code tools (Terraform Enterprise)

Solid understanding of CICD principles and implementation

Experience with LinuxUnix environments and scripting (Bash, Python, or similar)

Familiarity with containerization technologies (Docker, Kubernetes)

Strong knowledge of networking, security, and identity management in cloud environments

Preferred Qualifications

Experience with Unity Catalog and Databricks governance features

Knowledge of Apache Spark architecture and optimization

Experience with other IaC tools (Terraform)

Familiarity with cost optimization strategies in cloud environments

Azure certifications (AZ-900, AZ-104, AZ-305)

Databricks Certification

Experience with Git and version control workflows

Knowledge of data governance and data mesh architectures

Experience with monitoring tools (Datadog, New Relic, Azure Monitor)