Azure Databricks Infrastructure Engineer

We are seeking a Senior Azure Databricks Infrastructure Engineer with strong experience in Azure, Databricks, cloud security, networking, identity, and Infrastructure as Code. This role will lead Azure Databricks tenant-to-tenant migrations, from assessment and planning through production cutover and handover.

Key Responsibilities

• Lead Azure Databricks workspace and platform migrations between Azure tenants.
• Assess and plan migration dependencies across networking, DNS, Private Endpoints, identity, security, and data.
• Migrate and configure Databricks workspaces, jobs, workflows, repositories, compute policies, permissions, secret scopes, notebooks, libraries, and configurations.
• Manage Microsoft Entra ID, service principals, managed identities, RBAC, Key Vault, and Databricks access controls.
• Support Unity Catalog, Delta Lake, ADLS Gen2, and data/storage integrations.
• Develop migration wave plans, cutover/rollback strategies, runbooks, and operational handover documentation.
• Automate infrastructure and migration activities using Terraform, Bicep/ARM, Azure CLI, PowerShell, and Databricks APIs/CLI.
• Partner with architecture, security, platform, data engineering, and business teams to ensure secure and low-risk migrations.
• Ensure solutions follow enterprise security, governance, compliance, and Principle of Least Privilege standards.

Required Experience & Skills

• 12+ years of Azure/cloud experience.
• 7+ years of Azure Databricks administration/engineering experience.
• Proven experience with Azure tenant-to-tenant migrations and large-scale data platform migrations.
• Strong knowledge of:
  • Azure Databricks & Unity Catalog
  • Microsoft Entra ID, Azure RBAC & Managed Identities
  • Azure VNet, Private Endpoints & Key Vault
  • ADLS Gen2, Delta Lake & Azure Storage
  • Databricks Jobs, Repos, Cluster Policies, Secret Scopes & REST APIs
  • Terraform / Bicep / ARM
  • Azure DevOps / GitHub Actions
  • Azure CLI / PowerShell / Databricks CLI
• Experience working in enterprise and regulated environments.
• Strong documentation, communication, troubleshooting, and stakeholder-management skills.

Preferred

• Azure Solutions Architect Expert certification.
• Databricks Administrator or Solution Architect certification.
• Experience with Unity Catalog migrations.
• Experience supporting M&A, divestitures, or tenant transformation projects.