Azure Databricks Admin

Position: Azure Databricks Admin
Type/Duration: Full time//Permanent
Location: Plano, TX/Richardson, TX (5 Days onsite in a week)
 

Required Skill and Experience

• Architecture and administration experience with Microsoft Azure Databricks
• Experience with platform automation, DevOps, and CI/CD pipelines
• Knowledge of cloud security, networking, and identity management
• Establish platform monitoring, observability, and automated alerting frameworks for Azure Databricks platform components
• Platform automation through Infrastructure-as-Code using Terraform, DevOps, and CI/CD pipelines.
• Implement capacity planning, disaster recovery, and high availability strategies.
• Expertise in managing Unity Catalog, and data access control mechanisms.
• Experience of managing Databricks cluster policies, instance pools, autoscaling, auto-termination, and compute governance
• Support platform upgrades, runtime migrations, and feature enablement.
• Administer Azure Databricks workspaces across Development, QA, Production & DR environments.
• Integrate Azure Databricks with Microsoft Entra ID (Azure AD).
• Manage RBAC, SCIM provisioning, user lifecycle management, and group synchronization.
• Implement and maintain Unity Catalog permissions and governance policies.
• Configure Key Vault-backed secret scopes and credential management.
• Support secure networking using:
• Private Endpoints
• VNET Injection
• NSGs
• Firewall Rules
• Managed Identities
• Implement enterprise governance standards using Unity Catalog.
• Maintain audit logging and compliance reporting.
• Troubleshoot platform, cluster, notebook, and job execution issues.
• Establish operational runbooks, incident management procedures, and SLA monitoring.
• Implement cluster policies to control costs and prevent resource overprovisioning.

Preferred Skill and Experience

• Manage platform operations, incident management, and change governance
• Design scalable, resilient, and secure platform solutions aligned with enterprise architecture standards.
• Establish best practices for platform architecture, governance, and security.
• Certifications in cloud architecture (Azure, Databricks) preferred
• ITIL knowledge for service management and operations governance
• Deliver platform health metrics, risk assessments, and operational reports.
• Ensure adherence to SLA, compliance, and operational standards
• Experience of supporting migration of legacy systems (like HMS) to Databricks on cloud platforms like Azure, AWS, or Google Cloud Platform.
• Skillset of integrating guidelines to configure and integrate Databricks with other existing security tools relevant to data access control.
• Experience in Azure Databricks, real time data ingestion, data warehouses, serverless ETL, No SQL databases, DevOps, Kubernetes, virtual machines, web/function apps, monitoring, and security tools.
• Optimize cluster utilization and DBU consumption.
• Utilize autoscaling, instance pools, spot instances, and workload optimization techniques.
• Produce usage, cost, and governance dashboards for stakeholders.
• Provide guidance on Databricks best practices and platform standards.
• Support onboarding of new projects and workloads onto the platform.
• Managed Kubernetes cluster and workload including network architecture

 

Azure Databricks Admin1Azure,admin,DatabricksN/AFull TImeUnited States