Azure Databricks Engineer

Position: Senior Azure Databricks Engineer

Working Location: Glendale AZ

Duration : 12 Months

Experience: 8 to10 years

Requirements

·       At least 8-10 years of experience in designing and implementing enterprise data platforms;

·       At least 5 years of hands-on experience hands-on Azure Databricks experience in enterprise environments;

·       Experience with architecting modern Lakehouse solutions using Azure Databricks and Delta Lake;

·       Experience in administering Azure Databricks workspaces, clusters, Unity Catalog, compute policies, job orchestration, and workspace security;

·       Experience in developing data engineering solutions using Apache Spark (PySpark and/or Spark SQL);

·       Experience in implementing enterprise data governance utilizing Unity Catalog,  metadata management, data lineage, and fine-grained access controls;

·       Experience designing and working with enterprise security architectures including RBAC, Managed Identities, Private Endpoints, Key Vault integration, encryption, and network security;

·       Experience implementing enterprise data quality frameworks, monitoring, observability, and operational dashboards;

·       Strong understanding of data modelling techniques including dimensional modelling, Data Vault, medallion architecture, and enterprise data warehouse concepts;

·       Strong knowledge of Azure Data Lake Storage Gen2, Azure Key Vault, Azure Monitor, Microsoft Entra ID (Azure AD), and Azure networking;

·       Strong SQL programming skills and proficiency in Python and PySpark development

·       Strong experience implementing CI/CD pipelines utilizing Azure DevOps, automated testing, release management, and deployment automation