Azure Databricks Architect

HMG America LLC is the best Business Solutions focused Information Technology Company with IT consulting and services, software and web development, staff augmentation and other professional services. One of our direct clients is looking for Azure Databricks Architect in Seattle, WA. Below is the detailed job description.

Job Title: Azure Databricks Architect

Location: Seattle, WA (Hybrid 3 Days Onsite)
Duration:Full-Time

Job Summary

We are seeking an experienced Azure Databricks Architect to lead the design, implementation, and optimization of enterprise-scale data platforms on Azure. The ideal candidate will have deep expertise in Azure Databricks, Spark, Delta Lake, and modern Lakehouse architecture, with strong experience designing secure, scalable, and high-performance data solutions. Knowledge of Unity Catalog, Medallion Architecture, and Infrastructure as Code is highly preferred. These are all considered current best practices for enterprise Azure Databricks environments.

Required Skills

• 12+ years of IT experience with 5+ years as a Data/Cloud Architect.
• Strong hands-on experience with Azure Databricks.
• Expertise in PySpark, Spark SQL, Python, and SQL.
• Strong knowledge of Delta Lake, Lakehouse Architecture, and Medallion (Bronze/Silver/Gold) Architecture.
• Experience with Unity Catalog, data governance, RBAC, and data lineage.
• Experience designing scalable ETL/ELT pipelines.
• Hands-on experience with Azure Data Factory (ADF).
• Strong experience with Azure Data Lake Storage (ADLS Gen2).
• Experience with Azure Synapse Analytics or Microsoft Fabric is a plus.
• Experience with CI/CD using Azure DevOps, Git, and Infrastructure as Code (Terraform/Bicep/ARM).
• Strong understanding of performance tuning, cluster optimization, partitioning, caching, and cost optimization.
• Experience implementing data quality, monitoring, logging, and observability.
• Knowledge of streaming technologies (Structured Streaming/Event Hubs/Kafka) is preferred.
• Excellent communication and stakeholder management skills.

Responsibilities

• Design and architect enterprise-scale Azure Lakehouse solutions.
• Define end-to-end data architecture, ingestion, transformation, and serving layers.
• Lead Azure Databricks implementation using best practices.
• Design secure data governance using Unity Catalog and role-based access controls.
• Optimize Spark workloads for performance and cost.
• Implement reusable frameworks, coding standards, and CI/CD pipelines.
• Work closely with business, engineering, and cloud infrastructure teams.
• Provide architectural guidance, code reviews, and technical leadership.
• Ensure scalability, reliability, security, and disaster recovery across the platform.

Nice to Have

• Databricks Certified Professional/Architect certification.
• Azure Solutions Architect or Azure Data Engineer certification.
• Experience with MLflow, MLOps, or AI/GenAI workloads on Databricks.
• Experience with Power BI integration.