Databricks Architect

We are seeking an experienced Databricks Architect to design and lead enterprise-scale data and AI solutions using the Databricks Data Intelligence Platform. The ideal candidate will have strong hands-on experience with Databricks, Lakehouse architecture, Spark, Delta Lake, Unity Catalog, cloud platforms, and modern data engineering.

Key Responsibilities

• Design and implement enterprise Databricks Lakehouse architectures across development, test, and production environments.

• Define architecture for data engineering, analytics, BI, ML, and AI workloads.

• Architect solutions using Databricks, Delta Lake, Apache Spark, Unity Catalog, Databricks SQL, Workflows/Jobs, and Lakeflow.

• Lead migration of legacy data platforms, data warehouses, and Hive Metastore environments to Databricks/Unity Catalog.

• Design scalable batch and real-time/streaming data pipelines.

• Establish data governance, security, access control, lineage, and data-sharing strategies.

• Design cloud-native solutions across AWS, Azure, or Google Cloud Platform.

• Implement Terraform/IaC and CI/CD deployment strategies.

• Provide technical leadership for performance, scalability, reliability, and cost optimization.

• Conduct architecture reviews and provide technical guidance to data engineering and development teams.

• Work with business stakeholders to translate requirements into scalable technical solutions.

• Create architecture diagrams, technical documentation, standards, and implementation roadmaps.

Required Qualifications

• 12+ years of experience in Data Engineering, Data Architecture, Cloud Architecture, or related fields.

• 5+ years of strong Databricks experience, including architecture and implementation.

• Strong hands-on experience with:

  • Databricks Lakehouse

  • Apache Spark / PySpark

  • Delta Lake

  • Unity Catalog

  • Databricks SQL

  • Databricks Workflows / Jobs

  • Delta Live Tables / Lakeflow

  • Data governance and security

• Strong understanding of Data Lake, Data Warehouse, and Lakehouse architectures.

• Experience with enterprise-scale batch and streaming pipelines.

• Strong Python and/or Scala programming skills.

• Strong SQL development and performance-tuning experience.

• Experience with at least one major cloud platform: AWS, Azure, or Google Cloud Platform.

• Experience with cloud storage such as Amazon S3, Azure Data Lake Storage, or Google Cloud Storage.

• Experience with Terraform/IaC and CI/CD.

• Understanding of cloud networking, IAM, authentication, encryption, and security.

• Excellent communication, presentation, documentation, and stakeholder-management skills.