Databricks Solution Architect

About the Role

We are seeking an experienced Databricks Resident Solutions Architect (RSA) to lead the architecture, implementation, and optimization of enterprise Databricks Lakehouse solutions. This is a hands-on, customer-facing role requiring deep Databricks expertise, strong architecture skills, and the ability to guide engineering and data teams.

Key Responsibilities

• Lead the architecture and delivery of enterprise Databricks Lakehouse implementations.
• Design scalable solutions using Databricks, Delta Lake, Apache Spark, Unity Catalog, and Databricks Workflows.
• Provide hands-on technical leadership for complex Databricks data engineering projects.
• Optimize Spark and Databricks workloads for performance, scalability, reliability, and cost.
• Design and implement enterprise ETL/ELT pipelines using Databricks, Spark, Python, and SQL.
• Establish Unity Catalog governance, security, access controls, and data management practices.
• Support migration and modernization of legacy data platforms to Databricks Lakehouse.
• Implement CI/CD and DevOps practices for Databricks deployments.
• Partner with data science teams on MLflow, MLOps, machine learning, and GenAI/LLM workloads.
• Troubleshoot complex Databricks, Spark, Delta Lake, and pipeline issues.
• Conduct architecture workshops and advise customers on Databricks best practices and roadmap.
• Mentor engineering teams and serve as a trusted technical advisor to customer stakeholders.

Required Qualifications

• 7+ years of experience in data engineering, data architecture, or cloud data platforms.
• 3+ years of hands-on Databricks experience with multiple enterprise implementations.
• Strong expertise in:
• Databricks Lakehouse Platform
• Delta Lake
• Apache Spark
• Unity Catalog
• Databricks Workflows/Jobs
• Databricks SQL
• ETL/ELT and data engineering
• Strong Python, SQL, and PySpark skills.
• Experience with Azure Databricks, AWS Databricks, or Databricks on Google Cloud Platform.
• Strong understanding of Spark performance tuning and distributed processing.
• Experience with CI/CD, MLOps, and MLflow.
• Experience with Databricks governance, security, and data architecture.
• Strong customer-facing consulting, communication, and presentation skills.
• U.S. work authorization without current or future visa sponsorship.

Preferred Qualifications

• Databricks Certified Data Engineer Professional certification.
• Databricks Solutions Architect certification.
• Experience with Generative AI, LLMs, Vector Search, Model Serving, and Mosaic AI.
• Experience with Structured Streaming and real-time data processing.
• Experience migrating enterprise data warehouses/data lakes to Databricks.
• Experience in consulting or professional services environments.