Recently postedRemoteTechnology

Senior Databricks Solution Architect – Lakehouse & Data Engineering | Contract | Remote (PST)

Anagha Techno SoftUnited States🇺🇸United States

Quick overview

SeniorityMid Senior

Work modeRemote

LocationUnited States

SQLGitApache SparkAWSETLMLOpsMLflowApacheDatabricksGitHub ActionsGitLab CIGoogle CloudJenkinsUnityAzureMachine LearningData AnalysisCI/CDAzure DevOpsDelta Lake

Job description

Senior Databricks Solution Architect – Lakehouse & Data Engineering | Contract | Remote (PST)

Senior Databricks Solution Architect – Lakehouse & Data Engineering

Location: San Francisco, CA (Remote Considered)
Job Type: Contract
Experience: 12–15+ Years
Time Zone: Pacific Time

Ideal Candidate Profile

A hands-on architect who can confidently lead client conversations, make sound architecture decisions, build and optimize production solutions, and guide engineering teams through complex Databricks implementations.

Job Summary

We are seeking an experienced Senior Databricks Solution Architect with a strong combination of enterprise solution architecture, hands-on data engineering, and client-facing consulting experience.

The ideal candidate will have successfully designed, implemented, and optimized multiple enterprise-scale Databricks solutions, with deep expertise in Apache Spark, Delta Lake, Lakehouse architecture, and cloud-native data platforms. This role requires someone who can lead architecture discussions while remaining actively involved in technical design, development, performance tuning, and troubleshooting.

Key Responsibilities

Design and implement scalable, secure, and cost-efficient enterprise data platforms using the Databricks Lakehouse Platform.
Lead end-to-end Databricks implementations, from architecture and solution design through development, deployment, and production optimization.
Develop and optimize scalable ETL/ELT pipelines using PySpark, Apache Spark, Delta Lake, and Structured Streaming.
Design cloud-native data solutions across AWS, Microsoft Azure, and/or Google Cloud Platform.
Optimize distributed data processing through partitioning strategies, execution-plan analysis, memory management, and performance tuning.
Implement platform capabilities including Unity Catalog, Databricks Workflows, Delta Live Tables/Lakeflow, and MLflow.
Establish DevOps and CI/CD practices using Azure DevOps, GitHub Actions, GitLab CI/CD, Jenkins, or similar tools.
Apply data governance, security, monitoring, reliability, and cost-optimization best practices.
Provide hands-on technical leadership, troubleshoot complex platform challenges, and recommend appropriate architecture patterns.
Lead client workshops, communicate architectural decisions, and translate business requirements into technical solutions.

Required Qualifications

12–15+ years of experience in data engineering, data platforms, data analytics, or modern data warehouse solutions.
10+ years of consulting, client-facing delivery, or enterprise technology implementation experience.
Demonstrated success delivering multiple end-to-end Databricks implementations, ideally 6–8 or more.
Strong hands-on proficiency in at least one major cloud platform: AWS, Azure, or Google Cloud Platform.
Advanced knowledge of Apache Spark, PySpark, Spark SQL, distributed computing, and performance optimization.
Extensive experience with Databricks, Delta Lake, scalable ETL/ELT pipelines, and modern data engineering frameworks.
Strong understanding of Lakehouse architecture, Unity Catalog, workflow orchestration, and production-grade data platform design.
Experience with CI/CD, DevOps, production monitoring, and deployment automation.
Working knowledge of MLOps, MLflow, machine learning lifecycle management, and model monitoring.
Excellent analytical, troubleshooting, communication, and technical leadership skills.

Preferred Qualifications

Databricks Certified Data Engineer Professional certification or an equivalent advanced Databricks credential.
Databricks Solutions Architect Champion recognition.
Experience in public sector, banking/financial services, or healthcare environments.
Experience with large-scale platform modernization, data governance, and enterprise cloud migrations.