Start searching
Back to Jobs
Recently postedTechnology

Databricks Architect (Resident Solution Architect)

SysMind, LLCUnited States🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeHybrid
LocationUnited States
GitApache SparkAWSETLMLOpsMLflowApacheDatabricksGitHub ActionsGitLab CIGoogle CloudJenkinsUnityAzureMachine LearningData AnalysisCI/CDAzure DevOpsDelta Lake

Job description

Location: San Francisco, CA (Remote)
Work Arrangement: Remote
Employment Type: Contract
Duration: 8–12 Months
Experience Required: 12–15+ Years

Position Overview

We are seeking experienced Databricks Solution Architects with strong banking industry experience and expertise in enterprise data engineering, cloud data platforms, and modern data warehouse solutions. The ideal candidates will have successfully delivered multiple end-to-end Databricks implementations and possess hands-on architecture and development experience. This role requires deep knowledge of the Databricks Lakehouse Platform, Apache Spark, Delta Lake, cloud-native data solutions, and enterprise data platform modernization.

Primary Skills

Databricks Architecture

Databricks Lakehouse Platform

Apache Spark

Data Engineering

Delta Lake

Cloud Data Platforms

Secondary Skills

Structured Streaming

ETL/ELT Pipelines

Unity Catalog

Databricks Workflows

MLflow

Delta Live Tables

MLOps

DevOps and CI/CD

AWS

Microsoft Azure

Google Cloud Platform (Google Cloud Platform)

Key Responsibilities

  • Design and implement scalable, cloud-native data platforms using Databricks.

  • Lead end-to-end Databricks implementations, from architecture and design through development and production deployment.

  • Develop and optimize large-scale ETL/ELT pipelines using Databricks, Delta Lake, Structured Streaming, and modern data engineering frameworks.

  • Design enterprise data architecture aligned with banking business requirements and technical standards.

  • Optimize Apache Spark workloads through partitioning strategies, execution plan analysis, memory management, and performance tuning.

  • Implement and manage Databricks Lakehouse capabilities, including Unity Catalog, Workflows, MLflow, and Delta Live Tables.

  • Establish DevOps and CI/CD practices for production-grade data engineering solutions.

  • Apply MLOps principles to support machine learning lifecycle management, model deployment, and monitoring.

  • Troubleshoot complex data platform challenges and recommend scalable, reliable, and cost-efficient architecture solutions.

  • Provide technical leadership and architectural guidance to engineering and delivery teams.

Required Qualifications

  • 12–15+ years of experience in data engineering, data platforms, data analytics, and modern data warehouse solutions.

  • 10+ years of consulting and client-facing delivery experience.

  • Demonstrated success delivering at least 6–8 end-to-end Databricks implementations.

  • Hands-on experience as a Databricks developer, technical lead, or solution architect.

  • Strong expertise in cloud-native data platform architecture across AWS, Azure, or Google Cloud Platform.

  • Deep hands-on proficiency in at least one major cloud platform.

  • Advanced knowledge of Apache Spark, including performance optimization, partitioning, execution plans, memory management, and runtime internals.

  • Extensive experience with Databricks, Delta Lake, Structured Streaming, and scalable ETL/ELT pipelines.

  • Experience implementing CI/CD and DevOps practices using Azure DevOps, GitHub Actions, GitLab CI/CD, or Jenkins.

  • Working knowledge of MLOps, machine learning lifecycle management, model deployment, and monitoring.

  • Strong understanding of enterprise data architecture, distributed processing, and performance optimization.

  • Relevant experience in the banking or financial services industry.

  • Strong problem-solving and solution architecture capabilities.

Preferred Qualifications

  • Databricks Data Engineering Professional certification or an equivalent advanced Databricks certification.

  • Completion of recommended Databricks learning paths and coursework.

  • Databricks Solutions Architect Champion program participation.

  • Experience with advanced Databricks Lakehouse Platform capabilities and enterprise implementations.


All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, sexual orientation, and gender identity), national origin, age, disability, genetic information, protected veteran status, or any other characteristic protected by applicable federal, state, or local law.
We are committed to providing reasonable accommodations to qualified applicants with disabilities and for sincerely held religious beliefs, as required by applicable law. Applicants who require a reasonable accommodation during the hiring process may request assistance.
Employment may be contingent upon the successful completion of job-related pre-employment screening, which may include background and reference checks and, where permitted by applicable law and applicable to the position, drug testing. All such screening will be conducted in accordance with applicable federal, state, and local laws.

Similar jobs

Apply