Haystack
← Back to Jobs
Remote
Other
SY

Sr. Databricks Architect

SysMind, LLCUnited States🇺🇸United StatesPosted Oct 1, 2026

Why This Role Stands Out

This remote Sr. Databricks Architect role offers a fantastic opportunity to lead impactful data platform designs and gain extensive experience with cutting-edge Databricks Lakehouse technology. You'll thrive here if you have a strong background in data engineering, cloud platforms, and a passion for client-facing technical leadership. Apply today to leverage your expertise and contribute to significant public sector projects.

Quick Overview

Seniority
Mid Senior
Work mode
Remote
Location
United States
Posted
19 hours ago
AWSETLMLOpsMLflowMachine LearningApacheApache SparkAzureDatabricksGitHub ActionsGitLab CIGoogle CloudJenkinsStakeholder ManagementUnity

Job Description

Location: Remote
Work Hours: Must be able to work in Pacific Time (PST)
Duration: 12 Months
Employment Type: Contract
Industry Experience: Public Sector experience required

Position Overview

We are seeking an experienced Databricks Architect / Resident Solution Architect to design, develop, and deliver modern, scalable data platforms and solutions. The ideal candidate will have extensive experience in data engineering, cloud data platforms, Databricks implementations, and client-facing technical leadership.

This role requires strong hands-on expertise with Databricks and Apache Spark, along with the ability to translate complex business and technical requirements into scalable, secure, and cost-effective data architecture.

Primary Skills

  • 12–15+ years of experience in Data Engineering, Data Platforms, Data Analytics, and Modern Data Warehouse solutions
  • 10+ years of consulting and client-facing technical delivery experience
  • Proven experience delivering 6–8+ end-to-end Databricks implementations
  • Strong hands-on experience as a Databricks Developer, Technical Lead, or Solution Architect
  • Advanced expertise in Databricks Lakehouse Platform
  • Strong knowledge of Apache Spark, including:
    • Performance optimization
    • Partitioning strategies
    • Execution plans
    • Memory management
    • Spark runtime internals
  • Extensive experience building scalable ETL/ELT pipelines
  • Hands-on experience with:
    • Databricks
    • Delta Lake
    • Structured Streaming
    • Modern data engineering frameworks
  • Strong experience designing and implementing cloud-native data platforms across AWS, Azure, and/or Google Cloud Platform
  • Deep hands-on proficiency with at least one major cloud platform
  • Strong experience tuning large-scale distributed data workloads
  • Ability to design highly performant, scalable, and cost-efficient data processing solutions
  • Experience troubleshooting complex data platform challenges and recommending appropriate architecture patterns

Secondary Skills

  • Databricks Data Engineering Professional Certification or equivalent advanced Databricks certification
  • Completion of relevant Databricks learning paths and coursework
  • Databricks Solutions Architect Champion experience/program participation
  • DevOps and CI/CD experience for production-grade data solutions
  • Experience with:
    • Azure DevOps
    • GitHub Actions
    • GitLab CI/CD
    • Jenkins
  • Working knowledge of MLOps
  • Experience with machine learning lifecycle management, model deployment, and monitoring
  • Experience with:
    • Unity Catalog
    • Databricks Workflows
    • MLflow
    • Delta Live Tables
    • Other Databricks platform capabilities
  • Strong understanding of enterprise data architecture, governance, scalability, security, and reliability

Required Qualifications

  • 12–15+ years of relevant data engineering/data platform experience
  • Strong Databricks implementation background
  • Extensive Apache Spark experience
  • Public sector experience
  • Strong client-facing consulting experience
  • Excellent communication and stakeholder management skills
  • Ability to work remotely while supporting Pacific Time (PST) business hours
  • English language proficiency

Key Responsibilities

  • Lead the architecture and implementation of enterprise Databricks solutions
  • Design scalable Lakehouse and modern data platform architectures
  • Provide technical leadership across data engineering and analytics initiatives
  • Develop and optimize ETL/ELT pipelines using Databricks, Spark, Delta Lake, and related technologies
  • Establish performance, scalability, reliability, and cost-optimization strategies
  • Guide CI/CD and DevOps practices for data platform deployments
  • Troubleshoot complex data engineering and platform issues
  • Collaborate with business, engineering, data science, and technology stakeholders
  • Provide architecture recommendations aligned with business objectives and technical requirements
  • Support enterprise adoption of modern Databricks capabilities and best practices

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, sexual orientation, and gender identity), national origin, age, disability, genetic information, protected veteran status, or any other characteristic protected by applicable federal, state, or local law. 
We are committed to providing reasonable accommodations to qualified applicants with disabilities and for sincerely held religious beliefs, as required by applicable law. Applicants who require a reasonable accommodation during the hiring process may request assistance.
Employment may be contingent upon the successful completion of job-related pre-employment screening, which may include background and reference checks and, where permitted by applicable law and applicable to the position, drug testing. All such screening will be conducted in accordance with applicable federal, state, and local laws.

Similar jobs

Browse related jobs

Apply now