Haystack
← Back to Jobs
Engineering
SM

Sr. Databricks Architect/Lead/Engineer

SmartIMS Inc.Houston, TX🇺🇸United StatesPosted 25 Aug 2026

Why This Role Stands Out

You'll have the opportunity to shape cutting-edge data architectures and lead impactful projects within a hybrid work environment, fostering significant career growth and skill development. This role is ideal for experienced data architects who excel at designing robust, secure, and scalable solutions and are eager to mentor others. Embrace this chance to make a substantial impact and advance your expertise.

Quick Overview

Seniority
Mid Senior
Work mode
Hybrid
Location
Houston, TX, United States
Posted
Yesterday
Stakeholder ManagementUnity

Job Description

Title: Databricks Architect/Lead/Engineer
Location:Houston, TX — candidate must be based in Houston (local candidates only)
Work Model:Onsite, 3 days per week in office (hybrid)
Engagement Type: Contract
Duration: 2 years (with potential to extend up to 5 years)

Description:
•Design the overall Databricks Lakehouse architecture, including ingestion, storage (Delta Lake), processing, and consumption layers.
•Define reference architectures, design patterns, and best practices for data engineering, analytics, and ML workloads.
•Lead decisions on workspace organization, compute strategy (clusters, SQL warehouses, serverless), and cost optimization.
•Architect data governance and security using Unity Catalog, including access controls, lineage, and data classification.
•Establish CI/CD, DevOps, and Infrastructure-as-Code standards for the platform (e.g., Terraform, Databricks Asset Bundles).
•Guide integration with cloud services (AWS / Azure / Google Cloud Platform) and source systems.
•Partner with stakeholders, security, and leadership to translate business requirements into technical roadmaps.
•Provide technical oversight and mentorship to tech leads and engineers.

Required Qualifications
•10+ years in data architecture / data engineering, with 4+ years hands-on with Databricks.
•Deep expertise in Delta Lake, Spark (PySpark/Scala), and the Lakehouse architecture.
•Strong knowledge of Unity Catalog, data governance, and security models.
•Experience architecting on at least one major cloud (AWS, Azure, or Google Cloud Platform).
•Proven track record designing large-scale, production data platforms.
•Strong stakeholder management and the ability to communicate architecture to both technical and business audiences.

Preferred
•Databricks certifications (e.g., Databricks Certified Data Engineer Professional, Solutions Architect).
•Experience with streaming (Structured Streaming, Kafka), MLflow, and lakehouse ML workflows.
•Background in oil & gas, energy, or other large enterprise environments.

Similar jobs

Browse related jobs

Apply now