Haystack
← Back to Jobs
Technology
ST

Databricks Platform Engineer

SR Talent Solution IncHouston, TX🇺🇸United StatesPosted Oct 5, 2026

Quick Overview

Seniority
Mid Senior
Work mode
Hybrid
Location
Houston, TX, United States
Posted
21 hours ago
SQLAWSApacheApache SparkAzureDatabricksGoogle CloudKafkaTerraformUnity

Job Description

Databricks Platform Engineer
Locations: New York, NY; Houston, TX; Lisle, IL; Atlanta, GA; Austin, TX; Bellevue, WA; Bridgewater, NJ; Somerset, NJ; Charlotte, NC; Franklin, TN; Milwaukee, WI; Rogers, AR; San Ramon, CA; Wayne, PA.
Duration: Long term contract 

This opportunity is ideal for someone who operates beyond traditional data engineering and has hands-on experience owning the Databricks platform itself. The successful candidate will establish platform standards, automate deployments, strengthen governance and security, optimize performance and cloud consumption, and provide engineering teams with a reliable foundation for enterprise analytics and data workloads.
We are looking for a hands-on engineer who understands how Databricks functions as an enterprise platform—from workspace architecture and cluster management through Unity Catalog, Infrastructure as Code, CI/CD, security, governance, observability, and cost optimization.
What You'll Be Doing

  • Design, configure, administer, and continuously improve enterprise Databricks environments.

  • Manage Databricks workspaces, clusters, compute policies, jobs, permissions, configurations, and platform services.

  • Establish reusable platform standards, architecture patterns, guardrails, and engineering best practices.

  • Own and support Unity Catalog, including catalogs, schemas, permissions, RBAC, data access policies, governance, and lineage.

  • Build and maintain automated infrastructure and deployment processes using Terraform and Infrastructure as Code.

  • Develop and support CI/CD pipelines for Databricks configurations, notebooks, jobs, workflows, and platform components.

  • Partner with data engineers, analytics engineers, architects, security teams, and application teams to enable scalable development.

  • Integrate Databricks with cloud storage, data lakes, enterprise applications, BI platforms, orchestration tools, and other data services.

  • Support and optimize Apache Spark workloads across large-scale distributed environments.

  • Monitor platform health, cluster utilization, workload performance, reliability, and cloud consumption.

  • Troubleshoot performance, configuration, deployment, access, and integration issues.

  • Automate repetitive administration and operational processes.

  • Implement platform controls that support enterprise security, compliance, governance, and operational standards.

  • Drive ongoing improvements around performance, scalability, reliability, automation, and cost optimization.
What We're Looking For Required Experience

  • 5+ years of experience in Data Engineering, Data Platform Engineering, Cloud Platform Engineering, or Data Platform Administration.

  • 3+ years of hands-on Databricks experience focused on administration, platform engineering, architecture, or enterprise platform ownership.

  • Strong hands-on experience with Azure Databricks.

  • Deep understanding of Apache Spark, distributed computing, workload tuning, and performance optimization.

  • Hands-on experience implementing and managing Unity Catalog.


  • Strong knowledge of:

  • Role-Based Access Control (RBAC)

  • Identity and access management

  • Data governance

  • Platform security

  • Permissions and access policies

  • Experience building infrastructure using Terraform or comparable Infrastructure-as-Code technologies.

  • Experience designing or supporting CI/CD pipelines and automated Databricks deployments.

  • Strong understanding of cloud data architecture and services within Azure; experience with AWS or Google Cloud Platform is also valuable.

  • Experience operating enterprise data platforms where reliability, security, scalability, and governance are critical.

  • Bachelor's degree in Computer Science, Engineering, Data Science, Information Technology, or a related technical discipline.
Highly Desired

  • Experience designing or supporting a Lakehouse Architecture.

  • Strong experience with Delta Lake.

  • Experience with Databricks Workflows, Jobs, cluster policies, SQL warehouses, or related platform capabilities.

  • Experience integrating Databricks with enterprise data lakes and cloud storage.

  • Knowledge of Structured Streaming, Kafka, Azure Event Hubs, or other streaming technologies.

  • Experience implementing observability, logging, monitoring, alerting, and automated operational controls.

  • Experience with Databricks platform performance and FinOps / cloud cost optimization.

  • Exposure to secrets management, service principals, private networking, or enterprise cloud security patterns.
Certifications That Stand Out
Relevant certifications are valuable but not required, including:

  • Databricks Certified Data Engineer Professional

  • Databricks Certified Data Engineer Associate

  • Databricks platform-related certifications

  • Microsoft Certified: Azure Data Engineer

  • Microsoft Certified: Azure Solutions Architect

  • Comparable AWS or Google Cloud Platform cloud certifications

Similar jobs

Browse related jobs

Apply now