Back to Jobs
Recently postedTechnology

Databricks Platform Engineer- Unity Catalog and DevOps

eSolutionsFirst, LLCPhiladelphia, PA🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeHybrid
LocationPhiladelphia, PA, United States
SQLGitApache SparkDatabricksGitHub ActionsPythonTerraformUnityVaultAzurePower BIAgileCI/CDAzure DevOpsDelta Lake

Job description

Databricks Platform Engineer- Unity Catalog and DevOps

Job Type: Contract (6-12 Months) possible extension

Location: Philadelphia, PA - (Hybrid – Flexible Work Model)

Job description:

Job summary:

  • Program: Consolidation of five legacy data and BI platforms onto Azure Databricks and Power BI
  • We are seeking a platform engineer to build and automate an Azure Databricks environment governed by Unity Catalog.
  • Working under the platform lead, you will configure catalogs and permissions as code, build deployment pipelines, and support data engineering teams as they migrate workloads onto the platform.

Key responsibilities:

  • Configure Unity Catalog objects as code: catalogs, schemas, external locations, storage credentials and grants.
  • Implement access controls, including row filters and column masks, to the lead's design.
  • Build and maintain CI/CD pipelines for platform configuration, pipelines and jobs.
  • Manage repositories, branching and release promotion across environments.
  • Automate workspace, cluster policy and job configuration.
  • Monitor usage, cost and access using Databricks system tables.
  • Troubleshoot access, deployment and integration issues for engineering teams.
  • Write runbooks and support testing and production deployments.

Required qualifications:

  • 5+ years in data platform, DevOps or data engineering roles, including 2+ years on Databricks.
  • 1+ year of hands-on Unity Catalog configuration in a production environment.
  • Terraform, or Databricks Asset Bundles, for deploying Databricks resources.
  • CI/CD on Azure DevOps or GitHub Actions, with Git branching and pull-request workflows.
  • Working Python and SQL.
  • Azure fundamentals: Entra ID, ADLS Gen2, Key Vault.

Preferred qualifications:

  • Experience in financial services or another regulated industry.
  • PySpark, Delta Lake and Lakeflow.
  • Databricks Data Engineer Associate certification.
  • Experience working in an Agile team.
  • AI coding and analysis tools may be used only on endpoints the client has approved.
  • Experience in financial services, wealth management or another regulated industry is strongly preferred.

Similar jobs