Start searching
Back to Jobs
Recently postedTechnology

Lead – Unity Catalog (Azure DevOps, GitHub, Atlassian, Jira)

eSolutionsFirst, LLCPhiladelphia, PA🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeHybrid
LocationPhiladelphia, PA, United States
Stakeholder ManagementGitApache SparkApacheDatabricksUnityAzureJiraCI/CDAzure DevOpsDelta Lake

Job description

Lead / Developer – Unity Catalog (Azure DevOps, GitHub, Atlassian, Jira)

Job Type: Contract (6-12 Months)

Location: Philadelphia, PA

Note: We need both Leads and Developers and must close as many positions as possible ASAP.

Job description:

Job Summary

  • We are looking for an experienced Lead or Developer with strong expertise in Unity Catalog, data governance, and enterprise data platform management.
  • The ideal candidate should have hands-on experience with Unity Catalog, access control, metadata management, data lineage, and governance within a Databricks environment.
  • The candidate will lead data governance initiatives, manage platform integrations, and collaborate with data engineering, DevOps, security, and business teams to establish scalable and secure data management practices.
  • Experience with Azure DevOps, GitHub, Atlassian, and Jira is required.
  • Experience in Wealth Management or Financial Services is highly preferred.

Key Responsibilities

  • Lead the implementation, administration, and management of Databricks Unity Catalog.
  • Establish and manage data governance policies, access controls, permissions, and security standards.
  • Implement and maintain data lineage, metadata management, and data discovery processes.
  • Manage Unity Catalog integration with Databricks workspaces and enterprise data platforms.
  • Work with Azure DevOps and GitHub to manage CI/CD pipelines, version control, and deployment processes.
  • Establish and manage development workflows using GitHub and Azure DevOps.
  • Utilize Jira and Atlassian tools for project tracking, sprint planning, task management, and team collaboration.
  • Collaborate with Data Engineering, Cloud, Security, and Architecture teams to implement governance standards.
  • Define best practices for data access management, auditability, and compliance.
  • Troubleshoot platform, deployment, access, and integration-related issues.
  • Lead technical discussions, mentor team members, and coordinate with stakeholders.
  • Ensure data governance practices align with enterprise security and regulatory requirements.

Required Qualifications

  • 8+ years of experience in Data Engineering, Data Platform Management, or related technical roles.
  • Strong hands-on experience with Databricks Unity Catalog.
  • In-depth understanding of data governance, metadata management, data lineage, and access controls.
  • Experience with role-based access control (RBAC), permissions, and data security.
  • Hands-on experience with Azure DevOps and GitHub.
  • Experience managing CI/CD pipelines, Git repositories, and deployment workflows.
  • Working experience with Atlassian tools, particularly Jira.
  • Strong understanding of cloud-based data platforms and enterprise data architecture.
  • Excellent leadership, communication, problem-solving, and stakeholder management skills.

Preferred Qualifications

  • Experience with Azure Databricks and Microsoft Azure cloud services.
  • Knowledge of Delta Lake, Apache Spark, and lakehouse architecture.
  • Experience implementing enterprise data governance frameworks.
  • Understanding of financial services, wealth management, and regulatory compliance requirements.
  • Experience leading technical teams and enterprise data platform initiatives.

Similar jobs

Apply