← Back to Jobs

Technology

AS

Lead Data Engineer with Databricks

ALIS SoftwareCharlotte, NC🇺🇸United StatesPosted Oct 22, 2026

Quick Overview

Seniority

Mid Senior

Work mode

Hybrid

Location

Charlotte, NC, United States

Posted

20 hours ago

Apache SparkAWSETLDatabricksGoogle CloudPythonUnityAzureDelta Lake

Job Description

Job Description

Lead Data Engineer with Databricks
Location: Hybrid – Charlotte, NC

Roles & Responsibilities

Develop high-quality, scalable ETL/ELT pipelines using Databricks technologies including Delta Lake, Auto Loader etc.,
Excellent programming and debugging skills in Python
Strong hands-on experience with PySpark to build efficient data transformation and validation logic
Must be proficient in at least one cloud platform: AWS, Google Cloud Platform, or Azure
Need someone with Insurance domain experience - property and casualty
Implement ingestion patterns using Auto Loader with checkpointing and schema evolution for structured and semi-structured data
Build secure and observable DLT pipelines with DLT Expectations, supporting Bronze/Silver/Gold medallion layering
Configure Unity Catalog: set up catalogs, schemas, user/group access, enable audit logging, & define masking for PII fields
Enable secure data access across domains & workspaces via Unity Catalog External Locations, Volumes, & lineage tracking
Access and utilize data assets from the Databricks Marketplace to support enrichment, model training, or benchmarking
Collaborate with data sharing stakeholders to implement Delta Sharing — both internally and externally

Similar jobs