Databricks Data Engineer (W2 Role)

Title: Databricks Data Engineer

Location: Charlotte, NC (two days hybrid)

Interview: Teams with final round in person possible (must be agreed upon at submittal)

Duration: 12 months +
Position:

Databricks Engineer to design, build, and maintain scalable data pipelines, lakehouse architectures, and analytics solutions using Microsoft Azure and the Databricks platform. 

Key Responsibilities

• Pipeline Development: Design and build Extract, Transform, Load (ETL) and ELT data pipelines using Azure Data Factory (ADF) and Azure Databricks
• Big Data Processing: Write optimized data processing code using Apache Spark, PySpark, Python, Scala, and advanced SQL.
• Storage & Architecture: Implement and manage data lakes using Azure Data Lake Storage (ADLS Gen2) and Delta Lake / lakehouse patterns.
• Governance & Security: Configure data governance, lineage, and access controls using Databricks Unity Catalog and Azure IAM.
• Optimization & Operations: Monitor, troubleshoot, and tune Spark jobs and data pipelines for performance, reliability, and cost-efficiency.
• CI/CD & DevOps: Support deployment processes utilizing Azure DevOps, Git version control, and infrastructure-as-code tools like Terraform.

Required Skills & Qualifications

• Experience: 3 to 7+ years of total experience in data engineering or cloud-based data platform development.
• Core Technologies: Hands-on proficiency in Azure Databricks, Apache Spark, PySpark, and Azure Data Factory (ADF).
• Languages: Advanced SQL and solid programming skills in Python or Scala.
• Data Concepts: Strong grasp of Delta Lake, relational/NoSQL data storage, data modeling, and data quality standards.