Databricks Data Engineer -- W2 Profiles -- Onsite

Role: Databricks Data Engineer

Location: Charlotte, NC - Onsite

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using Databricks and PySpark.
• Build and optimize data processing solutions using Apache Spark, Python, and SQL.
• Implement Delta Lake tables and support schema evolution, data quality, and ACID transactions.
• Develop Bronze, Silver, and Gold data layers following the Medallion/Lakehouse architecture.
• Build batch and near-real-time data pipelines using Spark and Databricks.
• Develop and manage Databricks Jobs and Workflows.
• Integrate data from databases, APIs, cloud storage, and other enterprise data sources.
• Perform data transformation, cleansing, validation, and enrichment.
• Optimize Spark applications, SQL queries, and data pipelines for performance and cost.
• Implement data governance, security, and access controls using Unity Catalog.
• Monitor production pipelines, troubleshoot failures, and resolve data-quality issues.
• Collaborate with Data Architects, Analysts, Developers, and business stakeholders.
• Follow development best practices for version control, testing, deployment, and documentation.

Required Skills

• 8+ years of experience in Data Engineering.
• Strong hands-on experience with Databricks.
• Strong proficiency in PySpark / Apache Spark.
• Advanced SQL skills.
• Strong programming experience with Python.
• Hands-on experience with Delta Lake.
• Experience with ETL/ELT pipeline development.
• Knowledge of Lakehouse and Medallion architecture.
• Experience with Databricks Workflows/Jobs.
• Experience with Unity Catalog and data governance.
• Strong understanding of data modeling and data warehousing concepts.
• Experience working with cloud platforms such as AWS, Azure, or Google Cloud Platform.
• Experience with Git and CI/CD practices.