Data Engineer – Databricks / Scala

Data Engineer – Databricks / Scala (LOCALS ONLY : Quincy, MA )

Experience: 8–10 Years
Work Location: Quincy, MA – 100% Onsite
Candidate Preference: Local Candidates Only
Interview Requirement: Must be available for an in-person evaluation at Quincy or Princeton, NJ.

NEED ON W2

Must-Have Skills:

8+ years of Data Engineering experience.

Strong hands-on expertise in Azure Databricks, Scala and Apache Spark.

Scala / PySpark development and performance tuning.

Advanced SQL, ETL/ELT and data pipeline development.

Spark DataFrames, Datasets and Notebooks.

Large-scale data processing, streaming and transformations.

Delta Lake, Unity Catalog and Auto Loader.

Lakehouse and Medallion Architecture.

Spark Structured Streaming.

Databricks clusters, sizing, serverless and job orchestration.

Azure, DevOps pipelines and AKS troubleshooting.

Production monitoring, debugging and performance optimization.

Nice-to-Have Skills:

Apache Big Data ecosystem experience.

Java or R programming.

Docker and Kubernetes.

Containerization and cloud-native deployments.

Key Responsibilities:

Design and develop scalable Databricks data engineering solutions.

Build and optimize batch and streaming data pipelines.

Implement data ingestion, transformation and Lakehouse solutions.

Troubleshoot production Databricks jobs and DevOps pipelines.

Optimize Spark workloads and ensure data processing performance.

Key Focus: Azure Databricks | Scala | Apache Spark | PySpark | Delta Lake | Unity Catalog | Structured Streaming | SQL