Lead Data Engineer

Location: Raritan, NJ
Duration: Long-Term Contract
Employment Type: W2 Only
Experience Required: 12+ Years

Job Overview

We are seeking an experienced Lead Data Engineer with 12+ years of experience in data engineering, data warehousing, and modern cloud-based data platforms. The ideal candidate will have strong hands-on expertise with Databricks, Apache Spark/PySpark, SQL, Python, Microsoft Azure, Delta Lake, Unity Catalog, and Azure Data Factory (ADF), along with experience working with enterprise data platforms and governance.

Required Technical Skills

• Strong hands-on experience with Databricks, Apache Spark (PySpark), SQL, and Python.

• Expertise in Microsoft Azure, Delta Lake, Unity Catalog, and Azure Data Factory (ADF).

• Experience working with enterprise databases including Teradata and PostgreSQL.

• Strong knowledge of Informatica and Data Warehousing concepts and technologies.

• Familiarity with Power BI, Cognos, and Tableau is a plus.

• Experience implementing and supporting Data Quality, Data Governance, and Data Reconciliation processes.

• Proficiency with Jira, Confluence, Git, CI/CD, Release Management, and Agile Delivery methodologies.

• Knowledge of Generative AI, LLMs, Databricks AI, and Natural Language Querying is preferred to support future AI-enabled procurement capabilities.

• Knowledge of procurement and financial data domains is preferred.

Preferred Qualifications

• Experience designing and delivering scalable cloud-based data engineering solutions.

• Strong understanding of data integration, data transformation, data warehousing, and data governance.

• Ability to work effectively in an Agile delivery environment and collaborate with cross-functional teams.

• Experience supporting emerging AI/LLM capabilities within enterprise data platforms is highly desirable.

• Procurement and financial data domain experience is a strong advantage.