Role: Lead Data Engineer

Location: Cincinnati, OH (Onsite)
Duration: 12 Months

Role Overview:

We are seeking an experienced Lead Data Engineer with strong hands-on expertise in Databricks, Apache Spark, SQL, and Python/PySpark to lead enterprise data modernization initiatives, particularly migrating legacy data warehouse platforms such as IBM Netezza to Databricks. The ideal candidate will combine technical leadership, large-scale data engineering expertise, and experience managing distributed onshore and offshore teams.

Key Responsibilities:

• Lead onshore and offshore data engineering teams across client organizations, implementation partners, and multiple vendors.

• Drive end-to-end migration of legacy data warehouse workloads from IBM Netezza to Databricks.

• Convert business requirements and architecture designs into technical specifications, development plans, and engineering deliverables.

• Design, develop, and optimize scalable ETL/ELT pipelines, data transformations, and workflows using Databricks, Spark, SQL, and Python/PySpark.

• Modernize legacy SQL queries, stored procedures, scripts, and data processing logic for execution on Databricks.

• Implement data quality checks, automated testing, performance tuning, and coding standards to ensure reliable and efficient data pipelines.

• Coordinate with architects, project managers, business stakeholders, and vendor teams to manage technical dependencies and delivery milestones.

• Mentor junior and mid-level engineers, conduct code reviews, and provide technical guidance.

• Support deployment activities, production troubleshooting, incident resolution, and ongoing pipeline maintenance.

• Lead data validation, source-to-target reconciliation, migration cutover, and post-deployment stabilization.

• Leverage Azure and Google Cloud Platform cloud services to support data integration, storage, and modernization initiatives.

Required Qualifications:

• 8+ years of experience in data engineering, data warehousing, ETL/ELT development, or analytics engineering.

• 2+ years of experience leading onshore/offshore teams and coordinating distributed engineering groups.

• Strong hands-on expertise in Databricks (mandatory), Apache Spark, SQL, Python, and PySpark.

• Experience with IBM Netezza or comparable enterprise data warehouse platforms.

• Proven experience in data warehouse migration, cloud data platform modernization, or legacy system transformation.

• Strong understanding of data pipelines, Lakehouse architecture, distributed processing, and data modeling.

• Familiarity with Azure and Google Cloud Platform (Google Cloud Platform).

• Excellent communication, technical leadership, problem-solving, and troubleshooting skills.

Preferred Skills:

• Experience migrating Netezza SQL workloads and legacy ETL processes to Databricks.

• Knowledge of Spark optimization, partitioning, caching, job performance tuning, and scalable data processing.

• Familiarity with Databricks workflows, Delta Lake, and modern Lakehouse design patterns.

• Experience with migration testing, data reconciliation, production deployment, and cutover planning.

• Ability to manage multiple vendors, mentor engineers, and deliver projects in a collaborative, onsite environment.

Important: Hands-on Databricks expertise is a mandatory requirement for this role.