AWS Lead Data Engineer (AWS | PySpark | Python | Data Lake | Technical Lead)

Client: Confidential
Location: Newark, NJ (Hybrid – Minimum 3 Days Onsite/Week)
Employment Type: Contract-to-Hire (C2H)
Experience: 8+ Years (Lead Level)
Interview Process:

• Round 1: Virtual Technical Interview
• Round 2: Mandatory Onsite Interview (Non-Negotiable)

Important Submission Requirements

• Hybrid role – Minimum 3 days/week onsite in Newark, NJ
• Mandatory onsite interview
• Must have grown from a hands-on Data Engineer into a Lead role
• Candidates must be able to explain real hands-on implementation, architecture decisions, and production experience

Job Summary

Prudential is looking for a hands-on AWS Lead Data Engineer to architect, develop, and optimize enterprise-scale cloud data solutions on AWS. This is a technical leadership role requiring deep expertise in AWS, PySpark, Python, and modern data engineering, along with the ability to mentor a team of Data Engineers while remaining actively involved in solution design and implementation.

The ideal candidate will have strong experience building data lakes, scalable ETL/ELT pipelines, cloud-native architectures, and guiding engineering teams through architecture reviews and best practices.

Required Experience

• 8+ years of Data Engineering
• 3+ years in Lead or Technical Leadership roles
• AWS Cloud Data Engineering
• Data Lake & Data Warehouse Architecture
• Enterprise Data Platforms
• Team Leadership & Mentoring

Roles & Responsibilities

• Design and implement scalable AWS data pipelines and data lake architectures.
• Build end-to-end solutions for data ingestion, transformation, storage, and analytics.
• Lead architecture reviews and recommend scalable technical solutions.
• Mentor and guide a team of 4–5 Data Engineers and QA engineers.
• Collaborate with data scientists, analysts, and business stakeholders.
• Lead migration from legacy systems to AWS-based data platforms.
• Establish standards for data quality, governance, and security.
• Implement DevOps, CI/CD, and Infrastructure-as-Code practices.
• Optimize performance and cost of AWS data solutions.
• Troubleshoot complex production issues and provide technical leadership.

Required Qualifications

• Bachelor''s or Master''s degree in Computer Science, Engineering, or a related field.
• 8+ years of Data Engineering experience.
• Strong hands-on AWS data engineering expertise.
• Advanced Python and PySpark development.
• Strong SQL skills.
• Experience with AWS Glue, Redshift, S3, Lambda, EMR, Kinesis, Athena, and RDS.
• Experience building enterprise data lakes and ETL/ELT pipelines.
• Proven leadership experience mentoring engineering teams.
• Strong communication and problem-solving skills.

Preferred Qualifications

• AWS Solutions Architect Certification
• AWS Data Engineer Certification
• Experience with real-time streaming technologies
• Knowledge of data governance and security best practices
• Experience with Lakehouse architecture and modern data platforms