Lead Data Engineer

Location: New York City, NY Salary: $80.00 USD Hourly - $88.00 USD Hourly Description:
Lead Data Engineer

New York, NY (10001)

6+ months Contract to Perm

About the Role

We are seeking a Lead Software Engineer - Data Engineering to design and build scalable, cloud-native data platforms and pipelines. You will lead the development of robust data solutions, drive architectural decisions, and mentor engineers while ensuring reliability, security, and operational excellence.

Responsibilities

Design and develop large-scale data pipelines and data platforms.

Build cloud-native solutions using AWS services and Spark.

Lead data modeling, warehousing, migration, and analytics initiatives.

Improve system performance, resilience, monitoring, and automation.

Collaborate with cross-functional teams and mentor junior engineers.

Required Qualifications

9-12+ Experience in Data Engineering.

Strong proficiency in Python and/or Java with Spark/PySpark.

Experience with Data Pipelines, Data Modeling, Data Warehousing, and Data Migration.

Hands-on experience with AWS (Glue, EMR, S3, Athena, Kinesis/MSK, Airflow, Lambda, Step Functions, Lake Formation).

Knowledge of Kubernetes, CI/CD, Agile methodologies, and software design principles.

Experience with data formats such as Parquet, Iceberg, Avro, or JSON.

Understanding of AWS security concepts including IAM, KMS, Encryption, and Secrets Manager.

Preferred Qualifications

Experience building REST APIs using Java.

Knowledge of Hybrid Cloud architectures.

Exposure to Jenkins, Git, Linux Scripting, JIRA, and TDD.

Strong leadership, problem-solving, and communication skills.

Tech Stack: Python, Java, Spark, AWS, Kubernetes, Airflow, Glue, EMR, Athena, Kinesis, Parquet, Iceberg, CI/CD.

By providing your phone number, you consent to: (1) receive automated text messages and calls from the Judge Group, Inc. and its affiliates (collectively "Judge") to such phone number regarding job opportunities, your job application, and for other related purposes. Message & data rates apply and message frequency may vary. Consistent with Judge's Privacy Policy, information obtained from your consent will not be shared with third parties for marketing/promotional purposes. Reply STOP to opt out of receiving telephone calls and text messages from Judge and HELP for help.

Contact

This job and many more are available through The Judge Group. Please apply with us today!