Start searching
Back to Jobs
Recently postedTechnology

AWS Lead Data Engineer

Marlabs LLCNewark, NJ🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeOn Site
LocationNewark, NJ, United States
Apache SparkAWSApachePythonRedshift

Job description

Lead Data Engineer

Location : Newark NJ – Onsite (3 days a week)

Duration : 12 Month

F2F interview Must

Key Responsibilities

  • Lead the design and implementation of data ingestion, transformation and processing solutions using AWS native services including S3, Glue, EMR, Redshift and Athena.
  • Define appropriate engineering patterns for batch and large-scale data processing using Python and PySpark.
  • Guide implementation of modern lake/lakehouse patterns, including S3 and Apache Iceberg, where applicable.
  • Provide technical direction across design, development, testing, deployment and production support.
  • Lead onboarding of data from legacy and enterprise source systems into AWS.
  • Drive development of reusable pipelines and engineering components for specific enterprise data domains.
  • Support the ADS / Authoritative Data Source approach, where domain data is packaged as trusted data products for enterprise consumption.
  • Ensure datasets produced by the engineering team are consumable for downstream analytics and AI use cases.
  • Conduct design and code reviews for Python/PySpark and AWS data-engineering components.
  • Establish development standards, reusable patterns and engineering best practices.
  • Ensure appropriate data validation and reconciliation are incorporated into pipelines.
  • Troubleshoot performance and scalability issues across data pipelines and processing workloads.
  • Provide day-to-day technical guidance to Data Engineers.
  • Break functional/data requirements into engineering tasks and coordinate delivery.
  • Communicate technical risks, dependencies and design decisions to stakeholders.

Mentor engineers and build the team's AWS/Python/PySpark engineering capability

Similar jobs

Apply