Lead Data Engineer- AWS

As part of the Big Data Engineering and Analytics organization, this role will focus on enabling a modern data lake and data mesh architecture that supports high volume, high velocity, and high variety data across enterprise and platform domains.

The Lead Data Engineer will bring deep hands-on expertise in Kubernetes, Kafka streaming platforms. This role partners closely with data architects, analytics teams, and platform engineering to deliver reliable, governed, and high-quality data solutions.

Responsibilities

- Lead the design and implementation of scalable big data pipelines for batch and real time processing

- Build and operate streaming data platforms using Kafka

- Design and deploy cloud native data solutions across AWS

- Develop and manage containerized workloads using Kubernetes

- Enable data mesh architecture with domain oriented data products

- Design and implement data lake and lakehouse architectures

- Ensure data quality, reliability, and observability

- Implement governance capabilities including metadata and lineage

- Collaborate with analytics, AI, and business teams

- Optimize performance and cost efficiency

- Mentor engineering teams

Requirements

- 10+ years of experience in big data engineering

- Strong experience with Kafka

- Strong experience with Kubernetes

- Experience with AWS

- Experience building data lakes or lakehouse platforms

- Experience with data mesh concepts

- Strong programming skills in Python, Scala, or Java

- Experience with Spark, Flink, or Beam

- Experience with Airflow or orchestration tools

- Understanding of data modeling and governance

- Experience with CICD and infrastructure automation

- Experience supporting AI and machine learning workloads including large language models such as Claude or similar platforms

- Strong communication and leadership skills