Data Engineer

Job Title: Data Engineer
Experience: 12–14 Years
Job Type: Contract
Location: [Remote], Travel to Office once in a month

Job Summary

We are looking for an experienced Senior AWS Data Engineer with 12–14 years of overall experience in data engineering and strong hands-on expertise in AWS cloud services, PySpark, ETL, AWS Glue, AWS Lambda, and Amazon EC2.

The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines and cloud-based data solutions. The candidate should have strong experience working with large datasets, building ETL workflows, developing data transformation processes using PySpark, and leveraging AWS services to build reliable and scalable data platforms.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using AWS services.
• Develop robust ETL/ELT pipelines for batch and large-scale data processing.
• Build and optimize data transformation workflows using PySpark.
• Develop and maintain ETL jobs using AWS Glue.
• Develop serverless data processing and integration solutions using AWS Lambda.
• Configure, manage, and troubleshoot workloads running on Amazon EC2.
• Perform data extraction, transformation, cleansing, validation, and loading across multiple data sources.
• Optimize PySpark jobs and ETL pipelines for performance and scalability.
• Integrate data from structured and unstructured sources.
• Develop reusable data processing frameworks and components.
• Monitor data pipelines and troubleshoot production issues.
• Implement data quality, validation, and error-handling processes.
• Work with cloud storage and databases as part of end-to-end data engineering solutions.
• Collaborate with Data Architects, Developers, Analysts, and business stakeholders.
• Participate in code reviews, technical design discussions, testing, deployment, and production support.
• Follow best practices for security, scalability, reliability, and cost optimization within AWS.
• Mentor junior and mid-level data engineers and provide technical guidance.