Data Engineer - Required skills; ETL, SQL, AWS SPARK (EMR's) - W2 ONLY

Job Summary

We are looking for an experienced Senior Data Engineer with 10+ years of experience in data engineering, ETL development, SQL, and AWS-based big data technologies. The ideal candidate will have strong expertise in building scalable data pipelines, processing large datasets using Apache Spark, and leveraging Amazon EMR for distributed data processing.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Develop complex SQL queries, stored procedures, and optimize database performance.
• Build and optimize large-scale data processing solutions using Apache Spark and PySpark.
• Develop, deploy, and manage big data applications on AWS EMR.
• Work with AWS services to build reliable, scalable, and cost-effective data engineering solutions.
• Perform data extraction, transformation, cleansing, validation, and loading from multiple sources.
• Optimize Spark jobs for performance, scalability, and resource utilization.
• Implement data quality checks, error handling, and monitoring for data pipelines.
• Collaborate with data architects, analysts, data scientists, and business stakeholders.
• Troubleshoot production issues and ensure data pipeline reliability.
• Follow best practices for data security, governance, and cloud cost optimization.

Required Skills

• Experience: 10+ years in data engineering or related roles.
• ETL: Strong experience in ETL/ELT development and data integration.
• SQL: Advanced SQL skills, query optimization, and performance tuning.
• AWS: Hands-on experience with AWS cloud services for data engineering.
• Apache Spark: Strong experience with Spark and PySpark for distributed data processing.
• Amazon EMR: Experience developing, running, and optimizing Spark jobs on AWS EMR.
• Strong understanding of data warehousing, data modeling, and large-scale data processing.
• Experience handling structured and semi-structured data.
• Strong analytical, problem-solving, and communication skills.

Preferred Skills

• Experience with AWS S3, Glue, Lambda, Redshift, and IAM.
• Familiarity with Python or Scala.
• Experience with workflow orchestration tools such as Apache Airflow or AWS Step Functions.
• Knowledge of CI/CD, Git, and automated deployment practices.
• Experience with monitoring, logging, and production support for data pipelines.