Why This Role Stands Out
You will lead exciting cloud migration initiatives, leveraging your expertise in Cloudera/Hadoop and AWS to design and implement cutting-edge big data solutions. This hybrid role offers significant growth potential for seasoned engineers who thrive on complex challenges and enjoy mentoring others, all within a collaborative and forward-thinking environment.
Quick Overview
Seniority
Mid Senior
Work mode
Remote
Location
United States
Posted
2 weeks ago
ShellAWSAirflowBashComplianceEMRHadoopHivePythonTerraform
Job Description
Job Title: Lead Cloudera/Hadoop Migration Engineer
Duration of Contract: Long Term Contract
Target Date: Asap
Location: 100% Remote
Overview
Bachelor’s Degree in Computer Science, Information Technology, Information Systems, or related field.
10+ years of experience with enterprise and application design patterns that are applied on Hadoop and Big data platforms in aws.
The ideal candidate will possess deep expertise in both on-premises Hadoop ecosystems and modern cloud-native big data solutions on AWS.
Hands-on experience with Cloudera Hadoop stack (HDFS, Hive, Spark, Oozie, Python)
Proficiency in writing, maintaining, and troubleshooting shell scripts (Bash or similar) for automation and operational support.
Migration to AWS will need –
Analyze existing Cloudera Hadoop architecture, workflows, and data pipelines.
Develop a comprehensive migration strategy to transfer data, processing logic, and applications to AWS services.
Design target architecture using AWS big data services such as Amazon EMR, Airflow, Glue, Lambda.
Evaluate and select appropriate AWS tools for data ingestion, storage, processing, and analytics.
Oversee the migration process, including data transfer, application re-platforming, and validation.
Ensure the new architecture meets performance, scalability, reliability, and security requirements.
Collaborate with developers, data engineers, DevOps, and stakeholders to ensure a seamless migration.
Create detailed technical documentation and migration runbooks.
Mentor team members on AWS big data best practices.
Strong understanding of data security, compliance, and governance in the cloud.
Proven track record of migrating or modernizing big data workloads to AWS.
Experience with cloud automation and Infrastructure as Code (e.g. Terraform).
AWS certifications (e.g., AWS Certified Solutions Architect, AWS Certified Big Data – Specialty) are highly desirable.
10+ years of experience with enterprise and application design patterns that are applied on Hadoop and Big data platforms in aws.
The ideal candidate will possess deep expertise in both on-premises Hadoop ecosystems and modern cloud-native big data solutions on AWS.
Hands-on experience with Cloudera Hadoop stack (HDFS, Hive, Spark, Oozie, Python)
Proficiency in writing, maintaining, and troubleshooting shell scripts (Bash or similar) for automation and operational support.
Migration to AWS will need –
Analyze existing Cloudera Hadoop architecture, workflows, and data pipelines.
Develop a comprehensive migration strategy to transfer data, processing logic, and applications to AWS services.
Design target architecture using AWS big data services such as Amazon EMR, Airflow, Glue, Lambda.
Evaluate and select appropriate AWS tools for data ingestion, storage, processing, and analytics.
Oversee the migration process, including data transfer, application re-platforming, and validation.
Ensure the new architecture meets performance, scalability, reliability, and security requirements.
Collaborate with developers, data engineers, DevOps, and stakeholders to ensure a seamless migration.
Create detailed technical documentation and migration runbooks.
Mentor team members on AWS big data best practices.
Strong understanding of data security, compliance, and governance in the cloud.
Proven track record of migrating or modernizing big data workloads to AWS.
Experience with cloud automation and Infrastructure as Code (e.g. Terraform).
AWS certifications (e.g., AWS Certified Solutions Architect, AWS Certified Big Data – Specialty) are highly desirable.
Similar jobs
- IN
FHIR Data Engineer
NewIntone Networks Inc.
United States🇺🇸HybridYesterdaySQLScalaAWS+4Technology - SJ
Senior Data Engineer
NewSelby Jennings
Manhattan, NY🇺🇸Hybrid2 days agoSQLETLSnowflake+4Technology - BC
Data Engineer
NewBCforward
Spring, TX🇺🇸$47/hrHybridYesterdaySQLSpringETL+1Technology - NY
Essbase Data Engineer - Charlotte, NC
NewNew York Technology Partners
Charlotte, NC🇺🇸HybridYesterdayOracleSQLAWS+6Technology - IT
Snowflake Data engineer (F2F) Local to NY/NJ only
NewISite Technologies Inc
New York, NY🇺🇸Hybrid2 days agoSnowflakePythondbtTechnology - GU
Data Engineer with Security Clearance
NewGuidehouse
Huntsville, AL🇺🇸HybridYesterdayScalaETLScrum+6Technology