Lead Sr Software Engineer (Cloudera Hadoop stack)

Lead Sr Software Engineer (Cloudera Hadoop stack)
REMOTE

Visa Eligibility: Please confirm if the candidate is legally authorized to work in the United States

Experience Level: 10+ years
Contract 12 months

Qualifications:

·         Bachelor’s degree in computer science, Information Technology, Information Systems, or related field.

·         10+ years of experience with enterprise and application design patterns that are applied on Hadoop and Big data platforms in aws.

·         The ideal candidate will possess deep expertise in both on-premises Hadoop ecosystems and modern cloud-native big data solutions on AWS.

·         Hands-on experience with Cloudera Hadoop stack (HDFS, Hive, Spark, Oozie, Python) Proficiency in writing, maintaining, and troubleshooting shell scripts (Bash or similar) for automation and operational support.

·         Migration to AWS will need – Analyze existing Cloudera Hadoop architecture, workflows, and data pipelines.

·         Develop a comprehensive migration strategy to transfer data, processing logic, and applications to AWS services.

·         Design target architecture using AWS big data services such as Amazon EMR, Airflow, Glue, Lambda.

·         Evaluate and select appropriate AWS tools for data ingestion, storage, processing, and analytics.

·         Oversee the migration process, including data transfer, application re-platforming, and validation.

·         Ensure the new architecture meets performance, scalability, reliability, and security requirements.

·         Collaborate with developers, data engineers, DevOps, and stakeholders to ensure a seamless migration.

·         Create detailed technical documentation and migration runbooks. Mentor team members on AWS big data best practices. Strong understanding of data security, compliance, and governance in the cloud.

·         Proven track record of migrating or modernizing big data workloads to AWS.

·         Experience with cloud automation and Infrastructure as Code (e.g. Terraform).

·         AWS certifications (e.g., AWS Certified Solutions Architect, AWS Certified Big Data – Specialty) are highly desirable.

·         Additional skills required Familiarity with xml/json data formats Knowledge of monitoring and logging tools

·         5+ years of experience with platform engineering tools such as Bitbucket, Gradle, Octopus, Artifactory.

Preferred Qualifications

·         Bachelor’s degree in Computer Science, Engineering, or related field (often required)

·         Knowledge of healthcare data such as claims and entities such as providers, members and their relationship within the framework of a claim.