Python AWS Spark Engineer
Location: Plano, TX
Duration: Long-Term

Job Summary

Seeking an experienced Python AWS Spark Engineer to join the team in Plano, TX. The ideal candidate will have strong hands-on experience with Python, Apache Spark/PySpark, AWS cloud services, and data engineering. This role will focus on developing scalable data processing solutions, building cloud-based data pipelines, and supporting enterprise data platforms.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Python and Apache Spark/PySpark.
• Develop high-performance ETL/ELT workflows for large-volume datasets.
• Build and deploy data processing solutions on AWS.
• Work with AWS services such as S3, Glue, EMR, Lambda, Redshift, and CloudWatch.
• Develop and optimize Spark jobs for performance, scalability, and reliability.
• Perform data transformation, cleansing, validation, and integration across multiple data sources.
• Write efficient Python and SQL code for data processing and analytics.
• Troubleshoot production issues and perform root-cause analysis.
• Implement data quality, monitoring, logging, and error-handling mechanisms.
• Collaborate with data engineers, architects, application teams, and business stakeholders.
• Participate in code reviews, unit testing, deployment, and production support.
• Follow enterprise security, compliance, and development standards within the banking environment.

Required Skills

• 7+ years of experience in software/data engineering.
• Strong hands-on experience with Python.
• Strong experience with Apache Spark / PySpark.
• Strong experience with AWS cloud services.
• Hands-on experience with AWS S3, Glue, EMR, Lambda, and/or Redshift.
• Strong SQL skills and experience working with relational databases.
• Experience developing large-scale ETL/ELT data pipelines.
• Experience with data transformation and processing of large datasets.
• Strong understanding of distributed computing and Spark architecture.
• Experience with Git and CI/CD practices.
• Excellent problem-solving and communication skills.