PySpark Developer

Job Title: Senior PySpark Developer
Location: Remote
Duration: Long-Term

Job Summary:
Seeking a Senior PySpark Developer with strong experience in PySpark, Cloudera CDP, Apache Iceberg, Airflow, SQL, and AWS to support a large-scale cloud transformation and data modernization initiative.

Required Skills:

• 10+ years IT experience; 6+ years in Data Engineering/Big Data
• Strong hands-on PySpark/Spark development
• Experience with Cloudera Data Platform (CDP)
• Strong knowledge of Apache Iceberg and Data Lake/Lakehouse architecture
• Experience with Apache Airflow
• Strong SQL, ETL/ELT, and data modeling skills
• Large-scale structured/unstructured data processing
• Git, CI/CD, and DevOps practices
• Performance tuning, partitioning, and query optimization

Preferred:

• AWS: S3, Glue, Lambda, EMR, ECS/EKS, RDS, IAM, CloudWatch
• Kafka/event-driven processing
• Python/Java development
• Data quality, metadata, and governance
• Healthcare/Medicaid/Claims/Insurance experience
• Agile and cloud migration/modernization experience
• AWS or Cloudera certification