Senior Data Engineer (Pyspark)

Join Our Team as a Senior Data Engineer (PySpark) – Drive Innovation at the Forefront of Data Technology

Are you a seasoned Data Engineer with a passion for building scalable, efficient data solutions in the cloud? We are seeking an accomplished Senior Data Engineer (PySpark) to join a dynamic team in Jersey City, NJ. This is your opportunity to work on high-impact projects, leveraging cutting-edge AWS services and AI-driven tools to shape the future of data engineering.

What You'll Do:

• Design, develop, and maintain large-scale data pipelines using Python and PySpark.
• Troubleshoot and optimize data workflows, applying deep understanding of data structures and distributed processing.
• Write and optimize complex SQL queries; implement data warehousing solutions including SCD Type 1 & 2.
• Develop and deploy scalable APIs on AWS (ECS), ensuring high availability and performance.
• Collaborate with cross-functional teams to resolve infrastructure and application issues.
• Build automated CI/CD pipelines to streamline deployment processes.
• Utilize AI-powered development tools like GitHub Copilot to enhance productivity and code quality.

What We're Looking For:

• Extensive hands-on experience with AWS services: ECS, Lambda, SNS, DynamoDB.
• Strong programming skills in Python, with expert-level PySpark experience for data processing.
• Solid understanding of SQL, data modeling, and data warehousing concepts including Slowly Changing Dimensions.
• Proven ability to build and deploy APIs on AWS / ECS.
• Familiarity with DevOps practices, CI/CD pipelines, and automation.
• Experience using AI-assisted coding tools such as GitHub Copilot.
• Excellent problem-solving, debugging, and analytical skills.

Nice to Have:

• Experience with other cloud platforms or additional AI tools.
• Knowledge of containerization, orchestration, and serverless architectures.
• Certification in AWS or related fields.

Preferred Education & Experience:

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• 6-9 years of professional experience in data engineering, cloud deployment, and software development.

Additional Requirements:

• This position requires onsite presence in Jersey City, NJ.
• Ability to work in a fast-paced, collaborative environment.

Ready to make an impact in a cutting-edge data environment? If you’re a passionate, skilled engineer eager to leverage your expertise and drive innovation, we want to hear from you. Apply now and take the next step in your career!