Quick overview
Job description
We are seeking a hands-on Data Engineer to support a large-scale data platform modernization initiative. You will migrate approximately 1,000 datasets to Snowflake, adapt existing pipelines, and help deliver reliable orchestration with Apache Airflow 3. The architecture and technical approach are already defined, with the role focused on implementation and migration delivery through March 2027.
Enterprise experience strongly preferred.
Key Responsibilities
- Migrate pipelines from Apache Airflow 2 to Apache Airflow 3.
- Migrate Confluent connections between cloud regions.
- Migrate pipelines from Hive to Iceberg using Snowpark Connect.
- Adapt existing data pipelines to Snowflake while maintaining data quality and reliability.
- Support workflow scheduling and monitoring in Apache Airflow.
- Partner with the team to execute time-sensitive data migrations.
Must-Have Skills
- 2+ years of experience.
- Strong software engineering fundamentals.
- Hands-on Python development experience.
- Advanced SQL, including complex joins, CTEs, window functions, and performance tuning.
- Snowflake data modeling.
- Apache Airflow experience developing and testing pipelines in Airflow and Python.
- Apache Kafka experience.
- Apache Spark experience, including hands-on PySpark.
- Experience with Iceberg and Parquet data formats.
- AWS experience, particularly S3 data lakes, partitioning, and lifecycle policies.
- Testing strategy experience and the ability to debug and test code hands-on.
- At least 2 years of industry experience working with large-scale datasets, including Airflow and Spark.
- Strong communication skills.
- Ability to work independently.
Nice-to-Have Skills
- Hands-on Snowflake migration experience, including data warehouse modernization and performance and cost optimization.
- Apache Airflow migration experience, including version upgrades, executor changes, or platform migrations.
- Experience using AI tools for development.
Required Tools & Platforms
- Python and SQL.
- Snowflake, Apache Airflow, Apache Kafka, and Apache Spark/PySpark.
- Iceberg and Parquet.
- AWS S3.
- The migration environment also includes Confluent, Hive, and Snowpark Connect.
Location, Time & Engagement
- Open to candidates in eligible EMEA and APAC locations.
- Coverage until 12:00 PM Pacific Time is required.
- Remote contract engagement for 40 hours per week.
- Expected engagement through March 31, 2027; start date to be confirmed.
- Two openings are available.
- This is not currently a contract-to-hire opportunity.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- SW

Structured Data Engineer - AI Search (Remote, Full-Time) [HR226]Recently posted Auto Apply
Smart Working Solutions · India · Remote · Posted 18 Nov 2026 🇮🇳
SQLGitHTML+7TechnologySalary not listed - BM

AI Data Engineer (Speech & Language)Recently posted Auto Apply
Blue Machines AI · Bengaluru, Karnataka · On-site · Posted 19 Nov 2026 🇮🇳
GitNLPBash+6TechnologySalary not listed - CA

Snowflake Data EngineerRecently posted Auto Apply
Capco · Bengaluru · Hybrid · Posted 19 Nov 2026 🇮🇳
SQLGitETL+13TechnologySalary not listed - WP

Data EngineerRecently posted Auto Apply
WPP · Chennai · Hybrid · Posted 18 Nov 2026 🇮🇳
SQLApache SparkDatabricks+9TechnologySalary not listed - JU

Data Engineer - II (Biometrics)Recently posted Auto Apply
Jumio · Bengaluru · Hybrid · Posted 18 Nov 2026 🇮🇳
SQLAWSETL+5TechnologySalary not listed - CH

Senior Data EngineerRecently posted Auto Apply
Cohere Health · Hyderabad · Hybrid · Posted 17 Nov 2026 🇮🇳
SQLApache SparkAWS+6TechnologySalary not listed - B2
Senior Fabric Lead Data Engineer - R01572945 Auto Apply
Brillio 2 · Bengaluru · Hybrid · Posted 22 Oct 2026 🇮🇳
SQLGitApache Spark+6TechnologySalary not listed - WE
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekdayworks · Bengaluru · On-site · Posted 16 Nov 2026 🇮🇳
Technology₹1400k - ₹2000k/yr - V1
Senior Oracle Fusion Data Migration ConsultantRecently posted Auto Apply
Version 1 · Bengaluru, KA · Hybrid · Posted 16 Nov 2026 🇮🇳
OraclePL/SQLSQL+6TechnologySalary not listed - WA
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekday AI · Chennai, Tamil Nadu · On-site · Posted 16 Nov 2026 🇮🇳
GCPSQLAirflow+6TechnologySalary not listed - WA
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekday AI · Hyderabad, Telangana · On-site · Posted 16 Nov 2026 🇮🇳
GCPSQLAirflow+6TechnologySalary not listed - WA
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekday AI · Pune, Maharashtra · On-site · Posted 16 Nov 2026 🇮🇳
GCPSQLAirflow+6TechnologySalary not listed