Quick overview
Job description
In-depth working experience of distributed systems Hadoop, Spark, Hive, DBT and Airflow/Dagster
At least 5 years of solid production quality coding experience in data pipeline implementation in Python
Experience working with public cloud platforms, preferably AWS
Experience working with Databricks and/or Snowflake
Experience in Git, JIRA, Jenkins, shell scripting
Familiarity with Agile methodology, test-driven development, source control management and test automation
Experience supporting and working with cross-functional teams in a dynamic environment
You have excellent collaborative problem solving and communication skills and are empathetic to others
You have a track record of building fast, reliable, and high-quality data pipelines
Strong hands-on experience with PySpark, Python, SQL, Databricks, and AWS Cloud is required at a Lead level. The candidate should possess excellent problem-understanding and problem-solving capabilities, be able to lead technical discussions, guide team members, and effectively communicate and collaborate with both internal teams and client stakeholders.
You will build services that will integrate directly with Client's products and with external vendors.
You will work with modern data technologies such as Hadoop, Spark, DBT, Dagster/Airflow, Atlan, Trino, etc., modern data platforms such as Databricks and Snowflake and cloud technologies across AWS stack
Identify, design, and implement automation of manual processes, optimizing data delivery, reducing Cloud cost, etc.
Implement processes and systems to monitor Data Quality, Observability, Governance and Lineage.
Support operations to manage the production environment and help in resolving production issues with RCA
Write unit/integration tests, adopt Test-driven development, contribute to engineering wiki, and document design/implementation etc.
Graduate in Computer Science, Data Science, Information Technology, or a related field.
8+ years of experience and a bachelor’s degree in computer science or a related field, or equivalent work experience
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- SW

Structured Data Engineer - AI Search (Remote, Full-Time) [HR226]Recently posted Auto Apply
Smart Working Solutions · India · Remote · Posted 18 Nov 2026 🇮🇳
SQLGitHTML+7TechnologySalary not listed - BM

AI Data Engineer (Speech & Language)Recently posted Auto Apply
Blue Machines AI · Bengaluru, Karnataka · On-site · Posted 19 Nov 2026 🇮🇳
GitNLPBash+6TechnologySalary not listed - CA

Snowflake Data EngineerRecently posted Auto Apply
Capco · Bengaluru · Hybrid · Posted 19 Nov 2026 🇮🇳
SQLGitETL+13TechnologySalary not listed - WP

Data EngineerRecently posted Auto Apply
WPP · Chennai · Hybrid · Posted 18 Nov 2026 🇮🇳
SQLApache SparkDatabricks+9TechnologySalary not listed - JU

Data Engineer - II (Biometrics)Recently posted Auto Apply
Jumio · Bengaluru · Hybrid · Posted 18 Nov 2026 🇮🇳
SQLAWSETL+5TechnologySalary not listed - CH

Senior Data EngineerRecently posted Auto Apply
Cohere Health · Hyderabad · Hybrid · Posted 17 Nov 2026 🇮🇳
SQLApache SparkAWS+6TechnologySalary not listed - B2
Senior Fabric Lead Data Engineer - R01572945 Auto Apply
Brillio 2 · Bengaluru · Hybrid · Posted 22 Oct 2026 🇮🇳
SQLGitApache Spark+6TechnologySalary not listed - WE
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekdayworks · Bengaluru · On-site · Posted 16 Nov 2026 🇮🇳
Technology₹1400k - ₹2000k/yr - V1
Senior Oracle Fusion Data Migration ConsultantRecently posted Auto Apply
Version 1 · Bengaluru, KA · Hybrid · Posted 16 Nov 2026 🇮🇳
OraclePL/SQLSQL+6TechnologySalary not listed - WA
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekday AI · Chennai, Tamil Nadu · On-site · Posted 16 Nov 2026 🇮🇳
GCPSQLAirflow+6TechnologySalary not listed - WA
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekday AI · Hyderabad, Telangana · On-site · Posted 16 Nov 2026 🇮🇳
GCPSQLAirflow+6TechnologySalary not listed - WA
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekday AI · Pune, Maharashtra · On-site · Posted 16 Nov 2026 🇮🇳
GCPSQLAirflow+6TechnologySalary not listed
