Quick overview
Job description
Develop API-driven systems to govern, manage, and monitor large-scale batch Big Data applications.
Build scalable backend services and data engineering solutions that support data processing and operational workflows.
Develop and maintain data transformation processes using Spark, SQL, Hive, Python, Scala, and related technologies.
Work with AWS cloud services such as S3, EC2, EMR, Lambda, DynamoDB, and API Gateway to build cloud-native data and backend solutions.
Build and enhance workflow orchestration using Apache Airflow, including advanced DAG design, dependency management, scheduling, monitoring, and failure handling.
Define technical scope, objectives, and implementation approaches by participating in requirements gathering, technical research, and process definition.
Participate in architecture reviews, code reviews, performance tuning, and operational readiness activities.
Contribute to test planning and validation for application integrations, functional areas, and project deliverables.
Partner with product owners, data engineers, backend engineers, QA, DevOps, and other cross-functional teams to deliver high-quality solutions in an Agile/Scrum environment.
Troubleshoot production issues, improve system reliability, and drive automation across data and backend workflows.
5+ years of hands-on experience developing enterprise-scale applications, data platforms, or distributed systems.
5+ years of experience developing, and operating Big Data platforms and cloud-based infrastructure services, preferably using AWS EMR and the Hadoop ecosystem.
Hands-on experience with AWS, Spark, Python and/or Scala, Airflow, SQL, Hive, and related data processing technologies.
Proficiency in either Python or Scala, with the ability to build production-grade data pipelines and backend services.
Experience with AWS cloud services, including S3, EC2, EMR, Lambda, DynamoDB, and API Gateway.
Strong experience with Apache Airflow orchestration, including complex workflows, scheduling, dependency handling, monitoring, and operational support.
Strong understanding of data engineering concepts, including data pipelines, data transformation, batch processing, real-time processing, data quality, and performance optimization.
Excellent analytical and problem-solving skills.
Strong communication and presentation skills, with the ability to explain technical concepts clearly to both technical and non-technical audiences.
Experience with version control and CI/CD tools, including Git and Jenkins, for source code management, build automation, testing, and deployment workflows.
Ability to work effectively in cross-functional Agile/Scrum teams.
Experience with LLMs, Generative AI, Agentic AI, or AI-assisted engineering workflows.
Experience with API design, microservices, event-driven architecture, or serverless backend development.
Experience with CI/CD pipelines, infrastructure as code, automated testing, and production deployment practices.
Experience with data governance, metadata management, lineage, data observability, or operational controls for enterprise data platforms
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free
Similar jobs
- WP

Data EngineerRecently posted Auto Apply
WPP · Chennai · Hybrid · Posted 18 Nov 2026 🇮🇳
SQLApache SparkDatabricks+9TechnologySalary not listed - JU

Data Engineer - II (Biometrics)Recently posted Auto Apply
Jumio · Bengaluru · Hybrid · Posted 18 Nov 2026 🇮🇳
SQLAWSETL+5TechnologySalary not listed - CH

Senior Data EngineerRecently posted Auto Apply
Cohere Health · Hyderabad · Hybrid · Posted 17 Nov 2026 🇮🇳
SQLApache SparkAWS+6TechnologySalary not listed - B2
Senior Fabric Lead Data Engineer - R01572945 Auto Apply
Brillio 2 · Bengaluru · Hybrid · Posted 22 Oct 2026 🇮🇳
SQLGitApache Spark+6TechnologySalary not listed - WE
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekdayworks · Bengaluru · On-site · Posted 16 Nov 2026 🇮🇳
Technology₹1400k - ₹2000k/yr - V1
Senior Oracle Fusion Data Migration ConsultantRecently posted Auto Apply
Version 1 · Bengaluru, KA · Hybrid · Posted 16 Nov 2026 🇮🇳
OraclePL/SQLSQL+6TechnologySalary not listed - WA
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekday AI · Chennai, Tamil Nadu · On-site · Posted 16 Nov 2026 🇮🇳
GCPSQLAirflow+6TechnologySalary not listed - WA
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekday AI · Hyderabad, Telangana · On-site · Posted 16 Nov 2026 🇮🇳
GCPSQLAirflow+6TechnologySalary not listed - WA
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekday AI · Pune, Maharashtra · On-site · Posted 16 Nov 2026 🇮🇳
GCPSQLAirflow+6TechnologySalary not listed - WA
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekday AI · Bengaluru, Karnataka · On-site · Posted 16 Nov 2026 🇮🇳
GCPSQLAirflow+6TechnologySalary not listed - EX
Technical Data managerRecently posted Auto Apply
Excelya · Bengaluru, Karnataka · Hybrid · Posted 16 Nov 2026 🇮🇳
GCPOracleMicrosoft Excel+2TechnologySalary not listed - BG
KNIME SpecialistRecently posted Auto Apply
Bosch Group · bengaluru · On-site · Posted 13 Nov 2026 🇮🇳
SQLAWSETL+5TechnologySalary not listed