Quick overview
Job description
๐ง๐ต๐ถ๐ ๐ฟ๐ผ๐น๐ฒ ๐ถ๐ ๐ณ๐ผ๐ฟ ๐ผ๐ป๐ฒ ๐ผ๐ณ ๐๐ต๐ฒ ๐ช๐ฒ๐ฒ๐ธ๐ฑ๐ฎ๐'๐ ๐ฐ๐น๐ถ๐ฒ๐ป๐๐
๐ฆ๐ฎ๐น๐ฎ๐ฟ๐ ๐ฟ๐ฎ๐ป๐ด๐ฒ: ๐ฅ๐ ๐ฐ๐ฌ๐ฌ๐ฌ๐ฌ๐ฌ๐ฌ - ๐ฅ๐ ๐ฒ๐ฌ๐ฌ๐ฌ๐ฌ๐ฌ๐ฌ (๐ถ๐ฒ ๐๐ก๐ฅ ๐ฐ๐ฌ-๐ฒ๐ฌ ๐๐ฃ๐)
Experience: 6+ yrs
Location: NCR, Delhi, India
As a Lead Data Engineer, you will spearhead the design, development, and optimization of large-scale data pipelines that support both batch and realโtime data processing initiatives across the organization. You will collaborate closely with data scientists, product managers, and infrastructure teams to ensure data integrity, scalability, and high performance at every stage of the pipeline, while driving continuous improvement through automation, monitoring, and best practices. Your leadership will shape the data architecture, protect data quality, and enable timely, actionable insights that inform business decisions. In a fastโmoving environment, you will balance technical excellence with strategic vision, fostering a culture of collaboration and innovation within the data engineering team.
๐๐ฒ๐ ๐ฅ๐ฒ๐๐ฝ๐ผ๐ป๐๐ถ๐ฏ๐ถ๐น๐ถ๐๐ถ๐ฒ๐
โข Design and implement robust, endโtoโend batch data pipelines using tools such as Apache Spark, Hadoop, and Airflow.
โข Develop realโtime streaming pipelines with technologies like Apache Kafka, Flink, or Spark Structured Streaming to deliver lowโlatency analytics.
โข Optimize pipeline performance through partitioning, indexing, and efficient data format selection (e.g., Parquet, ORC).
โข Establish data governance, quality checks, and automated monitoring to ensure data reliability and compliance.
โข Lead capacity planning and scaling strategies for data infrastructure, including cluster management and cost optimization.
โข Mentor and coach junior data engineers, fostering a collaborative and growthโoriented team culture.
โข Collaborate with Data Scientists to translate analytical requirements into scalable pipeline solutions.
โข Drive integration of data pipelines with downstream applications, dashboards, and reporting tools.
โข Stay current with emerging data engineering technologies and evaluate their applicability to business needs.
โข Participate in incident response and root cause analysis to quickly resolve data pipeline failures.
๐ช๐ต๐ฎ๐ ๐ ๐ฎ๐ธ๐ฒ๐ ๐ฌ๐ผ๐ ๐ฎ ๐๐ฟ๐ฒ๐ฎ๐ ๐๐ถ๐
โข 6+ years of experience in data engineering with a proven track record in both batch and realโtime processing.
โข Deep knowledge of distributed computing frameworks such as Hadoop, Spark, and Flink.
โข Proficiency in message queuing and stream processing systems like Kafka, Pulsar, or RabbitMQ.
โข Strong scripting skills in Python, Scala, or Java, and experience with SQL on large datasets.
โข Handsโon experience with cloud data services (AWS Glue, Redshift, BigQuery, Snowflake, or Azure Data Factory).
โข Familiarity with containerization (Docker, Kubernetes) and CI/CD pipelines for data workloads.
โข Demonstrated ability to architect and scale data pipelines to handle petabyteโscale volumes.
โข Excellent communication skills, capable of translating technical concepts for nonโtechnical stakeholders.
โข A proactive problemโsolver with a passion for data quality, governance, and automation.
โข Leadership aptitude, with experience building, mentoring, and growing data engineering teams.
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Similar jobs
- B2
Senior Fabric Lead Data Engineer - R01572945 Auto Apply
Brillio 2 ยท Bengaluru ยท Hybrid ยท Posted 22 Oct 2026 ๐ฎ๐ณ
SQLGitApache Spark+6TechnologySalary not listed - KC
Data Engineer IRecently posted Auto Apply
Kaseya Careers ยท Bengaluru ยท Hybrid ยท Posted 16 Nov 2026 ๐ฎ๐ณ
SQLGitAWS+10TechnologySalary not listed - CO

Data EngineerRecently posted Auto Apply
Conga ยท Bengaluru ยท Hybrid ยท Posted 16 Nov 2026 ๐ฎ๐ณ
SQLAWSETL+10TechnologySalary not listed - WE
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekdayworks ยท Bengaluru ยท On-site ยท Posted 16 Nov 2026 ๐ฎ๐ณ
Technologyโน1400k - โน2000k/yr - V1
Senior Oracle Fusion Data Migration ConsultantRecently posted Auto Apply
Version 1 ยท Bengaluru, KA ยท Hybrid ยท Posted 16 Nov 2026 ๐ฎ๐ณ
OraclePL/SQLSQL+6TechnologySalary not listed - WA
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekday AI ยท Chennai, Tamil Nadu ยท On-site ยท Posted 16 Nov 2026 ๐ฎ๐ณ
GCPSQLAirflow+6TechnologySalary not listed - WA
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekday AI ยท Hyderabad, Telangana ยท On-site ยท Posted 16 Nov 2026 ๐ฎ๐ณ
GCPSQLAirflow+6TechnologySalary not listed - WA
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekday AI ยท Pune, Maharashtra ยท On-site ยท Posted 16 Nov 2026 ๐ฎ๐ณ
GCPSQLAirflow+6TechnologySalary not listed - WA
GCP(Big query + SQL) Data EngineerRecently posted Auto Apply
Weekday AI ยท Bengaluru, Karnataka ยท On-site ยท Posted 16 Nov 2026 ๐ฎ๐ณ
GCPSQLAirflow+6TechnologySalary not listed - EX
Technical Data managerRecently posted Auto Apply
Excelya ยท Bengaluru, Karnataka ยท Hybrid ยท Posted 16 Nov 2026 ๐ฎ๐ณ
GCPOracleMicrosoft Excel+2TechnologySalary not listed - PA
Data Analyst / Data EngineerRecently posted Auto Apply
Pavago ยท India ยท Remote ยท Posted 13 Nov 2026 ๐ฎ๐ณ
OnboardingSQLMicrosoft Excel+6TechnologySalary not listed - BG
KNIME SpecialistRecently posted Auto Apply
Bosch Group ยท bengaluru ยท On-site ยท Posted 13 Nov 2026 ๐ฎ๐ณ
SQLAWSETL+5TechnologySalary not listed