Quick overview
Job description
Primary Skills
- Design, develop, and maintain end-to-end data pipelines using Databricks and PySpark.
- Build and implement Lakehouse architectures utilizing Bronze, Silver, and Gold data layers.
- Develop and manage Delta Lake solutions with ACID transactions, schema enforcement, and data reliability features.
- Create, monitor, and optimize Delta Live Tables (DLT) pipelines.
- Implement scalable and efficient data ingestion processes using Auto Loader.
- Develop and manage real-time data processing solutions using Structured Streaming.
- Orchestrate, schedule, and monitor data workflows using Databricks Workflows.
- Design and implement Lakehouse data models to support reporting, analytics, and business intelligence requirements.
- Establish and enforce data governance, security, and access controls using Unity Catalog.
- Optimize Spark jobs, SQL queries, and overall platform performance to ensure efficiency and scalability.
- Collaborate with cross-functional teams, including data analysts, architects, and business stakeholders, to deliver high-quality data solutions.
- Databricks Platform
- Delta Lake
- Delta Live Tables (DLT)
- Unity Catalog
- Databricks Workflows
- PySpark and Apache Spark
- Structured Streaming
- Auto Loader
- SQL
- Lakehouse Data Modeling
- Strong understanding of data engineering best practices and scalable data architectures
- Azure Data Factory (ADF)
- Azure Synapse Analytics
- Microsoft Purview
- Microsoft Fabric
- AWS Glue
- AWS Lambda
- AWS Step Functions
- Apache Airflow
- DBT
- Fivetran
- Informatica
- Apache Kafka
- Power BI
- Collibra
- Alation
- BigQuery
- Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, or a related discipline.
- Proven experience in designing and implementing cloud-based data engineering solutions and scalable data pipelines.
- Strong analytical, troubleshooting, and problem-solving capabilities.
- Experience working in agile and collaborative environments.
- Excellent communication and stakeholder management skills.
- Hands-on experience with modern Lakehouse architectures and enterprise-scale data platforms.
- Strong understanding of data governance, security, and compliance frameworks.
- Experience delivering both batch and real-time data processing solutions.
- Ability to work independently while collaborating effectively across global teams.
Databricks Engineer
Role Overview
We are seeking a highly skilled Databricks Engineer to design, develop, and optimize scalable data engineering and analytics solutions on the Databricks Lakehouse Platform. The ideal candidate will possess strong expertise in Databricks, PySpark, and SQL, with hands-on experience building batch and real-time data pipelines, implementing Lakehouse architectures, and ensuring data governance and performance optimization.
Key Responsibilities
Required Skills (Must Have)
Preferred Skills (Good to Have)
Azure Ecosystem
AWS Ecosystem
Data Engineering & Integration
Streaming & Analytics
Data Governance
GCP
Qualifications
Preferred Candidate Profile
Key Technologies
Databricks | PySpark | Apache Spark | Delta Lake | Delta Live Tables (DLT) | Unity Catalog | Structured Streaming | Auto Loader | SQL | Lakehouse Architecture | Azure | AWS | Airflow | Kafka | Power BI
Specialization
- Databricks Engineering: Lead Data Engineer
Job requirements
- Design, develop, and maintain end-to-end data pipelines using Databricks and PySpark.
- Build and implement Lakehouse architectures utilizing Bronze, Silver, and Gold data layers.
- Develop and manage Delta Lake solutions with ACID transactions, schema enforcement, and data reliability features.
- Create, monitor, and optimize Delta Live Tables (DLT) pipelines.
- Implement scalable and efficient data ingestion processes using Auto Loader.
- Develop and manage real-time data processing solutions using Structured Streaming.
- Orchestrate, schedule, and monitor data workflows using Databricks Workflows.
- Design and implement Lakehouse data models to support reporting, analytics, and business intelligence requirements.
- Establish and enforce data governance, security, and access controls using Unity Catalog.
- Optimize Spark jobs, SQL queries, and overall platform performance to ensure efficiency and scalability.
- Collaborate with cross-functional teams, including data analysts, architects, and business stakeholders, to deliver high-quality data solutions.
- Databricks Platform
- Delta Lake
- Delta Live Tables (DLT)
- Unity Catalog
- Databricks Workflows
- PySpark and Apache Spark
- Structured Streaming
- Auto Loader
- SQL
- Lakehouse Data Modeling
- Strong understanding of data engineering best practices and scalable data architectures
- Azure Data Factory (ADF)
- Azure Synapse Analytics
- Microsoft Purview
- Microsoft Fabric
- AWS Glue
- AWS Lambda
- AWS Step Functions
- Apache Airflow
- DBT
- Fivetran
- Informatica
- Apache Kafka
- Power BI
- Collibra
- Alation
- BigQuery
- Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, or a related discipline.
- Proven experience in designing and implementing cloud-based data engineering solutions and scalable data pipelines.
- Strong analytical, troubleshooting, and problem-solving capabilities.
- Experience working in agile and collaborative environments.
- Excellent communication and stakeholder management skills.
- Hands-on experience with modern Lakehouse architectures and enterprise-scale data platforms.
- Strong understanding of data governance, security, and compliance frameworks.
- Experience delivering both batch and real-time data processing solutions.
- Ability to work independently while collaborating effectively across global teams.
Databricks Engineer
Role Overview
We are seeking a highly skilled Databricks Engineer to design, develop, and optimize scalable data engineering and analytics solutions on the Databricks Lakehouse Platform. The ideal candidate will possess strong expertise in Databricks, PySpark, and SQL, with hands-on experience building batch and real-time data pipelines, implementing Lakehouse architectures, and ensuring data governance and performance optimization.
Key Responsibilities
Required Skills (Must Have)
Preferred Skills (Good to Have)
Azure Ecosystem
AWS Ecosystem
Data Engineering & Integration
Streaming & Analytics
Data Governance
GCP
Qualifications
Preferred Candidate Profile
Key Technologies
Databricks | PySpark | Apache Spark | Delta Lake | Delta Live Tables (DLT) | Unity Catalog | Structured Streaming | Auto Loader | SQL | Lakehouse Architecture | Azure | AWS | Airflow | Kafka | Power BI
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Similar jobs
- CA

Data Engineer with Kafka - PuneRecently posted Auto Apply
Capco · Pune · Hybrid · Posted 15 Nov 2026 🇮🇳
SQLScalaApache Spark+8TechnologySalary not listed - PA
Data Analyst / Data EngineerRecently posted Auto Apply
Pavago · India · Remote · Posted 13 Nov 2026 🇮🇳
OnboardingSQLMicrosoft Excel+6TechnologySalary not listed - BG
KNIME SpecialistRecently posted Auto Apply
Bosch Group · bengaluru · On-site · Posted 13 Nov 2026 🇮🇳
SQLAWSETL+5TechnologySalary not listed - KN

Staff Data Engineer (Position located in Kochi, India)Recently posted Auto Apply
KnowBe4 · Kochi · Hybrid · Posted 13 Nov 2026 🇮🇳
SQLComplianceData Pipeline+1TechnologySalary not listed - KN

Sr. Data Engineer (Position located in Bengaluru, India)Recently posted Auto Apply
KnowBe4 · Bengaluru · Hybrid · Posted 13 Nov 2026 🇮🇳
SQLPythonTechnologySalary not listed - B2
Senior Lead Data Engineer - R01573561Recently posted Auto Apply
Brillio 2 · Bengaluru · Hybrid · Posted 13 Nov 2026 🇮🇳
GCPSQLShell+11TechnologySalary not listed - NI
Data EngineerRecently posted Auto Apply
NielsenIQ · Chennai, TN · On-site · Posted 13 Nov 2026 🇮🇳
SQLScalaShell+12TechnologySalary not listed - NS
Data EngineerRecently posted Auto Apply
Nemetschek SE2 · Hyderabad, Telangana · Hybrid · Posted 12 Nov 2026 🇮🇳
TechnologySalary not listed - AG

Data EngineerRecently posted Auto Apply
Articore Group · Gurugram, Haryana · Hybrid · Posted 12 Nov 2026 🇮🇳
GCPSQLAWS+8TechnologySalary not listed - SA
Data EngineerRecently posted Auto Apply
Sago · Gurugram, Haryana · Remote · Posted 12 Nov 2026 🇮🇳
SQLGitApache Spark+11TechnologySalary not listed - SL

Senior Data EngineerRecently posted Auto Apply
Sumo Logic · Noida · Hybrid · Posted 12 Nov 2026 🇮🇳
OnboardingSQLGit+13TechnologySalary not listed - SU

Data EngineerRecently posted Auto Apply
Suki · Bengaluru · Hybrid · Posted 12 Nov 2026 🇮🇳
GCPSQLGit+15TechnologySalary not listed