Databricks Architect /Sr ETL Developer

We are looking for a skilled Databricks Data/ETL Developer with experience in designing, developing, and optimizing data pipelines using Databricks, PySpark, and SQL. The candidate will be responsible for data integration, transformation, and processing to support enterprise data engineering and analytics initiatives.
Good customer presence and ability to answer a variety of Databricks questions, GitHub and CI/CD experience.

Roles & Responsibilities:

• Develop and maintain scalable ETL/ELT pipelines using Databricks, PySpark, and SQL.

• Build data ingestion and transformation workflows for structured and unstructured data.

• Work with Delta Lake, Apache Spark, and cloud platforms such as Azure or AWS.

• Implement data quality checks, error handling, and performance optimization.

• Develop incremental data loads, data cleansing, and data validation processes.

• Collaborate with data engineers, analysts, and business teams to deliver reliable data solutions.

• Troubleshoot pipeline failures and resolve data processing issues.

Required Skills:

• Data Modeling experience designing Delta Lake tables and the Bronze-Silver-Gold Medallion Architecture.
• Understanding of Spark for performance optimizations including Delta Lake optimizations.
• Experience with cloud service provider (Azure) services used in Databricks’ implementations (e.g., Azure ALDS, IAM)
• Strong experience with Databricks, PySpark, Python, and SQL.

• Hands-on experience with ETL/ELT development and data pipeline design.

• Knowledge of Spark, Delta Lake, and data lake/lakehouse architecture.

• Experience with Azure Data Factory or equivalent orchestration tools is preferred.

• Understanding of data modeling, data quality, and performance tuning.

• Experience with cloud data platforms such as Azure or AWS.