Databricks Python Developer

 

 

• 10+ years of experience in Data Engineering, Data Warehousing, or Big Data development.

• 5+ years of hands-on experience with Databricks, Python, and PySpark.

• Strong hands-on experience in Databricks, Python, and PySpark development.

• Experience building and supporting large-scale ETL/ELT data pipelines.

• Strong knowledge of Spark SQL, Delta Lake, and Data Lakehouse architecture.

• Experience working with structured and unstructured data from multiple sources.

• Strong SQL skills and experience with relational databases.

• Experience with data modeling, data transformation, and data quality processes.

• Knowledge of orchestration tools such as Airflow, ADF, or similar.

• Experience with Git, CI/CD, and Agile development methodologies.

• Strong analytical, debugging, and performance tuning skills.

• Excellent verbal and written communication skills.

 

• Design, develop, and maintain data pipelines using Databricks, Python, and PySpark.

• Build scalable data ingestion, transformation, and integration solutions.

• Collaborate with Data Architects, Analysts, and business teams to deliver data solutions.

• Develop and optimize batch and real-time data processing workflows.

• Ensure data quality, integrity, security, and governance standards are met.

• Monitor, troubleshoot, and resolve production data pipeline issues.

• Optimize Spark jobs, SQL queries, and Databricks workloads for performance.

• Participate in data modeling, code reviews, and platform enhancement activities.

• Create technical documentation and support production deployments.

• Participate in Agile ceremonies and contribute to continuous improvement initiatives.