Python Developer – Databricks

Python Developer – Databricks

Remote
Contract

Job Summary

We are looking for an experienced Python Developer with Databricks expertise to design, develop, and maintain Python-based data processing solutions. The ideal candidate will have strong Python programming skills, hands-on experience with Databricks notebooks and pipelines, and practical knowledge of data processing, analytics, and synthetic test-data generation.

Key Responsibilities

• Develop robust and scalable applications, utilities, and data-processing solutions using Python.
• Build and maintain Databricks notebooks for data transformation, processing, validation, and analysis.
• Develop and manage Databricks pipelines and workflows for automated data processing.
• Create Python scripts and frameworks for synthetic data generation and test-data preparation.
• Generate realistic datasets covering normal, boundary, and exception scenarios for testing.
• Perform data cleansing, transformation, validation, and analysis using Python and Databricks.
• Work with large datasets and troubleshoot data-processing and pipeline issues.
• Develop reusable Python modules, utilities, and automation scripts.
• Analyze data to identify inconsistencies, anomalies, and quality issues.
• Collaborate with QA, data engineering, analytics, and application development teams.
• Optimize Python code and Databricks processing workflows for performance and reliability.
• Troubleshoot failures in notebooks, pipelines, and data-processing jobs.
• Independently manage development activities in a fast-paced environment.

Required Skills

• Strong hands-on Python development experience.
• Hands-on experience with Databricks.
• Strong experience developing and working with Databricks notebooks.
• Experience creating and managing Databricks pipelines/workflows.
• Experience with synthetic data generation or test-data generation.
• Good understanding of data processing and data transformation concepts.
• Good understanding of data analytics and data validation.
• Strong debugging and problem-solving skills.
• Ability to work independently and take ownership of assigned deliverables.

Preferred Skills

• PySpark / Apache Spark
• SQL and database technologies
• ETL/ELT development
• Data quality and validation
• Python automation
• REST APIs
• Cloud platforms such as AWS, Azure, or Google Cloud Platform
• CI/CD and Git
• Agile/Scrum development

Must-Have Skills

Python + Databricks + Databricks Notebooks + Databricks Pipelines + Synthetic/Test Data Generation + Data Processing

Ideal Candidate

A strong candidate should be primarily a Python Developer with substantial hands-on Databricks experience, rather than a purely analytics-focused professional. The person should be comfortable writing production-quality Python code, developing Databricks notebooks and pipelines, and creating test datasets for development and validation activities.