Databricks / Python Data Engineer

Databricks / Python Data Engineer

📍 Location: Ohio(Hybrid)
💼 Duration: Long Term Contract

We are seeking an experienced Databricks / Python Data Engineer to design, develop, and optimize enterprise-scale data pipelines and Lakehouse solutions using Python, Databricks, Apache Spark and PySpark.

🔑 Key Responsibilities

• Build scalable ETL/ELT pipelines using Python, PySpark and Databricks
• Develop batch and near-real-time data pipelines
• Implement Bronze / Silver / Gold Medallion Architecture
• Work with Delta Lake and cloud object storage
• Optimize Spark jobs for performance, scalability and cost
• Implement data quality, schema validation, lineage and observability
• Develop enterprise integrations using APIs, files and cloud platforms
• Manage Databricks Workflows / DLT / Unity Catalog
• Build automated CI/CD pipelines
• Use GitHub-based development workflows
• Leverage GitHub Copilot, Claude, Cursor or similar AI coding assistants
• Participate in architecture, code reviews and Agile/Scrum ceremonies
• Mentor junior/mid-level data engineers

🛠️ Required Skills

• 5+ years Data Engineering
• Strong Python
• Databricks
• Apache Spark / PySpark
• SQL
• ETL/ELT
• Delta Lake
• Lakehouse / Medallion Architecture
• Data modeling
• Data quality & schema evolution
• Spark performance tuning
• Cloud object storage
• CI/CD & Git
• Workflow orchestration

⭐ Preferred

• Delta Live Tables
• Databricks Workflows
• Unity Catalog
• Structured Streaming / Kafka
• dbt
• Terraform / IaC
• Azure Data Factory / Azure DevOps
• Data observability
• Enterprise API integrations
• Claude / GitHub Copilot / Cursor / AI-assisted development