Databricks Engineering

Location: United States Experience: 5+ years Job Type: Permanent
### Top Skills & Technologies Databricks | Spark | SQL | PySpark | Python | Azure | ETL | Kubernetes
### Job Summary Thinking Minds' client is seeking a Lead Data Engineer to design and implement scalable solutions that ingest, store, transform, and distribute big data on Azure Cloud. This role combines hands-on engineering with technical leadership, supporting products, tools, and features that enable customer personalization at scale.

Working closely with Product, Data Science, and Architecture, the successful candidate will translate business needs into dependable data solutions and guide delivery throughout the SDLC.
### Ideal Candidate Should Have
Strong ownership of engineering features, from initial design through implementation, testing, and ongoing operations. The ability to connect business-domain understanding with technical decisions and communicate clearly across engineering and business teams. A collaborative leadership style, with an interest in mentoring junior engineers and improving development practices.

Confidence coordinating blended onshore and offshore engineering teams across development and operations.
###

Key Responsibilities
Lead the design and development of Databricks solutions using Spark, SQL, and PySpark to process and transform large datasets. Build data products and workflows using Databricks Workflows, Python, and JSON, supporting reliable data movement and distribution. Develop REST APIs hosted in Kubernetes, and execute unit and integration testing to validate application behavior. Establish technical strategy in partnership with Data Science, Product, Architecture, and business stakeholders.

Own delivery for internal and external-facing applications, support operational needs, and identify opportunities for process improvement.

Collaborate with senior engineers to maintain consistent development practices and mentor junior team members.
### Required Qualifications
Bachelor's degree in Computer Science, Management Information Systems, Mathematics, Business Analytics, or another technically strong discipline.
5+ years of professional data development experience and 5+ years developing with Spark, including Databricks and PySpark.
3+ years of Python development experience.
1+ year of development experience with REST APIs and 1+ year of experience with Kubernetes.

Strong understanding of ETL and data warehousing concepts, together with Agile Principles (Scrum).
###

Preferred Qualifications
Experience aligning technical designs with business-domain requirements and customer-personalization use cases.
Demonstrated ability to improve engineering processes, support shared technical standards, and introduce practical new approaches to complex problems.
Experience balancing hands-on delivery with team guidance and cross-functional collaboration.
Thinking Minds' client offers medical, dental, and vision coverage, retirement savings with matching contributions, wellness support, paid time off, and family leave.
#Dice