Lead Databricks Engineer

MUST BE LOCAL TO TX

Key Responsibilities

• Design and implement end-to-end data architecture using Databricks Lakehouse platform
• Define data engineering best practices and build scalable ETL/ELT pipelines
• Architect solutions leveraging Apache Spark (PySpark/Scala) and Databricks workflows
• Implement Delta Lake for data reliability, performance, and governance
• Collaborate with stakeholders to understand business requirements and translate them into technical solutions
• Optimize performance, cost, and scalability of data pipelines
• Provide technical leadership, mentoring data engineers and developers
• Ensure data security, governance, and compliance standards are met
• Integrate Databricks with cloud services (Azure, AWS, or Google Cloud Platform) and other enterprise systems
• Support real-time and batch data processing use cases

Required Skills & Qualifications

Technical Skills

• Strong hands-on experience with Databricks Platform
• Expertise in Apache Spark (PySpark / Scala / SQL)
• Experience with Delta Lake, Unity Catalog, and Lakehouse architecture
• Proficiency in data pipelines, ETL/ELT frameworks
• Strong experience in SQL and data modeling
• Hands-on experience with cloud platforms:
• Azure (ADF, ADLS, Synapse) OR
• AWS (S3, Glue, EMR) OR