Job Title: Databricks Technical Lead

Location: Irvine, CA onsite

Pay Rate: $65–$70/hr on W2

Experience: 10–15+ Years

Employment Type: W2 Contract

Job Description

We are seeking an experienced Databricks Technical Lead to drive the design, development, and implementation of scalable data engineering solutions. The ideal candidate will have strong hands-on expertise in Databricks, Apache Spark, PySpark, Python, SQL, and cloud-based data platforms, along with proven technical leadership skills.

Key Responsibilities

• Lead the design, development, and optimization of data pipelines using Databricks, Apache Spark, PySpark, and SQL.

• Develop scalable ETL/ELT workflows for processing large volumes of structured and unstructured data.

• Design and implement Lakehouse solutions using Delta Lake and modern data architecture principles.

• Optimize Spark jobs, queries, and data processing workflows for performance, scalability, and cost efficiency.

• Collaborate with architects, business stakeholders, and engineering teams to translate business requirements into technical solutions.

• Establish coding standards, conduct code reviews, troubleshoot issues, and mentor data engineers.

• Implement data quality checks, monitoring, error handling, and data validation processes.

• Support CI/CD pipelines, automated deployments, and version control using Git and related tools.

• Ensure data security, governance, reliability, and compliance with enterprise standards.

Required Skills

• 10–15+ years of overall IT or data engineering experience.

• Strong hands-on experience with Databricks, Apache Spark, PySpark, Python, and SQL.

• Experience building and maintaining enterprise-scale data pipelines and ETL/ELT processes.

• Strong understanding of Delta Lake, Lakehouse architecture, and data warehousing concepts.

• Experience with cloud platforms such as Microsoft Azure or AWS.