Sr. Databricks Data Engineer

Key Responsibilities:

• Design, develop, and optimize scalable data solutions on the Databricks platform.
• Develop large-scale data processing solutions using PySpark or Scala.
• Build and maintain data ingestion pipelines and Delta Live Tables (DLT).
• Implement and support Medallion Architecture (Bronze, Silver, Gold).
• Develop data models and implement data quality, validation, and governance processes.
• Create dashboards, Databricks Apps, and analytical solutions to support business reporting.
• Optimize workloads, monitoring, and operational processes for scalability, security, and cost efficiency.
• Analyze business requirements and identify effective technical solutions.
• Review existing system capabilities and recommend enhancements where appropriate.
• Prepare functional and technical documentation to support application development and system modifications.
• Collaborate with business users and technical teams to gather requirements and support solution delivery.
• Independently perform complex tasks while leveraging strong analytical, problem-solving, and decision-making skills.
• Utilize 8+ years of experience to plan, prioritize, and deliver data engineering solutions effectively.

Minimum Requirements: Candidates that do not meet or exceed the minimum stated requirements (skills/experience) will be displayed to customers but may not be chosen for this opportunity.
Years	Required/Preferred	Experience
8	Required	Experience in IT, supporting the design, development, deployment, or delivery of technology solutions.
8	Required	Experience with Databricks, including building and optimizing ETL/ELT data pipelines using Apache Spark.
8	Required	Experience in data warehousing and dimensional data modeling (star/snowflake schemas).
8	Required	Proficiency in SQL and Python (or Scala) for large-scale data processing.
8	Required	Experience designing and developing dashboards and applications natively within Databricks (e.g., Databricks SQL dashboards, Databricks Apps).
8	Required	Experience implementing data governance, data quality, and data security practices.
8	Required	Experience implementing Lakeflow Declarative Pipelines (formerly Delta Live Tables/DLT) for building and managing production data pipelines.
8	Required	Experience with Delta Lake, medallion architecture (bronze/silver/gold layers), data lakehouse design, and creating and scheduling offline jobs using Lakeflow Jobs (formerly Databricks Workflows) or similar orchestration tools (e.g., Airflow).
8	Required	Excellent communication skills, both verbal and written, including presenting insights to technical and business stakeholders.
1	Preferred	Experience working in public sector or state government environments.
1	Preferred	Databricks certification (e.g., Databricks Certified Data Engineer Associate/Professional).
1	Preferred	Experience with CI/CD practices for data pipelines (DevOps, Git-based workflows).