Quick overview
Job description
Primary Skills
- ETL Fundamentals, SQL, SQL (Basic + Advanced), Python, Data Warehousing, Modern Data Platform Fundamentals, Data Modelling Fundamentals, PLSQL, T-SQL, Stored Procedures
Specialization
- Databricks Engineering: Data Architect
Job requirements
-
Experience Range: With at least 5 years of experience in data engineering, including 2+ years specifically working with Databricks and modern data platforms
Key Responsibilities:
1. Architect and optimize scalable data solutions on Databricks to ensure high performance and reliability
2. Develop and maintain robust ETL pipelines using advanced SQL, Python, and Databricks features for processing large-scale datasets
3. Design and implement data warehousing solutions, including data modeling and schema design, to support analytics and business intelligence
4. Automate data transformations and integrations by developing, testing, and deploying complex stored procedures with PLSQL and T-SQL
5. Lead requirements gathering and solution architecture with cross-functional teams to deliver high-quality data products
6. Monitor and troubleshoot data workflows, optimizing performance to maintain data accuracy, integrity, and availability
7. Enforce best practices in data governance, security, and compliance within Databricks environments
8. Document technical designs, engineering processes, and operational procedures for ongoing support and knowledge sharingRequired Skills:
1. Expertise in Databricks platform and ecosystem
2. Advanced ETL pipeline development
3. Advanced SQL, including query optimization and complex joins
4. Python programming for data engineering
5. Data warehousing implementation and data modeling
6. PLSQL and T-SQL for stored procedure development
7. Experience with modern data platform architecturesPreferred Skills:
1. Integration of Databricks with Azure or AWS cloud platforms
2. Proficiency in Apache Spark and distributed data processing
3. Experience with CI/CD automation for data engineering
4. Familiarity with data quality frameworks
5. Experience with real-time data streaming and processingDesired Qualifications:
1. Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related discipline
2. Databricks Data Engineer Associate certification or higher
3. Microsoft Certified: Azure Data Engineer Associate or equivalent SQL/Python certification
Haystack's AI agent can fill in and submit this application for you. How AI auto apply works
Will your resume get past this employer's screening? Check your resume against this job, free