Senior Data Engineer (SAS to Databricks Migration)

Senior Data Engineer (SAS to Databricks Migration)

Location: Remote (USA)

Employment Type: Contract Role

Work Schedule: Flexible hours with core collaboration hours aligned to U.S. time zones

About the Role

Are you an experienced Data Engineer looking to make a massive impact in a remote setting? We are looking for a Senior Data Engineer with specialized expertise in migrating legacy systems to modern cloud architectures. In this role, you will lead the modernization of legacy workloads by migrating SAS-based processes to Databricks on Azure, delivering robust, high-performance data pipelines for advanced analytics and reporting.

Employment Type: Contract Role

Work Schedule: Flexible hours with core collaboration hours aligned to U.S. time zones

Key Responsibilities

Cloud Data Engineering: Design, build, and optimize scalable data pipelines on Azure utilizing Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Synapse, and Azure SQL.

Legacy Migration: Interpret, convert, and modernize legacy SAS workloads (including Base SAS and SAS macros) into modern, optimized Databricks environments.

Databricks & Spark Optimization: Leverage advanced Databricks capabilities, including Delta Lake and Apache Spark optimization, to enhance performance.

Query & Performance Tuning: Optimize the performance of complex Spark jobs and SQL queries.

Data Governance & Quality: Implement rigorous data quality, validation, and monitoring practices across all pipelines.

Version Control & CI/CD: Utilize Git and follow CI/CD best practices for seamless deployment and code management.

Required Qualifications

Experience: Minimum of 5+ years of hands-on data engineering experience.

Azure Expertise: Strong hands-on experience with Azure data services (ADF, ADLS, Synapse, Azure SQL).

Databricks & Spark: Advanced Databricks experience, including Delta Lake architecture and Spark optimization.

Programming: Proficiency in Python and PySpark.

SQL & Modeling: Expert-level SQL skills and solid experience with data modeling principles.

SAS Knowledge: Working knowledge of SAS (Base SAS, SAS macros) with a proven ability to interpret and translate legacy SAS code into modern frameworks.

Engineering Best Practices: Experience implementing data quality frameworks, monitoring practices, Git workflows, and CI/CD pipelines.