Data Engineer

Data Engineer - Hybrid - SC Cleared

To design, build, and optimise data pipelines and integration processes required for migrating HMRC's SAS estate (including SAS 9.4 and Viya 3.5 components) to SAS Viya 4 on DPS AWS. The Data Engineer will ensure data integrity, performance, and compliance throughout the migration and operational phases.

Key Responsibilities

Data Pipeline Development:

• Design and implement ETL processes to migrate data from Legacy SAS environments to SAS Viya 4.
• Optimise data flows for performance and scalability in a cloud-native architecture.

Integration & Transformation:

• Work with SAS developers to refactor existing jobs and ensure compatibility with Viya 4.
• Implement data transformations and validation rules to maintain accuracy and consistency.

Automation & CI/CD:

• Develop automated workflows for data ingestion and processing using DevOps tools (Jenkins, Git).
• Integrate data pipelines into CI/CD frameworks for continuous delivery.

Data Quality & Governance:

• Ensure compliance with HMRC data governance, security, and GDPR standards.
• Implement monitoring and alerting for data quality and pipeline health.

Collaboration:

• Work closely with architects, business analysts, and test teams to align data solutions with migration requirements.
• Support UAT and production readiness activities by providing test data and validation scripts.

Required Skills & Experience

Strong experience in:

• Building and managing ETL pipelines using tools such as SAS Data Integration, Pentaho, or Talend.
• Working with SAS technologies (SAS 9.4, SAS Viya 3.x/4).

Proficiency in:

• SQL, Python, and Scripting for data processing.
• Cloud platforms (AWS) and containerisation (Docker, Kubernetes).

Familiarity with:

• Data modelling, performance tuning, and large-scale data migration.
• Agile delivery and DevOps practices.

Desirable Qualifications:

• Certifications in SAS, AWS Data Engineering, or equivalent.
• Experience in government or regulated environments.
• Knowledge of data virtualisation tools (eg, Denodo).

If you are looking for your next opportunity, please contact me on my mobile