Lead Data Engineer (SC Cleared)

About Scrumconnect Consulting

Scrumconnect Consulting is a multi-award-winning UK public sector digital consultancy, positively influencing the lives of over forty million UK citizens. We bring deep expertise in agile delivery, digital transformation, and platform engineering across central government. With a team of over one hundred and twenty-five professionals - more than forty percent of whom are women - we are committed to building inclusive, diverse teams that reflect the communities we serve.

Role Overview

We are looking for Lead Data Engineer to join a dedicated Data Migration Squad supporting the migration of CRS data and reporting capabilities to a new platform.

The role requires strong specialist expertise in data migration, data modelling, ETL, database design, and Azure PostgreSQL. You will take technical ownership of designing and delivering the end-to-end data migration capability, ensuring data is migrated accurately, securely, and within critical programme timelines.

This is a senior data engineering and database-focused role and does not require Java or application development expertise.

Key Responsibilities

• Design and deliver the end-to-end data migration strategy for CRS.
• Develop and implement ETL processes to Azure PostgreSQL to support data migration and future operational requirements.
• Design and maintain logical and physical data models for the target platform.
• Establish the data migration capability as the new platform becomes available.
• Define and implement approaches for moving CRS data into staging and production database environments.
• Develop robust processes for data extraction, transformation and loading.
• Ensure data quality, validation, reconciliation and governance throughout the migration life cycle.
• Support the migration and transformation of data required for 140+ operational reports.
• Work closely with business and technical stakeholders to understand reporting requirements.
• Ensure target data structures support both current and future reporting requirements.
• Provide technical leadership and oversight across data migration and data modelling activities.
• Identify, assess and mitigate data migration risks.
• Troubleshoot complex data and database issues during migration.
• Ensure migration activities are delivered within agreed programme timescales.
• Establish repeatable and maintainable migration processes for ongoing operational use.

Essential Skills & Experience

• Extensive experience as a Senior/Lead Data Engineer with a strong focus on data migration.
• Proven experience designing and delivering complex data migration solutions.
• Strong hands-on experience with ETL development and implementation.
• Strong experience working with Azure PostgreSQL.
• Excellent knowledge of relational databases and SQL.
• Strong experience in logical and physical data modelling.
• Experience designing database structures for operational and reporting requirements.
• Strong understanding of data quality, validation, reconciliation and governance.
• Experience migrating large or complex datasets between platforms.
• Experience designing staging and production data environments.
• Strong understanding of data transformation and mapping.
• Experience supporting reporting and analytics through well-designed data structures.
• Ability to provide technical leadership across data migration workstreams.
• Strong stakeholder management and communication skills.
• Experience identifying and managing data migration risks and dependencies.

Azure/Data Platform Experience

Strong experience with relevant Azure data platform technologies, particularly:

• Azure PostgreSQL
• Azure Data Factory/ETL tooling
• Azure Storage/Data Lake
• Azure database services
• Azure data integration and migration capabilities
• Azure monitoring and operational tooling

Data Engineering & Database Skills

• Advanced SQL
• ETL/ELT
• Data migration
• Data modelling
• Database design
• Data transformation
• Data mapping
• Data validation
• Data reconciliation
• Data quality
• Data governance
• Performance optimisation
• Reporting data structures
• Relational database technologies

Reporting Experience

The role will support the migration and transformation of data required for 140+ operational reports.

Experience should therefore include:

• Understanding reporting data requirements.
• Designing data structures to support operational reporting.
• Data transformation for reporting purposes.
• Data validation and reconciliation between source and target systems.
• Supporting reporting teams and business stakeholders.
• Optimising data structures for reporting performance.

Technical Leadership

The successful candidate will be expected to:

• Take ownership of the technical data migration approach.
• Provide guidance on data modelling and database design.
• Review migration and ETL solutions.
• Identify technical risks and dependencies.
• Work collaboratively with technical architects, developers, reporting teams and business stakeholders.
• Establish good engineering and data migration practices within the squad.

Desirable Experience

• Experience working on large-scale government/public-sector data migration programmes.
• Experience migrating data from Legacy platforms.
• Experience working with Actian or similar Legacy database technologies.
• Experience with complex operational reporting environments.
• Experience working in Agile delivery teams.
• Experience with automated data validation and reconciliation.
• Azure certifications or relevant data engineering certifications.

Ideal Candidate Profile

The ideal candidate will be a Lead/Senior Data Engineer or Database/Data Migration specialist who has hands-on experience taking ownership of complex data migration programmes, particularly where ETL processes, Azure PostgreSQL, data modelling, data quality and reporting are key components.

Diversity and Inclusion

At Scrumconnect Consulting, we believe that diversity drives innovation. We are committed to creating an inclusive environment where every individual is respected, valued, and supported. We welcome applications from candidates of all backgrounds and experiences, and we actively encourage applications from women, people with disabilities, underrepresented communities, and those seeking flexible working arrangements.