Azure Data Engineer

About Scrumconnect Consulting

Scrumconnect Consulting is a multi-award-winning digital consultancy, recognised for delivering impactful and innovative technology solutions across UK government departments. Our work has positively influenced the lives of over 40 million UK citizens.We are passionate about user-centred design, agile delivery, and building digital services that make a real difference. Our teams work at the forefront of innovation, helping organisations transform and deliver high-quality, scalable solutions that truly matter.

Role Overview

We are looking for a proactive and hands-on Azure Data Engineer to join our engineering team. The successful candidate will be responsible for designing, developing, and supporting scalable data solutions using Microsoft Azure.

You will work closely with engineering, data, and delivery teams to build reliable data pipelines and cloud-based data platforms, while following best practices around infrastructure, automation, and source control.

Key Responsibilities

• Design, develop, and maintain scalable Azure data solutions and pipelines.
• Build and manage data pipelines using Azure Data Factory (ADF).
• Develop and optimise data solutions using Azure Synapse Analytics.
• Work with Azure Data Lake for data storage and processing.
• Implement and maintain cloud infrastructure using Terraform.
• Use GitLab for source control, version management, and CI/CD processes.
• Monitor, troubleshoot, and optimise data pipelines and Azure services.
• Ensure data solutions are secure, scalable, reliable, and maintainable.
• Collaborate with developers, data analysts, architects, and delivery teams.
• Proactively identify technical issues and opportunities for improvement.
• Contribute to automation and continuous improvement of engineering practices.

Essential Skills & Experience

• Strong experience working with Microsoft Azure cloud services.
• Hands-on experience with Azure Data Factory (ADF).
• Strong experience with Azure Synapse Analytics.
• Experience working with Azure Data Lake.
• Hands-on experience with Terraform and Infrastructure as Code (IaC).
• Experience using GitLab, including GitLab CI/CD.
• Strong data engineering and data pipeline development experience.
• Good understanding of cloud-based data architecture and integration.
• Strong troubleshooting and problem-solving skills.
• Proactive approach with the ability to work independently and as part of a team.
• Good communication and collaboration skills.

Nice to Have

• Experience with Databricks.
• Experience with Apache Kafka.
• Experience with Kafka Connect.
• Knowledge of event-driven data architectures and Real Time data processing.

Key Technologies

Azure | Azure Data Factory | Azure Synapse Analytics | Azure Data Lake | Terraform | GitLab | Databricks | Kafka | Kafka Connect

Diversity & Inclusion
At Scrumconnect Consulting, we believe that diversity drives innovation and better outcomes. We are committed to fostering an inclusive environment where every individual is valued, respected, and supported.

We welcome applications from candidates of all backgrounds and actively encourage individuals from underrepresented groups, including women, people with disabilities, and diverse communities, as well as those seeking flexible working arrangements.

As a Disability Confident Level 1 employer, we are committed to providing reasonable adjustments throughout the recruitment process and during employment to ensure equal opportunities for all.