Senior Data Engineer - Python, Active SC (Outside IR35)

Senior Data Engineer

Ledbury/Malvern/Gloucester/Cheltenham Area

On-site - 5 Days Per Week

Security Clearance: Active SC Clearance Required

Methods Business and Digital Technology is a £100M+ IT Services Consultancy delivering complex digital transformation programmes across government and regulated sectors. We are seeking an experienced Senior Data Engineer to join a growing team supporting the delivery of secure, mission-critical data platforms.

This role requires an individual who is comfortable operating across hybrid environments, combining on-premises infrastructure with Azure cloud technologies to deliver scalable, secure and high-performing data solutions.

The Role:

As a Senior Data Engineer, you will design, build and maintain sophisticated data infrastructure supporting analytics, reporting and operational decision-making. You will work across the full data life cycle, developing robust data pipelines, implementing automation and supporting Real Time data streaming capabilities.

Responsibilities:

Design, develop and maintain scalable data pipelines using Python, Go and Azure Data Factory
Develop ETL and ELT processes supporting structured, semi-structured and unstructured data
Implement and manage hybrid data solutions across on-premises and Azure environments
Build and maintain PostgreSQL database solutions
Manage Elasticsearch deployments and optimisation
Implement workflow automation using Azure Data Factory
Develop and support Real Time event streaming solutions using Kafka and NATS
Support security, authentication and access management using technologies such as Keycloak
Collaborate with technical teams to deliver resilient, secure and scalable data platforms

Skills:

Strong commercial Python development experience
Experience developing ETL/ELT solutions within enterprise environments
Azure Data Factory expertise
Experience with Kafka and/or NATS event streaming technologies
PostgreSQL administration and development experience
Elasticsearch experience
Strong understanding of data security, access controls and platform governance
Experience operating within hybrid cloud and on-premises environments
Active SC Clearance

Desirable:

Azure certifications
Experience with Go development
DevOps and automation experience
Experience supporting highly secure government or defence environments
Willingness and eligibility to progress to DV Clearance

Please note: Active SC Clearance is mandatory for this position. Candidates must also be willing to undergo DV clearance as part of future project requirements.