Senior Data Engineer

HM81: Senior Data Engineer
Location: Remote Contract until 30/04/2027
Salary: Competitive Day Rate IR35: In Scope
Overview:
First Military Recruitment are currently seeking a Senior Data Engineer on behalf of one of our clients.
As a Lead Data Engineer supporting the enterprise AI buildout, this role is responsible for leading the design and implementation of complex data flows, pipelines and integrations to enable advanced AI and automated workflow solutions. It will set data engineering best practices, ensure data readiness and security, and coordinate work to turn operational data into secure, production-ready AI capabilities. From an enterprise perspective, the role will ensure that deliverables align and interoperate with broader departmental data architectures and roadmaps.
Our client encourages applications from ex-military personnel however all candidates will be given due consideration.
Duties and Responsibilities:

• Establish data integration standards and lead the engineering of automated data pipelines, ETL/ELT workflows and API connections connecting enterprise SaaS and internal databases to AI applications whilst ensuring alignment with existing pipelines and standards.
• Implement identity and access controls, including SSO, conduct data quality, sensitivity and access checks before indexing, and ensure AI applications respect source-system permissions to prevent data leakage.
• Conduct early-stage data quality diagnostics to test dataset viability for AI automation, configure synthetic test datasets within isolated sandboxes, and structure domain-specific knowledge engines.
• Establish repeatable, secure and maintainable data solutions, managing metadata repositories and configuring observability pipelines across AI workflows, reusing existing repositories and observability pipelines where possible.
• Act as the key bridge between technical engineering partners and business leads, interpreting data requirements and translating complex data risks into clear choices for stakeholders.

Skills and Qualifications:

• Hands-on expertise in building scalable ETL/ELT pipelines, designing robust API integrations, and configuring vector indices or retrieval systems for large-scale enterprise AI architectures.
• Expert knowledge of Identity and Access Management, identity federation, data classification standards, and UK GDPR/DPA compliance frameworks within AI environments.
• Substantial experience conducting quantitative data quality assessments, structuring synthetic testing environments, and evaluating dataset readiness for complex automation or ML models.
• Advanced proficiency in AWS, Python, SQL, and query optimisation, alongside demonstrated experience implementing CI/CD pipelines using Terraform, automated testing, and metadata management best practices.
• Proven ability to translate complex technical data risks into strategic guidance for senior leadership, alongside a track record of leading cross-functional engineering teams in agile environments.
• SC Security Clearance.

Location: Remote Contract until 30/04/2027
Salary: Competitive Day Rate IR35: In Scope