Quick Overview
Seniority
Mid Senior
Employment type
Temporary/Casual
Work mode
Hybrid
Location
Wiltshire, United Kingdom
SQLMachine LearningAnsibleApacheGitLab CIHadoopKubernetesPythonRESTTerraform
Job Description
We are currently recruiting for a Senior Data Engineer to support a Leonardo defence project. This is an opportunity to work on secure, complex data engineering solutions within a Defence environment, collaborating with technical specialists and customer stakeholders.
Key Responsibilities
- Design, build and maintain secure, scalable data solutions across the full life cycle, including data acquisition, integration, storage, processing and analysis.
- Develop and maintain data ingestion and transformation pipelines using Apache NiFi, Python and SQL.
- Design data models, schemas, data contracts and Interface Control Documents (ICDs) to support reliable system integration.
- Integrate data from REST APIs and other source systems, including Parquet and S3-compatible object storage.
- Implement data validation, assurance, metadata management and data traceability.
- Develop tools to support data-driven insights and task automation.
- Work with customer and internal stakeholders to define data, interface and tool requirements.
- Collaborate with DevOps Engineers, Data Scientists, Analysts and Cyber Consultants.
- Take ownership of technical deliverables, ensuring quality, security and maintainability.
Essential Skills & Experience
Candidates should have demonstrable experience in:
- Data engineering within a project or production environment.
- Strong Python and SQL skills, including developing, testing and optimising data pipelines.
- Hands-on experience with Apache NiFi for data ingestion, routing, transformation and orchestration.
- Data modelling and schema design, including data contracts or Interface Control Documents (ICDs).
- Parquet and S3-compatible object storage.
- REST API integration and troubleshooting data exchanges between systems.
- OpenShift or Kubernetes environments.
- GitLab CI/CD practices and automated build, test and deployment pipelines.
- Data validation, quality controls and assurance.
- Metadata management, data lineage and traceability.
- Strong problem-solving, critical thinking and stakeholder communication skills.
Desirable Experience
- A degree in STEM, Data, AI, Programming or a related discipline.
- Experience influencing technical decisions and supporting capability development.
- Exposure to big data technologies such as Spark, Hadoop or NiFi.
- Experience with Ansible and Terraform.
- Understanding of machine learning algorithms and deployment.
Similar jobs
- CB
Data Engineer - DV Cleared
NewCBS Butler
Bristol, Somerset🇬🇧£55 - £60/hrHybrid30 minutes agoSQLAnsibleApache+7Technology - PF
Data Engineering Manager
NewPlaces for People
United Kingdom🇬🇧HybridYesterdayGCPAWSETL+6 - PF
Principal Data Engineer
Places for People
United Kingdom🇬🇧Hybrid4 days agoGCPSQLAirflow+3Technology - AM
Data Engineer
NewAnson Mccade
Bristol, South West England🇬🇧£45k/yrHybrid2 days agoDynamoDBSQLScala+5Technology - NG
Data Engineer - NatWest Group
NewNatWest Group
London🇬🇧Hybrid15 hours agoGCPSQLAWS+3Technology - OR
Azure Databricks Engineer
NewOpus Recruitment Solutions Ltd
London🇬🇧On-site2 hours agoSQLETLAzure+2