Data Engineer

Albany Beck is a specialist consultancy that partners with leading organisations to deliver high-quality talent, technology solutions, and data-driven transformation. With a strong focus on innovation, capability development, and long-term collaboration, Albany Beck helps clients modernise their operations and build scalable digital futures.

As a Data Engineer within Albany Beck, you will play a key role in supporting one of our major clients by designing, building, and maintaining high-quality data pipelines and models. You will contribute to a modern cloud-based data platform that enables smarter decision-making, improved operational efficiency, and continuous innovation.

Key Responsibilities

• Design, build, and maintain ELT pipelines using Azure Data Factory, Databricks, and SQL Server.
• Implement Medallion-layer data models with clean, well-structured, and maintainable code.
• Ensure pipelines meet high standards for scalability, performance, reliability, and security.
• Embed data quality checks, lineage, and observability into all data workflows.
• Translate business requirements into robust, scalable data engineering solutions.
• Collaborate with architects, governance teams, and application engineers.
• Contribute to technical design discussions and help shape best practice standards.
• Contribute to Agile delivery processes including estimation, refinement, and peer reviews.
• Enhance CI/CD pipelines, testing automation, and engineering quality standards.
• Optimise existing pipelines and investigate production issues, ensuring long-term stability.

Requirements

• Hands‑on experience with Azure Data Factory, Databricks (Python/SQL), and SQL Server.
• Strong understanding of ELT development, data modelling, and data governance.
• Experience working with CI/CD, automation, and testing frameworks.
• Strong communication skills with the ability to collaborate effectively across teams.
• Experience with Microsoft Purview or other metadata catalogue tools.
• Familiarity with Medallion architecture and modern cloud‑based data platforms.

Seniority level

Mid‑Senior level

Employment type

Full‑time

Job function

Finance

Industries

Business Consulting and Services

#J-18808-Ljbffr