Azure Cosmos DB Engineer

Role Overview

An Azure Cosmos DB Engineer to design, build and optimise cloud-native applications and data pipelines. The role focuses on Cosmos DB (SQL and Mongo APIs), PySpark on Microsoft Fabric, and CI/CD through GitLab. The successful candidate will work closely with solution architects and DevOps teams to integrate Cosmos DB into microservices, while meeting security, governance and data protection standards.

Key Responsibilities

• Develop cloud-native applications using Azure Cosmos DB (SQL API and Mongo API).
• Design NoSQL data models, indexing strategies and partitioning schemes for the best performance, and tune queries to cut RU consumption and improve response times.
• Design and implement PySpark data pipelines on Microsoft Fabric.
• Build and maintain CI/CD pipelines through GitLab DevOps, with automated unit and integration testing in Pytest.
• Work with solution architects and DevOps teams to integrate Cosmos DB into microservices, and ensure compliance with security, governance and data protection standards.

Top 5 Skills

• Azure Cosmos DB: Proven hands-on experience, including query optimisation, throughput management, and a strong understanding of partitioning, indexing and consistency levels, plus the SQL API.
• Change Data Feed and Spark: Strong knowledge of the Cosmos DB Change Data Feed, including integrating and debugging Spark with it.
• PySpark: High proficiency in PySpark and Spark distributed computing concepts.
• SDKs and application design: High proficiency with Cosmos DB SDKs (.NET, Java, Python or Node.js), and experience with concurrency patterns, the CLR and scalable application design; C#, ASP.NET and .NET Core with strong OOP principles desirable.
• Testing and tooling: Hands-on experience with Pytest, Git and CI tools; financial services experience desirable.

Contract Details:

• Rate: £450 per day (IR35 TBC)
• Location: Hybrid in London (1-3 days per week on-site)
• Duration: 6 months Initial