Senior Data Engineer with Java, Apache Spark

We are seeking a Senior Data Engineer to join a technology-driven organization operating in the financial data domain, building a production-grade analytics and data ingestion platform processing large-scale financial datasets. This role focuses on designing, building and operating large-scale data ingestion and analytics solutions for batch and real-time processing of financial data, supporting critical downstream analytics and reporting use cases. Responsibilities Design, develop and maintain Apache Spark batch pipelines and Kafka streaming solutions Build serverless microservices using AWS Lambda and Azure Functions Develop data ingestion connectors for databases, file systems, message queues and financial data feeds Write production-grade code in Java, Scala and Python Manage infrastructure using Terraform across multiple environments Containerize and deploy applications using Docker and Kubernetes (EKS/AKS) Implement and maintain CI/CD pipelines in GitLab CI Write and maintain unit and integration tests with high code coverage Monitor platform health and troubleshoot production issues in distributed systems Requirements 5+ years of experience in data engineering or backend platform development Proficiency in Java and/or Scala as primary languages with Python as secondary Expertise in Apache Spark including Spark SQL and Spark Streaming Skills in Apache Kafka and event-driven architectures Competency in SQL and data modeling Familiarity with AWS cloud services and serverless computing Nice to have Knowledge of Terraform for Infrastructure as Code Skills in Docker and Kubernetes (EKS/AKS) Familiarity with CI/CD pipelines such as GitLab CI, Jenkins or equivalent Background in cloud data warehouses such as Amazon Redshift Understanding of Microsoft Azure services and Azure-native tooling Knowledge of Iceberg, Hudi or Delta Lake