Start searching
Back to Jobs
Recently postedTechnology

Kafka & Integration Engineer

Apton IncSan Francisco, CA🇺🇸United States

Quick overview

SeniorityMid Senior
Work modeHybrid
LocationSan Francisco, CA, United States
DockerAnsibleApacheJavaKafkaPythonTerraformAzureKubernetesCI/CDAzure DevOps

Job description

Domain / Group: IT-Integration & API
Role Overview
We are looking for an experienced Kafka & Integration Engineer to join our IT-Integration & API team. In this role, you will design, deploy, and maintain highly available Apache Kafka and Confluent streaming platforms on Microsoft Azure, powering our event-driven enterprise architecture.
Key Responsibilities
  • Platform Engineering & Architecture:
  • Design, implement, and manage highly available and scalable Apache Kafka / Confluent platforms hosted on Microsoft Azure.
  • Manage core cluster components including brokers, topics, partitions, replication, security, and platform upgrades.
  • Application & Data Integration:
  • Develop and maintain robust Kafka producers, consumers, Kafka Connect integrations, Kafka Streams applications, and event-driven architectures.
  • Build stream processing applications using Java, Python, or similar modern software engineering stacks.
  • Disaster Recovery & High Availability:
  • Implement and manage Confluent Cluster Linking, cross-cluster replication, and end-to-end Disaster Recovery (DR) solutions.
  • Define and execute failover/failback strategies to satisfy enterprise business continuity requirements.
  • Monitoring & Performance Optimization:
  • Monitor, troubleshoot, and optimize Kafka production and development environments for peak performance, reliability, and scalability.
  • Manage and minimize consumer lag, cluster throughput bottlenecks, and latency issues across environments.
  • Automation, Infrastructure & DevOps:
  • Automate Kafka platform operations and infrastructure management using Terraform, Ansible, Kubernetes, Docker, and Azure DevOps CI/CD pipelines.
  • Establish and maintain platform standards, security controls, operational monitoring, technical documentation, and production support workflows.
Required Technical Qualifications & Skills
  • Event Streaming & Messaging: Apache Kafka, Confluent Platform / Confluent Cloud, Kafka Connect, Kafka Streams, Schema Registry, Confluent Cluster Linking.
  • Cloud Platform: Microsoft Azure.
  • Programming Languages: Java, Python, or equivalent object-oriented / functional languages.
  • DevOps & Infrastructure as Code (IaC): Terraform, Ansible, Docker, Kubernetes (AKS), Azure DevOps CI/CD.
  • Observability & Operations: Consumer lag monitoring, cluster tuning, disaster recovery, security controls, and enterprise support frameworks.

Similar jobs

Apply