Kafka Engineer/Administrator

Hi Associate,

Greetings for the day,

We have a position as a Kafka Engineer/Administrator role in St. Louis, MO (or) Greenwood Village, CO. Please find the job description below for this position.

Role: Kafka Engineer/Administrator

Location: St. Louis, MO (or) Greenwood Village, CO

Duration: 24 Months

They are currently working on migrating all Cloudera Kafka from on prem to AWS. They need this person to handle to workload

Key Responsibilities:

**Kafka Administration & Ongoing Support (On-Prem + Cloud)**

• Collaborate with the platform team to administer and operate Kafka clusters across Cloudera, Confluent Platform, and AWS MSK.
• Provide day-to-day operational support: incident response, on-call participation, root-cause analysis, and continuous reliability improvements.
• Manage topics, partitions, replication, retention, quotas, ACLs, and consumer groups.
• Tune brokers, producers, and consumers for throughput, latency, and reliability.
• Handle patching, upgrades, capacity planning, and cluster health/performance troubleshooting.
• Configure and maintain security: TLS/SSL, SASL, mTLS, IAM (for MSK), RBAC/ACLs, and encryption at rest and in transit.

*AWS MSK Modernization (Parallel Workstream) **

• Partner with the team on ongoing workload-transition efforts from on-prem (Cloudera/Confluent) to AWS MSK.
• Contribute to transition strategies (e.g., MirrorMaker 2 / Confluent Replicator / cluster linking) with minimal downtime and no data loss.
• Validate topic parity, offsets, throughput, and data integrity before and after transitions.
• Help define rollback plans and run dry-runs in lower environments.

**CI/CD & Infrastructure as Code**

• Build and maintain Kafka/MSK infrastructure using **Terraform**.
• Manage automated deployments and configuration through **GitLab CI/CD** pipelines.
• Implement GitOps practices for topic, ACL, and cluster configuration management.
• Participate in code reviews and enforce version control and environment promotion (dev -> test -> prod).

**Observability & Monitoring**

• Build and maintain monitoring, alerting, and dashboards in **Datadog** for Kafka/MSK (broker metrics, consumer lag, partition health, JVM, throughput).
• Work with the team to establish SLAs/SLOs and proactive alerting to reduce incidents and mean-time-to-resolution.
• Correlate Kafka metrics with application and infrastructure telemetry.

**Producer / Consumer Coordination & Teamwork**

• Act as a technical liaison between the Kafka platform team and producer/consumer application teams.
• Advise application teams on client configuration, serialization/schema management, partitioning strategy, idempotence, and exactly-once/at-least-once semantics.
• Support onboarding of new producers/consumers and troubleshoot client-side issues (rebalancing, lag, poison messages).
• Coordinate schedules, testing, and change windows with impacted teams.
• Share knowledge, document runbooks, and contribute to a strong team culture of collaboration and continuous improvement.

Required Qualifications:

• **5+ years in data/platform/infrastructure engineering, with 4+ years operating Apache Kafka in production.**

• Hands-on administration experience with **Cloudera** and **Confluent Platform**.
• Production experience with **AWS MSK** (provisioning, configuration, IAM auth, networking).
• Experience supporting and operating Kafka in production, including on-call and incident response.
• Strong **Terraform** and **GitLab CI/CD** experience for infrastructure automation.
• Hands-on **Datadog** experience for monitoring, dashboards, and alerting.
• Deep understanding of Kafka internals: partitions, replication, ISR, offsets, consumer groups, and delivery semantics.
• Solid AWS fundamentals: VPC, security groups, IAM, KMS, CloudWatch, Private Link.
• Strong Linux, networking, and scripting skills (Bash, Python).
• Excellent communication and collaboration skills, with a track record of working effectively in a team.

Preferred / Nice to Have:

• Experience with **MirrorMaker 2**, Confluent Replicator, or cluster linking for replication and workload transition.

• Schema Registry, Kafka Connect, and ksqlDB experience.
• Kafka Streams or Flink familiarity.
• Kubernetes / EKS experience (e.g., Strimzi).
• Certifications: Confluent Certified Administrator, AWS Solutions Architect/DevOps.
• Experience in a regulated or large enterprise environment.

Thanks & Regards

RAJU REDDY |Sr. Team Manager

CIS Technology Inc

Plano, TX 75074

Direct: