Software Engineer, Backend - Kafka (Core & Internal Distribution)

PhonePe Limited (Formerly PhonePe Private Limited) is a technology company that builds digital platforms for Payments, Digital Distribution Services and Financial Services. Headquartered in India, the PhonePe digital payments app was launched in 2016. As of April 2026, PhonePe has over 70 Crore life-till-date registered users and a digital payments acceptance network spread across over 5 Crore merchants. PhonePe’s products and services include Consumer Payments (including Digital Distribution Services), Merchant Payments, Lending and Insurance Distribution services, and New Platforms, which comprise Share.Market (stock broking and mutual funds distribution platform), and Indus Appstore (Android-based mobile app marketplace). Culture:

At PhonePe, we go the extra mile to make sure you can bring your best self to work, Everyday!. And that starts with creating the right environment for you. We empower people and trust them to do the right thing. Here, you own your work from start to finish, right from day one. PhonePe-rs solve complex problems and execute quickly; often building frameworks from scratch. If you’re excited by the idea of building platforms that touch millions, ideating with some of the best minds in the country and executing on your dreams with purpose and speed, join us!

About the Role

We are building an in-house team to maintain and contribute to our internal Apache Kafka distribution. This is not an operations role - you will work directly on the Kafka codebase itself: i.e. the Broker, KRaft controller, protocol layer, clients, and storage engine. 

Working alongside senior technical anchors, you will help maintain our internal Kafka distribution, backport critical fixes, contribute upstream to Apache Kafka, and build benchmarking and testing pipelines for a large-scale production fleet.

We already do this successfully with GlusterFS, where we maintain an in-house distribution backed by deep upstream expertise, and we are replicating that model for Kafka.

Key Responsibilities

• Build and maintain performance testing suites, benchmarks, and validation tools for local patches.

• Maintain our internal Kafka distribution: patch integration, bug fixes, and release testing.

• Contribute upstream to Apache Kafka through test-cases, doc-reviews, bug-reports, bug fixes, code reviews, and community design discussions (mid to long term).

• Diagnose and fix broker and controller issues related to replication, partition handling, KRaft consensus, and storage management.

• Collaborate with platform engineering teams to advise on broker behavior, configuration, and capacity decisions.

• Document internal patches, architectural changes, and upgrade procedures.

What Are We Looking For?

• Codebase & Systems Familiarity: Strong background in Java or Scala with experience reading, modifying, or debugging JVM-based system code.

• Distributed Systems Fundamentals: Understanding of consensus protocols, replication models, log-structured storage, and distributed state management.

• Systems Programming: Familiarity with JVM internals (garbage collection, memory layout) and operating system I/O (page cache, zero-copy, socket tuning).

• Development Experience: Several years of demonstrable experience in systems or infrastructure software engineering, working with distributed middleware or database engines.

Good to Have

• Prior contributions to Apache Kafka or comparable distributed systems (e.g., Pulsar, BookKeeper, ZooKeeper, etcd).

• Familiarity with KRaft consensus or ZooKeeper-less Kafka architecture.

• Experience with Kafka Streams or Kafka Connect internals.

• Experience using Kafka system tests, Trogdor, or duct tape test frameworks.

What This Role Is NOT

• Not an SRE/Operations Role: Dedicated platform teams operate the clusters; you will write and fix the software they run.

• Not Application Development: You will work directly inside Kafka rather than building consumer applications.

• Not a Consulting Role: You will write code, review patches, and ship production-ready software.

What We Offer

• Dedicated focus and time for open-source upstream contributions.

• Opportunity to validate code against a massive production fleet and observe large-scale distributed behavior.

• Direct mentorship from senior systems engineers and technical anchors.

• Proven distribution model and strong organizational backing for core open-source development.