Breaking the Rules: RPC Pattern with Apache Kafka and Karafka

Table of Contents

• 1 Introduction
• 2 Architecture Overview2.1 Core Components2.2 Implementation Flow2.3 Design Considerations
• 2.1 Core Components
• 2.2 Implementation Flow
• 2.3 Design Considerations
• 3 Implementation Components3.1 Topic Configuration3.2 Command Consumer3.3 Synchronization Mechanism3.4 The Client3.5 Using the Implementation
• 3.1 Topic Configuration
• 3.2 Command Consumer
• 3.3 Synchronization Mechanism
• 3.4 The Client
• 3.5 Using the Implementation
• 4 Performance Considerations4.1 Local Testing Results
• 4.1 Local Testing Results
• 5 Summary

Introduction

Using Kafka for Remote Procedure Calls (RPC) might raise eyebrows among seasoned developers. At its core, RPC is a programming technique that creates the illusion of running a function on a local machine when it executes on a remote server. When you make an RPC call, your application sends a request to a remote service, waits for it to execute some code, and then receives the results - all while making it feel like a regular function call in your code.

Apache Kafka, however, was designed as an event log, optimizing for throughput over latency. Yet, sometimes unconventional approaches yield