Guided Project: Reactive Stream Processing with Kafka

As we discussed in Lesson 11, where we were introduced to Reactive Kafka, we delved into the term “reactive systems”. Systems that are flexible, loosely-coupled, scalable, and highly responsive are referred to as reactive systems. Applications must display elasticity, resilience, and responsiveness through a foundation of asynchronous, non-blocking communication in order to achieve this condition. This asynchronous communication establishes a boundary between components that ensures loose coupling, isolation, and location transparency.

Receivers of non-blocking communication can only use resources when they are actively engaged, which reduces system overhead. When explicit message-passing is employed, the system's message queues are shaped, monitored, and, when necessary, backpressure is applied to assist with load management, elasticity, and flow control.

Basically, a Stream of data is sequential information or a record that is transferred from one system to another. They are usually processed using the FIFO (First-In-First-Out) pattern. However, the traditional blocking methodology of data streaming often prohibits a system from processing real-time data while streaming records. Thus, a group of prominent developers gradually realized that they would need an approach to build a “reactive” systems architecture that would facilitate the processing of data while streaming. Consequently, they signed a manifesto, popularly known as the Reactive Manifesto.