I have been working on Event broker application. We process messages in real time using Kafka brokers. Kafka is a distributed publish subscribe messaging service. Kafka helped us achieve scalability, fault tolerance and high throughput of more than 1 million messages per second. We selected Kafka Streams to process the messages that are stored in kafka. Kafka streams reads the message in the source topic, does processing and then writes the message into sink topic. We have a jdbc consumer which reads the data from the sink topic and then ingests it into the database.
