compute aggregations on event data received using Spring Cloud Stream Kafka

Question

I have just starting using Spring Cloud Stream Kafka.

I made a simple producer and consumer application and that works fine with Spring Cloud Stream Kafka.

I now have a requirement to calculate aggregations(SUM,AVG) on the event data received via the topic and persisted in Kafka.

The aggregations computed would then be served in the UI via Request-Response layer.

I want to know if there is any support for aggregations in Spring cloud stream and/or Kafka.

I am also considering moving the data from Kafka to Mongo store and then doing a aggregation query there (OR) compute the aggregations and only persist them instead of the raw data to mongo data store.

If anyone has worked on a similar scenario can you please let me you know your thoughts on how I can proceed.

Answer 1

You can make use of reactive Spring Cloud Stream application which provides these aggregation, windowing operators out of the box. Here is an example of such application.