How to read records from Kafka using Kafka-Stream by interval

Question

i want to use in Kafka-Stream consumer to read record from Kafka, there is an option to read the records every given interval? for example every 1 minutes?

Answer 1

I am not sure if I understand your question correctly. However, I think there are multiple approaches (depending on what you actually want to achieve which in unclear from the question).

1. Use Kafka Streams DSL (Kafka 0.10): Using Kafka Streams (a Java stream processing library) you can specify a window aggregation as a tumbling window of any size
2. Exploit timestamps (Kafka 0.10): if you want to use KafkaConsumer you can just read messages and check their timestamps to chunk data by intervals
3. System-time based (all Kafka versions): just read messages from Kafka and put messaged in intervals bases in system time. Ie, before you process the next record, you check your local clock to put messages in intervals.