I have a use case which for which i want to make my system scalable. Lets say some producers are pushing different events in a Message queue. Event 1,Event2, Event3 etc. In backend consumer I want to create configurations for these events, for example aggregate and send notifications(assuming i have a api) for event 1 every hour(cron expression), send notification for event 2 every 5 mins etc. Can someone help me as to how to achieve this design?
Example lets say for event 1 my previous run according to cron expression was 9 am and next run is going to be 10 am, I want to aggregate all events from message queue between 9 am and 10 am and send batched notification. Queue might also have event2 in it.
Using event and messages interchangeably.
create cron jobs for each event: event 1: every 1 hour, event 2: every 5 mins, ...
read the messages from the queue: increment the counter for the current read event(ex: event 1)
When a cron job is called on the scheduled time: read the current value of the counter for this event(ex: event 2) and send notification using the API
you may use redis INCR for aggregating and reading the events count