Create an azure timer function to fetch events from Event Hubs for every 2 mins

Question

Currently, I have a EventHub trigger function that triggers for every event and now my requirement has changed as I need to monitor 2 mins data that we receive and remove any noise from the data. So, Can we create timer function and receive batch data at once? or any other way to receive data from EventHub's for every 2 mins?

Answer 1

Sure, you can use timer trigger specified for 2 mins or used more advanced concept of Azure Functions -> durable functions with monitor pattern

Answer 2

A Timer triggered function is tricky in this case since you then have to manually abort the function after two minutes and manually do the checkpointing. It is not a recommended approach to create batches of Event Data.

An alternative could be to create an Azure Stream Analytics job that gets the data from the Event Hub, does some transformation per data collected during a 2 minute window and send the result to an Azure Function for further processing.

Another alternative is to have a continuous process receive the data and apply transformations in a 2 minute period. Reactive Extensions can be used to apply windows of 2 minutes to the stream of data.

Answer 3

Option 1:

You could use stream analytic job by writing the required query and redirect the required data to new event hub or Azure function as the output. In this case input would be your existing event hub and output could be the Azure function.

Option 2:

You can enable capture feature present in Event Hub and set the destination as blob storage, this can become your source of truth for writing timer trigger or Azure Blob storage trigger for Azure Functions.