Siddhi partition by request time stamp

Question

How do you partition a request by timestamp?

The first minute goes to the first stream and then the rest goes to a second stream for three hours. Then after The three hours, the first minute goes the first stream and the rest go the second stream.

Thanks

Answer 1

On a high level we can do something like this:

1. We would need a Siddhi table, where we store a pivot timestamp. We can route events to streams based on this pivot timestamp. Let's call this table: TimeStampTable(pivotTimestamp long).

2. At each event of IncomingStream(eventTimestamp long, ..otherAttributes..), we do a join with TimeStampTable, and check if an entry exists at TimeStampTable.

   • A. If an entry doesn't exist, insert eventTimestamp to TimeStampTable. This is going to function as the pivot timestamp. Get 0 as timeDiff.
   • B. If exists, get eventTimestamp - pivotTimestamp as timeDiff.

3. Based on timeDiff, put a label to each event (basically add a string attribute and pass it to the next stream). I.e, WITHIN_FIRST_MINUTE, and WITHIN_THREE_HOURS.

4. Route events to appropriate streams based on the above label.

5. At step (2.B), if timeDiff >= 3 hours, replace pivotTimestamp with the current timestamp.