wso2: cep: siddhi: how to make time windows persistent

Question

we are currently testing siddhi wso2 cep.

we need to be able to restart the wso2 server process without loosing data which has been accumulated in a window (for example time window).

Is there a way to make time windows persistent so that the server can be restarted without loosing its state ?

Do we have to configure the server to use persistence ?

Or do we just have to adapt the query and use event table ?

Here a sample of our query:

from every data= DataInStream[state == 1] -> 
   every event = EventInStream[event.no == data.no] within 24h
       insert into duplicatesOutStream data.id as id

Answer 1

You can make the CEP persist it's state by enabling snapshots. The documentation can be found here. But note that this will persist the "entire state" of the CEP "periodically" (configurable in minutes). Since this is done only periodically, you may lose the latest few events that arrived after the last snapshot operation. If you want cep to recover after a crash, this is the easiest option.

Apart from this, there's no other way to make a pre-defined window persistent. But if you want, you can write a your own custom time window (documentation) that will do the persistence and plug it to CEP.

You can also use event tables instead. When you use event tables, you can make sure that all arrived events are persisted. (but one disadvantage of this approach is this can be a performance hit compared to windows + snapshots approach).