When should I choose STH-Comet or QuantumLeap instead of Cygnus, Draco, Cosmos?
I have been looking at the GE FIWARE catalogue documentation and it is clear that there are several components for data persistence. I have noted that a distinction is made between time series and data persistence. I wanted to know the methods followed to choose the right GE for your solution to persist the context information.
Another question, STH-Comet refers to Short Time Historic data, but what if I want to persist long term time series data? Would QuantumLeap be the option?
Thanks
Multiple Generic Enablers exist to serve the same role, but they usually differ in the detail as to what they actually offer. This is going to be a matter of the throughput and persistence you actually need. Ask yourself the following questions:
Question 1: is this NGSI-v2 (JSON based interoperability used in individual Smart Systems) or NGSI-LD (JSON-LD based interoperability used for Federations and Data Spaces).
If you are using NGSI-LD you can ignore STH-Comet as it only offers a NGSI-v2. If you are using NGSI-v2, your context broker will be Orion and Orion already uses MongoDB, so maybe a single Mongo instance is sufficient for your needs.
Question 2: Which database are you looking at for storage?
As stated, STH-Comet uses MongoDB, QuantumLeap uses CrateDB or PostGres. If you look at Cygnus and Draco they offer persistence to multiple sinks, but their lists don't quite overlap - Cygnus is the only one that offers DynamoDB for example.
Question 3: Will you need to manipulate the data before storage?
Cygnus uses Apache Flume, Draco uses Apache NiFi, so it is easier to verify the manipulation of the data flow using a graphical tool like NiFi.
Question 4: Do you need row-based storage? column based storage or a proper timeseries?
Cygnus offers both row an column based storage, but for proper timeseries (e.g. aggregation queries) something like QuantumLeap will work better.
Question 5: Do you actually need a separate temporal storage at all ?
If you are using an NGSI-LD broker such as Scorpio or Stellio, you can directly use the NGSI-LD Temporal API.
Also note that Time-series can create a lot of data very quickly. For really long term storage, you will have to think about where do you plan to archive the data. Without knowing the full architecture of your use case, it is impossible to give accurate advice.