Multi-Node in TimeScaleDB

Question

I have been trying to run the TimeScaleDB in cluster mode. Totally I have four nodes(service) one is for the access node, and another three are for the data node. The access node stores the chunks in the data node.

My problem is when any one of the data nodes is down, I could not perform any query in access node, Because the access node trying to reach the failed data node. and after a few minutes, I got errors like request connection timeout.

In my scenario, If any one of the data nodes is failed, then the access node inserts the data or gets the data from the other data node. Is this possible? If anyone knows the answer, let me know.

I have followed the following document for the multi-node setup

https://docs.timescale.com/mst/latest/mst-multi-node/

https://docs.timescale.com/timescaledb/latest/how-to-guides/multinode-timescaledb/multinode-config/

https://docs.timescale.com/timescaledb/latest/how-to-guides/multinode-timescaledb/multinode-ha/#node-failures

Answer 1

Based on the design of timescaledb you cannot just store data somewhere else, since the chunk (which belong to a specific time range (and possibly additional dimension) is fixed. Storing somewhere else means that you'd break the time-chunk-relation assumption which would make the data not unavailable afterwards.

There are people who tried (not sure they succeeded) to use something like pgbouncer or pgpool between the access node and the data nodes to achieve data node level hot standby servers which in turn would be their own mini clusters and failover to the hot standby when the primary data node fails.

That said, it'd look like something along those lines:

(access node) -> (pg pooling) -> (dn1 primary | dn1 secondary) + (dn2 primary | dn2 secondary) + (dn3 primary | dn3 secondary)