Nodejs Cluster with MQTT client

Question

I have a nodejs engine that uses MQTT to subscribe to messages from many IoT devices. As the number of IoT devices has increased, I want to run the nodejs engine in a cluster mode. This results in every cluster receiving all the MQTT messages. Is there a way to avoid this? A way in which one MQTT message will be received only once and the load of servicing the messages can be equally distributed.

Setup:

Nodejs Engine with MQTT client running via pm2 in an EC2 instance. MQTT broker running in another EC2 instance.

Answer 1

You need to use a broker that supports shared subscriptions.

This allows multiple clients to connect to the broker and then all subscribe to the same topic[s] and the broker will deliver each message to only one client in the group in a round robin fashion.

Shared subscriptions are an option part of the MQTT v5 spec and some brokers have none standard implementations in their MQTT v3 brokers.

You can read more about shared subscriptions here