I want to test how many subscribers I can connect to a publisher, which is sending out messages fast, but not with a maximum speed, e.g. every microsecond.
The reason is, if I send out messages with maximum speed, I miss messages at the receiver ( High-water-mark ).
I thought, I can use nanosleep(), and it works nice with 20 messages a second ( sleep: 50000000 [ns] ). But with a shorter sleeping time, it gets worse: 195 (5000000), 1700(500000), 16000 (50000) messages. And with even shorter sleeping times, I don't really get more messages. It seems that the sleep-function itself needs some time, I can see this, if I print out timestamps.
So, I think, it is the wrong way to run a function with a specific rate. But I didn't find a way to do that in another way.
Is there a possibility to send out roughly 1000000 messages a second?
Q: How to send out messages with a defined rate?
Given API is v.4.2.3+, one can use a
{ pgm:// | epgm:// }-transport class for this very purpose and setup the adequately tuned.setsockopt( ZMQ_RATE, <kbps> )plus exhibit some additional performance related tweaking of buffer-sizing (ZMQ_SNDBUF,ZMQ_IMMEDIATE,ZMQ_AFFINITY,ZMQ_TOSandZMQ_MULTICAST_MAXTPDU) with some priority-mapping e.t.c., so as to safely get as close to the hardware limits as needed.Q: Is there a possibility to send out roughly 1,000,000 messages a second?
Well, given not more than about a 1000 [ns] per a message-to-wire dispatch latency, a carefull engineering is due to take place.
The best candidate for such rate would be to use the
inproc://transport class, as this does not rely on ZeroMQ'sContext-instance IO-thread(s) performance / bottlenecks inside an external O/S scheduler ( and will definitely work faster than any other kind of available transport-classes ). Still, it depends, if it can meet less than the required 1000 [ns] latency, based on your application design and message sizes ( Zero-Copy being our friend here to better meet the latency deadline ).