I'm trying to implement a program that has a producer and N (N >= 1) workers. They communicate using a message queue. The idea is that the producer sends to the queue "tasks". The workers do a msgrcv() call to get a task and execute some code. After the worker accomplish the task it sends to the queue the result of the computation. The producer will get this message and save the results.
I'm using POSIX message queues and the producer and the workers work concurrently.
The problem behind this program is that exists a scenario that compromises the communication. Each message has a size of ~5000 bytes. The maximum queue size is ~16000 bytes in UNIX Systems, which is the case.
The scenario is: There are 3 tasks in the queue (5000*3 = 15000 bytes). Some worker get one message from the queue (now the queue has 10000 bytes). The worker starts executing the task and, due to the amount of bytes that a worker has to process in each task, the producer sends to the queue another message (the queue is now full). Now after the task is complete the worker tries to send the result to the queue and becomes blocked (the queue is full). The producer tries to send another task to the queue and becomes blocked too.
If I run this program with only one worker this scenario have a considerable probability to occur.
Does anyone have an idea to avoid this situation?
If you can't change the queue size, number of queues to use, or use a different queuing API, then what about queuing less data?
You could place the actual data in shared memory objects or temporary files. Then instead of putting the data in the message, you'd put a file name or shared memory object name and possibly an offset in the message instead. The producer process can then clean it up after receiving the results.
It doesn't necessarily have to be shared memory or temporary files, but the idea is to put the data somewhere other than in the message, and include in the message whatever information is needed for the other process to access it.