I am using python 2.6.4. I have some code which fetches some data, puts it in a queue. After some time, I start pulling the data from the queue.
q = multiprocessing.Manager().Queue()
p = multiprocessing.Process(target=build_queue, args=(q,))
# build_queue calls q.put and puts an object into queue.
# The process of getting the objects to put into queue is slow.
# Later, I use asynchronous processes to pull data from the queue
pool = Pool(processes=4)
for i in range(0, 4):
pool.apply_async(pull_data_from_queue, (q))
time.sleep(120)
pool.close()
# Wait for processes to finish
pool.join()
In the middle of the operations, the program exits with message: “File "(string)", line 2, in put”
If I put a try-except block around the q.put() and print Exception, err - the print message is: [Errno 32] Broken pipe
The exact error is:
File "test.py", line 16, in build_queue
q.put(y)
File "<string>", line 2, in put
File "/opt/python/default-2.6/lib/python2.6/multiprocessing/managers.py", line 725, in _callmethod
conn.send((self._id, methodname, args, kwds))
IOError: [Errno 32] Broken pipe
Any idea why the queue.put throws a message “File "(string)", line 2, in put” and how could I solve this?