zeroRPC: keep running process after response is sent

Question

I'm using Python 2.7 and zeroRPC to make a client and a server communicate. I want the client to send a request to the server and I want the server to send a response to confirm it has received the request. But then I want the server to perform some heavy computations on that request. These computations will take hours and will produce no response, so the client shouldn't keep waiting; the client-server connection should terminate right after the server confirms it has received the request. How can I do this?

Here's (a simplification of) what I have now.

Server code:

impor time
import zerorpc

class HelloRPC(object):
    def hey(self, name):
        print 'Hey, %s' % name # just so I can check that the request was received
        # send response confirming that request was received
        # terminate connection
        time.sleep(100000000000000) # some heavy computations

s = zerorpc.Server(HelloRPC())
s.bind('tcp://0.0.0.0:4242')
s.run()

Client code:

import zerorpc

c = zerorpc.Client()
c.connect('tcp://MyServerName:4242')
c.hey('macarena')

It doesn't work: I get zerorpc.exceptions.LostRemote: Lost remote after 10s heartbeat. I know I can use the heartbeat argument to make the connection last indefinitely but as I said, the computations will take hours and will produce no response, so I don't think I should keep the connection alive.

I've read about gevent but I can't figure out how to use it for this purpose (is it even the right tool for the job?). Should I use Python's multiprocessing package to spawn a subprocess or something like that? How does one go about this sort of thing?

Answer 1

That happens because the server is not sending back heartbeats. The main thread is blocked in some non-gevent-cooperative loop for roughly 10 seconds (2 times the default heartbeat frequency).

Gevent runs your coroutines (greenlets) cooperatively. If a coroutine never yield back to the gevent loop, it blocks the loop.

Maybe your task is CPU bound, in this case, you are trying to stay cooperative, while churning bits on the CPU for a long time. There is two way in which you can run CPU bound code with zerorpc (and gevent):

• periodically yield back to the gevent IOLoop (gevent.sleep(0)). Since the default heartbeat is 5s and a disconnection is assumed after the double of the heartbeat frequency, you must yield roughly every 5s to be safe.

• A more general solution, is to run CPU bound code in theirs own process (for example, one process per CPU):

  • One process will be your service API, accessible to the network (zerorpc.Server with heartbeat enabled). This process never blocks, and respect cooperative scheduling.
  • As many process as you have CPU for your intensive computation.
  • Your service API and your CPU bound processes communicate with zerorpc, but this time with the heartbeat turned off (and probably a really long timeout value).
  • In the service API process, you can control yourself the contention to your CPU bound processes.