Ruby How to create daemon process that will spawn multiple workers

Question

I have a script called 'worker.rb'. When ran this script will perform processing for a while (an hour lets say) and then die.

I need to have another script which is going to be responsible for spawning the worker script above. Let's call this script 'runner.rb'. 'runner.rb' will be called with an argument dictating how many workers it is allowed to spawn.

I'd like runner.rb to do the following: (e.g. 'ruby runner.rb 5') - Query the database for specific values (e.g. got 100 values) - Spawn 5 instances of 'worker.rb' (passing the first 5 values respectively) - Keep checking for any of the instances of 'worker.rb' spawned above to finish and then call 'worker.rb' again with the 6th value from the database and continue this process indefinitely.

I'm using the Daemons gem but am lost as the best way to go about this. The 'runner' script should definitely be daemonized - but should worker also be daemonized?

How should 'runner' go about checking if 'worker' has finished or not? Can this be done using a PID stored in a file?

Answer 1

I used Daemons gem before. But somehow it didn't do well on keep the number of child processes. Then I made a another one, called light_daemon. You could let light_daemon to prefork certain number of worker processes. If one of the worker dies for any reason, the light_daemon will spawn a new one to replace it. If your worker process may cause memory leaking issue, you could let the work to actively die before it gets too big. The parent process will keep the number of the worker processes constant. I used it in the produce site of one of my projects. I worked pretty well.

The following is an example daemon using the light-daemon gem.

require 'rubygems'
require 'light_daemon'

class Client
  def initialize
    @count = 0
  end

  def call
    `echo "process: #{Process.pid}" >> /tmp/light-daemon.txt`
    sleep 3
    @count +=1
    (@count < 100)? true : false
  end
end

LightDaemon::Daemon.start(Client.new, :children=> 2, :pid_file => "/tmp/light-daemon.pid" )

In the daemon, the worker process dies after the method "call" is invoked 100 times. Then a new worker process is spawned and the process continues.