A child for DynamicSupervisor – long-running jobs

Question

Let's say I have a simple module

defmodule MyWorker do
  def do_long_running_work(a, b, c) do
    # ......
  end
end

And DynamicSupervisor

defmodule MyDynamicSupervisor do
  use DynamicSupervisor

  def start_link(_arg) do
    DynamicSupervisor.start_link(__MODULE__, :ok, name: __MODULE__)
  end

  def init(:ok) do
    DynamicSupervisor.init(strategy: :one_for_one)
  end

  def add_my_worker(worker_name, game_id) do
    child_spec = {MyWorker, {worker_name, game_id}}
    DynamicSupervisor.start_child(__MODULE__, child_spec)
  end

  def remove_my_worker(worker_pid) do
    DynamicSupervisor.terminate_child(__MODULE__, worker_pid)
  end

  def children do
    DynamicSupervisor.which_children(__MODULE__)
  end

  def count_children do
    DynamicSupervisor.count_children(__MODULE__)
  end
end

The documentation says the MyWorker has to has have a start_link method. Moreover, the examples there suggest that MyWorker be GenServer. Altough it could also instead contain child_spec without having to use GenServer

However, MyWorker would be doing a long-running a job in do_long_running_work() -- the one which could last hours. Whereas GenServer isn't meant to run long-running jobs in itself, right?

How would I then go about running MyWorker then? What would a simple implementation of MyWorker look like?

What would start_link look like it it wasn't a GenServer in my case?

---

And there'd be also thousands of instances of MyWorker created and run via MyDynamicSupervisor.

I don't want to simply create tasks. I want a) manage them b) see the state of each one c) one how many there're d) have a supervisor to restart them if need be. At runtime, dynamically.

What DynamicSupervisor is for then?

Answer 1

The documentation says the MyWorker has to has have a start_link method. Moreover, the examples there suggest that MyWorker be GenServer.

No. The child specification tells the DynamicSupervisor how to start the child:

The child specification is a map containing up to 6 elements. The first two keys in the following list are required, and the remaining ones are optional:

:id - any term used to identify the child specification internally by the supervisor; defaults to the given module. This key is required. For supervisors, in the case of conflicting :id values, the supervisor will refuse to initialize and require explicit IDs. This is not the case for dynamic supervisors though.

:start - a tuple with the module-function-args to be invoked to start the child process. This key is required.

https://hexdocs.pm/elixir/1.14.4/Supervisor.html#module-child-specification

For instance:

  def add_my_worker(worker_module_name, 
                    worker_function_name, 
                    list_of_worker_function_args, 
                    game_id) do
    child_spec = %{
          id: game_id,
          start: {worker_module_name, worker_function_name, list_of_worker_function_args}
    }
    DynamicSupervisor.start_child(__MODULE__, child_spec)
  end

You could call add_my_worker() like this:

add_my_worker(MyWorker, 
              :do_long_running_work, 
              [10, :abc, "hello"],
              23561)

The DynamicSupervisor would then call:

spawn_link(MyWorker, :do_long_running_work, [10, :abc, "hello"]) 

Answer 2

The long-running job is best backed by a [supervised] Task.

Furthermore, one does not need DynamicSupervisor for that, Task.Supervisor would suffice.