How to recreate dynamically created persisted AKKA actors after crash

Question

I am evaluating Akka for a project and I am trying to figure out whether I can achieve High Availability of a service with Akka-Persistence by saving the actors state in a high available data store. (I am not indending to use Akka-Cluster)

I understood the concept of saving an actors state however I am struggling to find a way to recreate actors (not the actors state) on recovery of my failed (crashed) service on a backup host.

In my design there are many dynamically created actors and only a few that are created upon service startup.

How can I find out what actors I have to create upon recovery? Or am I trying to do something fundamentally wrong?

Answer 1

I think that your question is not precise. For instance, you ask How can I find out what actors I have to create upon recovery?... how could we know? Only you know which actors exist in your system. Why not restart them all? Only you seem to know why it is not appropriate, but you didn't tell us why and so we can't help much.

That said, I'll try to make a few guesses and suggest a way to move forward.

The first guesses I'm going to do are that:

• on top of dynamically creating actors, you also dynamically terminate them.
• you want to restart all actors which were dynamically created, and which were still running at the time of the crash, but not those which you already terminated.

If that's right, read on... otherwise, please review your question.

Consider that which actors are running now is part of the state of your system. That state changes with the following events:

• an actor was started dynamically
• an actor was terminated

You could have a persistent actor managing that state, updating it when the above events are emitted.

You should add that actor to the list of those which are always started on service startup.

When the replay of that actor is finished (it will receive the RecoveryCompleted message: http://doc.akka.io/docs/akka/current/scala/persistence.html#recovery-status), you will have rebuilt the which actors are running now state. You then just have to dynamically re-create all those actors based on the state.