Using AKKA FSM with many concurrent instances

Question

Looking for a bit of sage advice please.

I have a simple order management FSM with half a dozen states. I am spec'ing my system to support 10K orders an hour peak. Each order will take between 10 to 120 seconds to traverse the FSM. Some transitions will call out to third party APIs. There is also a persistent data store to hold information on the order and progress.

I am thinking of using AKKA FSM with an instance of the FSM per concurrent order. Before I spend too much time on this project I'm looking for a sanity check and, if not a silly idea, I would appreciate pointers to any areas which I should be paying extra attention.

Thanks for any help!

Answer 1

Representing each order as an FSM actor should be fine. My "bit of sage advice" is that if any of the third-party API calls are blocking or long-running, then delegate those calls to other actors that run on a dedicated dispatcher, as described here.

For example, the following actor makes a blocking call, simulating the use of a third-party API, and sends the result of that call to the sender:

class LegacyApiActor extends Actor {
  implicit val executionContext: ExecutionContext =
    context.system.dispatchers.lookup("my-blocking-dispatcher")

  def receive = {
    case MakeApiCall =>
      val currentSender = sender()

      Future {
        Thread.sleep(10000)
        currentSender ! ApiResult("result")
      }
  }
}

And the following is an excerpt from an FSM actor that has a state that waits for the result of the API call, where legacyActor is a reference to a LegacyApiActor:

onTransition {
  case SomeState -> WaitForLegacyApiResult =>
    legacyActor ! MakeApiCall
  case ...
}

when(WaitForLegacyApiResult) {
  case Event(ApiResult(res), ...) => // response from legacyActor
    goto(DifferentState) ...
}

To reiterate the point, don't make blocking or long-running calls in your order/FSM actors; isolate those calls in other actors that use dedicated dispatchers.