I'm trying to implement CQRS + Event Sourcing backend based on AWS serverless architecture. The issue is with readmodel updating.
When event is saved to Event Store it is published to SNS. SNS then invokes UpdateReadModel lambda.
When several sequential events are published to SNS several lambdas are invoked.
The first issue is that all of them perform same ReadModel update and actually only the one lambda must be invoked for all events.
The second issue is that it may happen that the final ReadModel state would be corrupted after several lambdas executions.
It's required that exactly one lambda is invoked per ReadModel.
The possible solutions:
Use MessageBroker (RabbitMQ or Kafka) + EC2 instance to invoke
UpdateReadModelLambda.Add SQS after SNS + EC2 instance and write code that invokes lambdas and get events from SQS.
But I would prefer not to use instances because it's not fit in with serverless approach and developers should manage and scale containers or MessageBus's manually.
Maybe anyone has already solved that problem or have other solutions for Cloud Services + CQRS + EventSourcing?
Using message brokers for projections is usually a bad idea. You cannot guarantee ordering, you cannot guarantee only-once delivery, you cannot replay events for one read model only because other read models will get the same events again, you cannot have event-driven system since when replaying they will get integration events too. I won't do that.
You can use lambda functions triggered by event store actions (Dynamo inserts for example). Each lambda can be a collection of projections for one aggregate type, or just one single projection. Still, remember, if you get retries, you have a chance of events coming out of order and I am very unsure you can disable parallelisation there.