I have a User Microservice, that publishes events when a user is created, modified, or deleted. In addition, I have a Calendar Microservice that needs the user data. The Calendar service, subscribes to the user events and keeps a read only copy of the needed data.
I have just added an Account Microservice that needs the existing user data and I have used the same model as above. What is the best way to get the existing user data into the Account Microservice?
When I did the first Microservice I republished the user-modified event for every user. It was simple, because nothing else was using this event. If I republish now, for the new Account Microservice, the existing Calendar Microservice will also get the events. My logic is idempotent, but this is a lot of wasted work and will only get worse as I get more services.
I've loaded data a lot in the past. I know how to make it work, but I am looking for a best-practice and a way to do it with minimal coordination and dependencies with other services.
What you probably want here is a design in which you pull, rather than push, copies of the events to new systems: see Greg Young, Polyglot Data.
In broad strokes - the subscriber keeps track of the high water mark, and asks for more events after some mark, and then the event store answers that query with zero or more new events, in order. The twitter timeline api can give you a sense for how that might work -- sadly, the images that explained the ideas seem to have been removed from the docs, but some of them have been captured here at stack overflow, and by the wayback machine