In one of the application, In memory cache is being used, Is there a way to refresh/update the same considering there are multiple instances of that application is running. Use of centralised cache like redis is seems to be an overkill to that problem.
One solution i thought(it seems to overkill too) but will not work, to have a rabbitmq queue, with fan-out of type exchange but that fan-out basically works with queues not consumers, so if one queue have 5 consumers, published event will go only to 1 consumers in round robin manner.
Second one, i am thinking to use redis pub-sub model, in which i am assuming published event will consume by all redis-clients of single application instances. But i am not sure it will work or not.
Wondering if there is any way in which one can update the in memory cache to all application instances(these instances can be scale up or down according to throughput). Or anyone solved this type of problem statement.
For your problem, a centralized cache is the best option, without being overkill.
MQ solution: You should have queues (RabbitMQ) or consumer groups (Kafka) for each instance, so that all instances can receive data updates.
Redis pub/sub: All Redis clients should subscribe to the same channel, and it will work fine in your situation. Once message published, all subscribers will receive the updated data. Redis pub/sub works in a broadcasting manner.
Alternatively, you can implement a service center to manage all the instances you need. Once an instance is initialized, it can establish a socket connection to the service center to receive or publish any changes. And when receiving any updates from instance, service center will broadcasting the updates to all instances it managed using established socket connections.
It seems that a centralized cache is the easiest way to solve your problem.