I have an existing PHP webservice that is used to do a complex calculation that takes about 10 seconds to run. The particular client usage pattern is that multiple clients will almost always request the same calculation within the 10 second window.
At the moment, we are simply allowing all to run in paralel. Obviously this is expensive and unnecessary, so looking to see the simplest approach to preventing this regular cache stampede.
In theory, what we would like to do is simply have the first client request go and calculate the result, then put the calculated result into a cache. If subsequent client requests arrive and are looking for that result:
a) If the result is already cached - Great! b) If the result is not already cached, but is being worked on, wait until it appears in the cache before returning that value to the client.
I have ready several different approaches, but would be very interested in hearing any practical advice or recommendations to help with deciding on the simplest and most fool-proof approach.
We haven't made a cache decision yet, so also open to suggestions on which cache to use. Volume is low, so in-memory is potentially fine. Are there any cache implementations that will actually take care of the problem and block on a cache read where another thread is actually working on populating the cache entry?
Is caching the right thing to do? Should we use queue's instead?!
Any thoughts or advice based on practical experience here would be greatly appreciated.
In terms of solutions you can implement I would go for a worker thread to do the calculation based on an incoming request queue. So for the first request the worker would check the cache (miss), calculate the value, store in cache, and return the value. Any subsequent request from the queue will then get the cached value from the worker.
Now if you do not like worker threads and queues you can achieve the same with a mutex on the "check cache, do calculation, store in cache" code block.
In terms of caching solutions that prevent cache stampede (sometimes also called dogpiling) is to use a cache that supports read-through caching. Read-through caching basically means the cache knows where to get the value from in case of a miss. In this case it would be a cache that supports scripting to calculate the value in case it is not in the cache.