Handling large number of same requests in Azure/IIS WebRole

Question

I have a Azure Cloud Service based HTTP API which is currently serving its data out of an Azure SQL database. We also have a in role cache at the WebRole side.

Generally this model is working fine for us but sometimes what happening is that we get a large number of requests for the same resource within a short period of time span and if that resource is not there in the cache, all the requests went directly to our DB which is a problem for us as many time DB is not able to take that much load.

By looking at the nature of the problem, it seems like it should be a pretty common problem which most of the people build API would face. I was thinking if somehow, I can send only 1st request to DB and hold all the remaining till the time when 1st one completes, to control the load going to DB but I did get any good of doing it. Is there any standard/recommended way of doing it in Azure/IIS?

Answer 1

The way we're handling this kind of scenario is by putting calls to the DB in a lock statement. That way only one caller will hit the DB. Here's pseudo code that you can try:

        var cachedItem = ReadFromCache();
        if (cachedItem != null)
        {
            return cachedItem;
        }
        lock(object)
        {
            cachedItem = ReadFromCache();
            if (cachedItem != null)
            {
                return cachedItem;
            }
            var itemsFromDB = ReadFromDB();
            putItemsInCache(itemsFromDB);
            reurn itemsFromDB;
        }