Azure Cache/DataCache style Regions in Redis

Question

I am in the planning process of moving a C# ASP.Net web application over to Azure (currently hosted on a single dedicated server) and am looking at caching options. Currently, because we only have one instance of the application running at one time, we have an 'in process' memory cache to relieve the SQL DB of some identical requests.

The process at the moment is to clear certain parts of the cache when the managers/services make a change to those parts of the database, e.g. we have a users table and we'll have keys like "User.{0}" returning a single User record/object and "Users.ForeignKey.{0}" returning all users related to the foreign key. If we update a single user record then we remove the "User.1" key (if the userid = 1) and for ease all of the list collections as they could have changed. We do this by removing keys by pattern, this means that only the affected keys are removed and all others persist.

We've been planning this move to Azure for a while now and when we first started looking at everything the Azure Redis Cache service wasn't available, at least supported, so we looked at the Azure Cache service, based on AppFabric. Using this we decided that we would use DataCache regions to separate the different object types and then just flush the region that was affected, not quite as exact as our current method but OK. Now, since Redis has come on to the scene, we've been looking at that and would prefer to use it if possible. However, it seems that to achieve the same thing we would have to have separate Redis caches for each 'Region'/section, which from how I understand it would mean we would pay for lots of small instances of the Redis Cache service from Azure which would cost quite a lot given that we would need 10+ separately flushable sections to the cache.

Anyone know how to achieve something similar to Azure DataCache Regions with Redis or can you suggest something glaringly obvious that I'm probably missing.

Sorry for such a long question/explanation but I found it difficult to explain what I'm trying to achieve without background/context.

Thanks, Gareth

Update:

I've found a few bash commands that can do the job of deleting keys by pattern, including using the 'KEYS' command here and the lua script EVAL command here.

I'm planning on using the StackExchange.Redis client to interact, does anyone know how to use these types of commands or alternatives to those (to delete keys by pattern) when using StackExchange.Redis?

Thanks for reading, Gareth

Answer 1

You can use this method which leverage the async/await features and redis pipelining to delete keys by pattern using stack exchange redis client

private static Task DeleteKeysByPatternAsync(string pattern)
{
    IDatabase cache1 = Connection.GetDatabase();
    var redisServer1 = Connection.GetServer(Connection.GetEndPoints().First());
    var deleteTasks = new List<Task>();
    var counter = 0;
    foreach (var key in redisServer1.Keys(pattern: pattern, database: 0, pageSize: 5000))
    {
        deleteTasks.Add(cache1.KeyDeleteAsync(key));
        counter++;
        if (counter % 1000 == 0)
            Console.WriteLine($"Delete key tasks created: {counter}");
    }
    return Task.WhenAll(deleteTasks);
}

Then you can use it like this:

DeleteKeysByPatternAsync("*user:*").Wait(); //If you are calling from main method for example where you cant use await.

or

await DeleteKeysByPatternAsync("*user:*"); //If you run from async method

You can tweak the pageSize or receive as method param.

Answer 2

For what I understand by your question you need to group your data according to some criteria (user in your case), so that whenever record related to that criteria is changed, all of data related to that record is also invalidated in cache using a single cache api call.

You can achieve this in Azure using NCache for Azure, a distributed caching solution for azure by Alachisoft which has a rich set of features along with multiple caching topologies.

NCache allows multiple ways to perform this type of operations. One suitable for your use case is data grouping feature that will allow you to group data in groups/subgroups on addition. Data can be later fetched/removed on the basis of groups/subgroups.

NCache also allows to add tags with items being added. These tags can then be used for removing/fetching all data containing one or more specified tags. Querying feature (Delete query) provided in NCache can also be used to remove data satisfying a particular criteria.