Redis Cache Static List search by filter

Question

I have a static Object List about 500 items.

This Object has property like (id [int],Name [string], attribute1 [string], attribute2 [string]).

I have serialized this object into string and stored into redis as a string key value. But I need to filter this list of 500 items based on different user search filters fro object attributes and give subset of this list to user.

I could do it in two ways one is add this list to a table and index and apply search filter using sql. Other is i pull this list from redis each time and de-serialize into list of object and apply the filter using linq. i have redis on a different server so is DB , and i do not want to have a copy of that cache on the each web server as well.

So, what is the best way to do it for a best performance? or is there a different way to do it more faster?

Answer 1

Instead of the two approaches you had listed (using SQL or deserializing and filtering in the client) I'd like to suggest a slightly different one that is essentially a Redis index.

The idea is very simple: assuming your objects' key names are obj1, obj2...objn and that attribute1's values are val1, val2...valm, for each value x of attribute1 create a Redis set with a key name like attribute1:valx:index. The members of the set will be key names of objects that have their attribute1=valx. To make this happen, make sure that:

• When you create an objy with attribute1 value of valx, add objy to the attribute1:valx:index key.
• When you update attribute1 of objy from valx to valz, remove objy from attribute1:valx:index and add it to attribute1:valz:index.
• When you delete objy with attribute1=valx, remove objy from attribute1:valx:index.
• Optionally, the above should happen atomically (i.e. with Lua or a WATCH/MULTI/EXEC block), if that's important to your app.

These points are what's needed to maintain the index. Using the index is done by fetching the members (e.g. SMEMBERS) of the attribute1:valx:index set when looking for objects with filtered by attribute1=valx, and then fetching the actual objects via their key names (e.g. objy). Alternatively, using SORT .. GET (e.g. SORT attribute1:valx:index GET obj*) is another way to do that using just one command.

Repeat the same for attribute2. A couple of additional points:

• Look into using Redis' hashes to store your objects (e.g. HSET objy attribute1 valx) to save on serialization overheads and easier manipulation of values in Redis.
• The simple index described above can be taken much further - consider using set operations such as SUNION, SDIFF & SINTER for more complex filtering options.