Efficiently search a string serialized python dict stored in Redis

Question

I'm using python Twisted server with Redis as a persistent storage medium. I'm using the txredis library to make calls to Redis from Twisted. Now, suppose I have multiple python dicts like:

{"ID" : 10}
{"ID" : 11}
{"ID" : 12}
{"ID" : 13}

I first convert them to JSON strings using json.dumps() and store them in a Redis list using the RPUSH command.

Now, my problem is, I need to search this list (stored in Redis) for a specific ID number and determine its index in the list. I believe I could write a loop, starting from index zero, retrieve each item of the list from Redis, convert it to dict using json.loads() and check the ID. But I feel that it would be very inefficient (making so many calls to Redis). Is there any other efficient way to do this task? Or if someone can suggest a new format I can use to store a list of dicts in Redis and be able to search those dicts.

Answer 1

Why not keep an auxiliary hash in Redis that just contains ID_number -> index in dict, then you can just reference that?

If that's no good for whatever reason (i.e. you don't want to use up the extra space), you can write a short lua script that will deserialize your json and search it for whatever you need server-side, so you don't have to pass the entire json blob back.