In Redis how to separate records from 1st set by excluding records from 2nd sorted sets.?

Question

I have two sorted sets, I want to separate records from 1st set and store in new list/sorted set by excluding records from 2nd sorted sets.

Below is an example:

set1: 1,2,3,4,5 set2: 3,5,7,8,9

output: 1,2,4

EDIT: I have figured out the way to load the script and use eval to execute script from nodejs.

Strange is when I executed your script even for 5-10 records it taking 1 second to process, which making me doubtful how much scalable it is if I have thousands of records.

Below is my sample nodejs code:

hsetxx = 'redis.call("ZINTERSTORE","temp",2,"set11","set21","weights",1,0) redis.call("ZUNIONSTORE","result",2,"set11","temp","weights",1,-1) redis.call("ZREMRANGEBYSCORE","result",0,0)';

var redis = require('redis');
var client = redis.createClient('14470', connection);

client.on('connect', function() {
    console.log('Connected to Redis');
});

client.script('load',hsetxx,function(err, result) {
     console.log(err+'------------'+result);
 });

client.zadd('set11', 1,1,1,2,1,3,1,4,1,5);
client.zadd('set21', 1,1,1,5);

client.evalsha(
 '39c0da298cab6a6223b4d1e8222cf6d6a84e67b1', //lua source 
 0,
 function(err, result) {
     client.zrange('result', 0, -1, function(err, result) {
          console.log(err+'------------'+result);
      });
 }
);

Answer 1

Check this question:

What you could do is first create a temporary set with ZUNIONSTORE and set the intersect's scores to 0. Then do a range excluding the 0, e.g.:

127.0.0.1:6379> ZADD all 1 one 2 two 3 three
(integer) 3
127.0.0.1:6379> SADD disabled two
(integer) 1
127.0.0.1:6379> ZUNIONSTORE tmp 2 all disabled WEIGHTS 1 0 AGGREGATE MIN
(integer) 3
127.0.0.1:6379> ZREVRANGEBYSCORE tmp +inf 1 WITHSCORES
1) "three"
2) "3"
3) "one"
4) "1"

Answer 2

it was great having the discussion earlier. As promised, another approach to address the challenge is with Lua and Redis' EVAL. I don't know how performant it will be but here's a (not too tested) script that mimics SDIFF but for Sorted Sets:

~/src/redis-lua-scripts$ cat zdiff.lua 
-- ZDIFF key [key ...]
-- Returns the elements in the first key that are also present in all other keys

local key = table.remove(KEYS,1)
local elems = redis.call('ZRANGE', key, 0, -1)
local reply = {}

if #KEYS > 0 and #elems > 0 then
  for i, e in ipairs(elems) do
    local exists = true
    for j, k in ipairs(KEYS) do
      local score = redis.call('ZSCORE', k, e)
      if not score then
        exists = false
        break
      end
    end
    if exists then
      reply[#reply+1] = e
    end
  end
end

return reply
~/src/redis-lua-scripts$ redis-cli SCRIPT LOAD "`cat zdiff.lua`"
"e25d895f05dc638be87d13aed64e8d5780f17c99"
~/src/redis-lua-scripts$ redis-cli ZADD zset1 0 a 0 b 0 c 0 d 0 e
(integer) 5
~/src/redis-lua-scripts$ redis-cli ZADD zset2 0 a
(integer) 1
~/src/redis-lua-scripts$ redis-cli ZADD zset3 0 a 0 b
(integer) 2
~/src/redis-lua-scripts$ redis-cli EVALSHA e25d895f05dc638be87d13aed64e8d5780f17c99 3 zset1 zset2 zset3
1) "a"

Answer 3

I think you are looking for SDIFF:

key1 = {a,b,c,d}
key2 = {c}
key3 = {a,c,e}
SDIFF key1 key2 key3 = {b,d}

https://redis.io/commands/sdiff

However, there is no equivalent for sorted sets.