BookSleeve BlockingRemoveLeft is only returning some of the items pushed into the list

Question

I'm trying to put together a super simple queuing sample based on BookSleeve.

Here's what I have so far for the queue consumer side:

using System;
using System.Text;
using BookSleeve;

static class Program
{
    static void Main()
    {
        using (var redis = new RedisConnection("127.0.0.1"))
        {
            redis.Open();
            while (true)
            {
                // BRPOP the queue
                var t = redis.Lists.BlockingRemoveFirst(0, new[] { "test" }, 0);
                t.Wait();
                var val = Encoding.UTF8.GetString(t.Result.Item2);
                Console.WriteLine("Got: {0}", val);
            }
        }
    }
}

I'm running the following in LINQPad as the producer:

using(var redis = new RedisConnection("localhost")) 
{
    redis.Open();
    foreach(var i in Enumerable.Range(0, 10)) 
    {
        // LPUSH
        redis.Lists.AddFirst(0, "test", "foo" + i)
            // Call wait to keep things in order
            .Wait();
    }   

    Thread.Sleep(500); // Let Redis do it's thing
    Console.WriteLine("queue length: " + redis.Lists.GetLength(0, "test").Result);
}

I'm getting some really weird results though:

1st run:

Got: foo2
Got: foo5
Got: foo7
Got: foo9

2nd run:

Got: foo1
Got: foo4
Got: foo7

3rd run:

Got: foo0
Got: foo3
Got: foo6
Got: foo8

Also, after every run, LINQPad outputs: queue length: 0, and running LLEN test using the actual redis client returns (integer) 0.

I can't help but feel that I'm missing something here, most likely having to do with the async stuff, but I just can't see it.

My redis version is 2.8.3, BookSleeve version is 1.3.41.

The actual question here is: Why is BookSleeve only returning a subset of the messages that are sent to the redis list?

Answer 1

I simply can't get it to error. Are you absolutey sure you only have 1 consumer? Note that the reason the order is unpredictable is because you are using it as a stack rather than a queue - to get reliable order you should either add to the start and remove from the end, or add to the end and remove from the start. If you add and remove from the start it is random-ish order.

However, it works fine for me. Console output:

Got: foo0
Got: foo1
Got: foo3
Got: foo5
Got: foo6
Got: foo8
Got: foo9
Got: foo7
Got: foo4
Got: foo2
queue length: 0

I suggest using redis-cli in monitor mode - for me, I get the following output:

redis 127.0.0.1:6379> monitor
OK
1389454168.068869 [0 127.0.0.1:4957] "INFO"
1389454168.068869 [0 127.0.0.1:4957] "CONFIG" "GET" "timeout"
1389454168.068869 [0 127.0.0.1:4957] "DEL" "test"
1389454168.129869 [0 127.0.0.1:4958] "INFO"
1389454168.129869 [0 127.0.0.1:4958] "CONFIG" "GET" "timeout"
1389454168.136869 [0 127.0.0.1:4958] "BLPOP" "test" "0"
1389454168.139872 [0 127.0.0.1:4959] "INFO"
1389454168.139872 [0 127.0.0.1:4959] "CONFIG" "GET" "timeout"
1389454168.139872 [0 127.0.0.1:4959] "LPUSH" "test" "foo0"
1389454168.142869 [0 127.0.0.1:4959] "LPUSH" "test" "foo1"
1389454168.142869 [0 127.0.0.1:4958] "BLPOP" "test" "0"
1389454168.142869 [0 127.0.0.1:4959] "LPUSH" "test" "foo2"
1389454168.143871 [0 127.0.0.1:4959] "LPUSH" "test" "foo3"
1389454168.143871 [0 127.0.0.1:4958] "BLPOP" "test" "0"
1389454168.143871 [0 127.0.0.1:4959] "LPUSH" "test" "foo4"
1389454168.143871 [0 127.0.0.1:4959] "LPUSH" "test" "foo5"
1389454168.144870 [0 127.0.0.1:4958] "BLPOP" "test" "0"
1389454168.144870 [0 127.0.0.1:4959] "LPUSH" "test" "foo6"
1389454168.144870 [0 127.0.0.1:4958] "BLPOP" "test" "0"
1389454168.144870 [0 127.0.0.1:4959] "LPUSH" "test" "foo7"
1389454168.145869 [0 127.0.0.1:4959] "LPUSH" "test" "foo8"
1389454168.145869 [0 127.0.0.1:4958] "BLPOP" "test" "0"
1389454168.145869 [0 127.0.0.1:4959] "LPUSH" "test" "foo9"
1389454168.145869 [0 127.0.0.1:4958] "BLPOP" "test" "0"
1389454168.146869 [0 127.0.0.1:4958] "BLPOP" "test" "0"
1389454168.146869 [0 127.0.0.1:4958] "BLPOP" "test" "0"
1389454168.147870 [0 127.0.0.1:4958] "BLPOP" "test" "0"
1389454168.147870 [0 127.0.0.1:4958] "BLPOP" "test" "0"
1389454168.648594 [0 127.0.0.1:4959] "LLEN" "test"

Here connection 4957 is one that just removes the key (to ensure a known state); 4958 is the consumer, and 4959 is the producer.

For reference, I'm using the 2.6.12 Windows build in the above test (because I'm on my Windows laptop, and didn't have time to set up a linux VM) - but: I would expect the same on the official linux builds too.

So again: are you sure (via redis-cli) that you only have 2 connections involved here?

If I change to AddLast:

Got: foo0
Got: foo1
Got: foo2
Got: foo3
Got: foo4
Got: foo5
Got: foo6
Got: foo7
Got: foo8
Got: foo9
queue length: 0