StackExchange.Redis ConnectionMultiplexer.Connect() Intermittently Works

52.5k views Asked by At

I am using StackExchange.Redis to talk to 3 different Redis instances: 1 on the same subnet and 2 remotely. Here's my configuration code:

var configurationOptions = new ConfigurationOptions
{
    EndPoints =
    {
        { host, port }
    },
    KeepAlive = 180,
    Password = password,
    DefaultVersion = new Version("2.8.5"),
    // Needed for cache clear
    AllowAdmin = true
};

var connectionMultiplexer = ConnectionMultiplexer.Connect(configurationOptions );

the last line throws a connection exception approximately 70% of the time:

It was not possible to connect to the redis server(s); to create a disconnected multiplexer, disable AbortOnConnectFail

Why is this intermittent and/or what am I doing wrong? When I ping the Redis server in a command prompt, there is 0% packet loss and a <1 ms response. The network is stable.

Thanks!

EDIT

Here is what the log outputs when it fails:

10.48.68.28:6379,keepAlive=180,version=2.8.5

1 unique nodes specified
Requesting tie-break from 10.48.68.28:6379 > __Booksleeve_TieBreak...
Allowing endpoints 00:00:01 to respond...
10.48.68.28:6379 did not respond
10.48.68.28:6379 failed to nominate (WaitingForActivation)
No masters detected
10.48.68.28:6379: Standalone v2.8.5, master; keep-alive: 00:03:00; int: Connecting; sub: ConnectedEstablished, 1 active; not in use: DidNotRespond
10.48.68.28:6379: int ops=0, qu=4, qs=0, qc=0, wr=0, socks=1; sub ops=2, qu=0, qs=0, qc=0, wr=0, subs=1, sync=2, socks=1
Circular op-count snapshot; int: 0 (0.00 ops/s; spans 10s); sub: 0+2=2 (0.20 ops/s; spans 10s)
Sync timeouts: 0; fire and forget: 0; last heartbeat: -1s ago
Starting heartbeat...
4

There are 4 answers

0
shawn.r.stevens On

This seems to be a somewhat common issue: https://github.com/StackExchange/StackExchange.Redis/issues/42

I just downloaded the new build and haven't seen the problem again. Yet.

0
pranav rastogi On

I was able to work around it by setting a ConnectTimeout on the client when connecting to Redis. Here was my code

 ConnectionMultiplexer connection = 
        ConnectionMultiplexer.Connect("endpoint,password=password,ConnectTimeout=10000");
   
0
Rocks On

For me this error was because total number of connection reached to its max limit i.e. 10K. Restarting Client and kill some of them solved the problem. You can check Azure portal and figure out if you have reached max limit for any of the resources.

0
ankur On

The error you are getting is usually a sign that you have not set abortConnect=false in your connection string. The default value for abortConnect is true, which makes it so that StackExchange.Redis won't reconnect to the server automatically under some conditions. We strongly recommend that you set abortConnect=false in your connection string so that Redis will auto-reconnect in the background if a network blip occurs.

Source: https://stackoverflow.com/a/30918632/2236811

So my init() looked like this

ConfigurationOptions co = new ConfigurationOptions()
{
    SyncTimeout = 500000,
    EndPoints =
    {
        {url,portNumber }
    },
    AbortOnConnectFail = false // this prevents that error
};

seClient = ConnectionMultiplexer.Connect(co);

Thanks