Issue connecting Bull Queue to redis through docker containers

Question

i have seen this question asked around a couple times, but never found a proper answer that works for me, so i will give it a go.

I have sat up a worker service in its own docker container which uses Bull Queue. Then i have redis in a seperate container. All worked fine when redis was in a container and the worker was running locally, but now the worker just prints Error connecting to Redis: Error: connect ECONNREFUSED 127.0.0.1:6379 at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1278:16) { errno: -111, code: 'ECONNREFUSED', syscall: 'connect', address: '127.0.0.1', port: 6379 }

Which i find kind of strange, since i am referring to the container name when i set up Bull. In fact no matter what i change the host to in the setup, it prints the same error. Unless i run the worker locally, then it prints whichever host i input. I have also tried changing the "connectionName" field without any changes.

Docker compose:

version: '3.1'
services:
  redis:
    image: redis:latest
    restart: always
    container_name: my_cache
    ports:
      - '6379:6379'
    command: redis-server --save 20 1 --loglevel warning --requirepass pirate
    volumes:
      - redis-data:/data
  worker:
    image: worker
    build: ../worker
    environment:
      - REDIS_HOST=local:redis:6379
    ports:
      - "6000:6000"
    links:
      - redis
volumes:
  redis-data:
    driver: local

queue.ts

import Bull from "bull";

export const gameEventQueue = new Bull("game-event", {
    redis: {
        host: 'redis',
        password: "pirate"
    }
});

Bull versions "bull": "^4.12.2", "bull-board": "^2.1.3",

i also tried to set ENV in docker compose like this

 worker:
    image: worker
    build: ../worker
    environment:
      - REDIS_HOST=local:redis:6379

And connect like this,

import Bull from "bull";

export const gameEventQueue = new Bull("game-event", {
    redis: {
        host: process.env.REDIS_HOST,
        password: "pirate"
    }
});

But still no Joy. I appreciate all help

Answer 1

The network name for the Redis container will simply be redis. Here's a setup to demonstrate.

docker-compose.yml

version: '3.1'
services:
  redis:
    image: redis:latest
    restart: always
    container_name: cache
    ports:
     - '6379:6379'
    command: redis-server --save 20 1 --loglevel warning --requirepass pirate
    logging:
      driver: "none"
  worker:
    image: worker
    container_name: worker
    build:
      context: .
      dockerfile: Dockerfile
    environment:
      - REDIS_HOST=redis
      - REDIS_PORT=6379
      - REDIS_PASSWORD=pirate

I'm suppressing output from the redis service just to keep things clean. There are three environment variables being passed to the worker service, with the network host and port for the Redis container and the password.

Now a simple Docker image for the worker. The iputils-ping package is just required so that we can run ping in the worker.

Dockerfile

FROM ubuntu:22.04

RUN apt update -q && apt install -q -y redis-tools iputils-ping

COPY worker.sh .

CMD ["bash", "worker.sh"]

Finally a worker script, which is just a simple BASH script that demonstrates connecting to the Redis container. This is what it does:

1. Uses ping to echo packets off the Redis container.
2. Uses redis-cli to send the PING command to the Redis server.

worker.sh

#!/bin/bash

echo "Worker starting... done!"

echo "Redis at $REDIS_HOST:$REDIS_PORT."

ping -c 5 $REDIS_HOST

echo "Sending PING to Redis..."

redis-cli -h $REDIS_HOST -p $REDIS_PORT -a $REDIS_PASSWORD PING

echo "Done!"

echo "Worker stopping... done!"

And here's what that looks like:

If you look carefully you'll see:

1. Five network pings (immediately after the log message that begins with PING redis. This establishes that the worker container can see the Redis container.
2. A successful Redis PING command, to which the server responds with a PONG (which is also visible in the logs).

Answer 2

My issue was currently that the dockerfile did not run the build steps correctly, so it kept deploying the outdated host to my container, which is why it worked locally