How to use repmgr with dockerized Postgresql?

Question

I'm trying to create Postgresql setup with replication and automatic failover. I wanted it to be reusable, scalable and portable, so I tried to use docker to run Postgres. I also didnt want to reinvent the wheel, so I tried to use repmgr as it is recommended tool when setting up postgres replication.

On first machine, I started master node using docker-compose and simple Dockerfile. Then I added bash script to run after start of container. It looks like master was setup properly.

Then on second machine I tried to run standby. I started postgres in docker and experimented. As we can read on repmgr github repo, to run standby I need to clone master with repmgr command, which uses pg_basebackup internally. Pg_basebackup wont let me clone db into not-empty directory (pgdata), so I need to remove pgdata created by dockerized postgres before cloning. When I do that, container dies, because of course the only service, postgres, dies without its files. So I can not run repmgr command afterwards, because there is no container to run it inside :)

So I tried creating another Dockerfile for standbies only. Inside pgdata is removed, then repmgr clones master, then postgres is started properly. There are no errors. But when I log into master machine and verify if replication is running - it is not.

1. Is it possible to run repmgr for dockerized postgres?
2. If so, how? What am I doing wrong?
3. If not, how can I create postgres cluster replication without manually configuring each server? You know, not treating servers as pets but as cattle etc :)

Files I used and created: docker-compose.yml (both master and standby node):

version: '3'
services:
  db:
    build:
      context: .
    volumes:
      - ${DATABASE_STORAGE_PATH}:/var/lib/postgresql/data
    env_file:
      - .env
    environment:
      - PGDATA=/var/lib/postgresql/data/pgdata
    ports:
      - ${DB_PORT}:5432

Master Dockerfile:

FROM postgres:9.6.4

# install repmgr
RUN apt-get update \
    && apt-get install -y repmgr
# COPY conf files
COPY repmgr.conf .
COPY postgres.replication.conf /var/lib/postgresql/data/pgdata/
RUN echo "include '/var/lib/postgresql/data/pgdata/postgres.replication.conf'" >> /var/lib/postgresql/data/pgdata/postgresql.conf

CMD ["postgres"]

Bash script to run when postgres container (master) is alive:

#!/bin/bash
CONTAINER_ID="replicatest2_db_1"

docker exec -d $CONTAINER_ID createuser -s repmgr -U postgres
docker exec -d $CONTAINER_ID createdb repmgr -O repmgr -U postgres
docker exec -d $CONTAINER_ID su postgres -c 'repmgr -f repmgr.conf master register'

Standby docker-compose is the same. Standby Dockerfile:

FROM postgres:9.6.4

# install repmgr
RUN apt-get update \
    && apt-get install -y repmgr
# COPY repmgr conf
COPY repmgr.conf .

RUN rm -rf /var/lib/postgresql/data/pgdata
RUN su postgres -c "repmgr -h {master-host} -p {master-port} -U repmgr -d repmgr -D /var/lib/postgresql/data/pgdata -f repmgr.conf standby clone"

CMD ["postgres"]

Answer 1

You can use pg-dock, with pg-dock you can run dockerized postgres cluster with repmgr, otherwise you can explore the project and get understand how the things are work.

Good Luck.