bitnami/postgresql-repmgr docker containers not finding each other

Question

I am using bitnami/postgresql-repmgr docker image for two database containers. In case that both containers running in one server and connecting to each other with my local docker network, they are working with no problem and the replication is done (with the default config noticed in bitnami's dockerhub), but when I seperate them in two servers and publish their ports (the same as all common configs we do for our servers) the replication can't be done and the containers can't find each other as partner nodes.

part of docker-compose config for database one:

  pg-0:
    image: bitnami/postgresql-repmgr:14
    ports:
      - "5432:5432"
    container_name: db1
    volumes:
      - pg_0_data:/bitnami/postgresql
    environment:
      - POSTGRESQL_POSTGRES_PASSWORD=adminpassword
      - POSTGRESQL_USERNAME=customuser
      - POSTGRESQL_PASSWORD=custompassword
      - POSTGRESQL_DATABASE=customdatabase
      - REPMGR_PASSWORD=repmgrpassword
      - REPMGR_PRIMARY_HOST=pg-0
      - REPMGR_PARTNER_NODES=pg-0,< second database servers public ip >
      - REPMGR_NODE_NAME=pg-0
      - REPMGR_NODE_NETWORK_NAME=pg-0
    networks:
      - main

and for database two:

  pg-1:
    image: bitnami/postgresql-repmgr:14
    ports:
      - "5432:5432"
    container_name: db2
    volumes:
      - pg_1_data:/bitnami/postgresql
    environment:
      - POSTGRESQL_POSTGRES_PASSWORD=adminpassword
      - POSTGRESQL_USERNAME=customuser
      - POSTGRESQL_PASSWORD=custompassword
      - POSTGRESQL_DATABASE=customdatabase
      - REPMGR_PASSWORD=repmgrpassword
      - REPMGR_PRIMARY_HOST=< first database servers public ip> 
      - REPMGR_PARTNER_NODES=< first database servers public ip >,pg-1
      - REPMGR_NODE_NAME=pg-1
      - REPMGR_NODE_NETWORK_NAME=pg-1
    networks:
      - main

note that my pgpool container which is on the third server can connect to server one and write in database but when it comes to replication they are not working. thanks for your attention!

Answer 1

I have tried to do it via an overlapping docker network. when i depoly them on a single host the replica works.

when i deploy them on different hosts the network cant be established. and the logs show:

postgresql-repmgr 03:18:22.32 INFO  ==> Waiting for primary node...

I have tested the basic network configuration with the docker alpine container and i can ping the different container on different host.

Also a config to run the image as a docker swarm is not possible.

Answer 2

I found a solution.

1. Create an overlay network where the two hosts are located.
2. Run following command: sudo ethtool -K <network interface> tx off
3. Create the Container with their name, and access them with their name.

More details about this issue (general) can be found under: https://serverfault.com/questions/1073337/docker-swarm-containers-in-one-overlay-network-but-on-different-nodes-cant-rea