Error when using TLS server with pgBackRest : [113] No route to host

Question

I´m trying to implement the TLS server feature available with pgBackRest to use a secure connection between the DB server and the repo server, replacing the previous SSH passwordless setup (that was working fine).

After following the online documentation, I´m having the following error when issuing the stanza-create command :

pgbackrest@pgb-repo$ pgbackrest --stanza=training --log-level-console=info stanza-create
2022-06-13 12:56:55.677 P00   INFO: stanza-create command begin 2.39: --buffer-size=16MB --exec-id=8994-62e5ecac --log-level-console=info --log-level-file=info --pg1-host=pg1-primary --pg1-host-ca-file=/etc/pgbackrest/cert/ca.crt --pg1-host-cert-file=/etc/pgbackrest/cert/pg1-primary.crt --pg1-host-key-file=/etc/pgbackrest/cert/pg1-primary.key --pg1-host-type=tls --pg1-host-user=postgres --pg1-path=/data/postgres/13/pg_data --repo1-path=/backup/pgbackrest --stanza=training
WARN: unable to check pg1: [HostConnectError] unable to connect to 'pg1-primary:8432': [113] No route to host
ERROR: [056]: unable to find primary cluster - cannot proceed
       HINT: are all available clusters in recovery?
2022-06-13 12:58:55.835 P00   INFO: stanza-create command end: aborted with exception [056]

The PostgreSQL server is up and running on the the DB host:

[postgres@pg1-primary ~]$ psql -c "SELECT pg_is_in_recovery();"
 pg_is_in_recovery
-------------------
 f
(1 row)

Question

Why am I having this [113] No route to host error ?

Configuration for each server :

pg1-primary

[postgres@pg1-primary ~]$ cat /etc/pgbackrest/pgbackrest.conf
[global]
repo1-path=/backup/pgbackrest
repo1-host-ca-file=/etc/pgbackrest/cert/ca.crt
repo1-host-cert-file=/etc/pgbackrest/cert/pgb-repo.crt
repo1-host-key-file=/etc/pgbackrest/cert/pgb-repo.key
repo1-host-type=tls
tls-server-address=*
tls-server-auth=pgb-repo=training
tls-server-ca-file=/etc/pgbackrest/cert/ca.crt
tls-server-cert-file=/etc/pgbackrest/cert/pg1-primary.crt
tls-server-key-file=/etc/pgbackrest/cert/pg1-primary.key

[postgres@pg1-primary ~]$ cat /etc/pgbackrest/conf.d/training.conf
[training]
pg1-path=/data/postgres/13/pg_data
pg1-socket-path=/tmp
repo1-host=pgb-repo
repo1-host-user=pgbackrest

[postgres@pg1-primary ~]$ ll /etc/pgbackrest/cert/
total 20
-rw-------. 1 postgres postgres 1090 Jun 13 12:12 ca.crt
-rw-------. 1 postgres postgres  977 Jun 13 12:12 pg1-primary.crt
-rw-------. 1 postgres postgres 1708 Jun 13 12:12 pg1-primary.key
-rw-------. 1 postgres postgres  977 Jun 13 12:23 pgb-repo.crt
-rw-------. 1 postgres postgres 1704 Jun 13 12:23 pgb-repo.key

pgb-repo

pgbackrest@pgb-repo$ cat /etc/pgbackrest/pgbackrest.conf
[global]
repo1-path=/backup/pgbackrest
tls-server-address=*
tls-server-auth=pg1-primary=training
tls-server-ca-file=/etc/pgbackrest/cert/ca.crt
tls-server-cert-file=/etc/pgbackrest/cert/pgb-repo.crt
tls-server-key-file=/etc/pgbackrest/cert/pgb-repo.key

pgbackrest@pgb-repo$ cat /etc/pgbackrest/conf.d/training.conf
[training]
pg1-host=pg1-primary
pg1-host-user=postgres
pg1-path=/data/postgres/13/pg_data
pg1-host-ca-file=/etc/pgbackrest/cert/ca.crt
pg1-host-cert-file=/etc/pgbackrest/cert/pg1-primary.crt
pg1-host-key-file=/etc/pgbackrest/cert/pg1-primary.key
pg1-host-type=tls

pgbackrest@pgb-repo$ ll /etc/pgbackrest/cert/
total 20
-rw-------. 1 pgbackrest pgbackrest 1090 Jun 13 12:27 ca.crt
-rw-------. 1 pgbackrest pgbackrest  977 Jun 13 12:27 pg1-primary.crt
-rw-------. 1 pgbackrest pgbackrest 1708 Jun 13 12:27 pg1-primary.key
-rw-------. 1 pgbackrest pgbackrest  977 Jun 13 12:27 pgb-repo.crt
-rw-------. 1 pgbackrest pgbackrest 1704 Jun 13 12:27 pgb-repo.key

The servers are reachable from one another:

[postgres@pg1-primary ~]$ ping pgb-repo
PING pgb-repo.xxxx.com (XXX.XX.XXX.117) 56(84) bytes of data.
64 bytes from pgb-repo.xxxx.com (XXX.XX.XXX.117): icmp_seq=1 ttl=64 time=0.365 ms
64 bytes from pgb-repo.xxxx.com (XXX.XX.XXX.117): icmp_seq=2 ttl=64 time=0.421 ms

pgbackrest@pgb-repo$ ping pg1-primary
PING pg1-primary.xxxx.com (XXX.XX.XXX.116) 56(84) bytes of data.
64 bytes from pg1-primary.xxxx.com (XXX.XX.XXX.116): icmp_seq=1 ttl=64 time=0.325 ms
64 bytes from pg1-primary.xxxx.com (XXX.XX.XXX.116): icmp_seq=2 ttl=64 time=0.298 ms

Answer 1

So actually the issue had to do with the firewall preventing access to the default TLS port (8432) used by pgBackRest.

[root@pgb-server ~]# firewall-cmd --zone=public --add-port=8432/tcp --permanent
[root@pgb-server ~]# firewall-cmd --reload

Once the port was accessible through the firewall I could issue a telnet command successfully (for testing access) - and of course run my pgBackRest commands too.

[pgbackrest@pgb-server]$ telnet pg1-server 8432
Trying 172.XX.XXX.XXX...
Connected to pg1-server.
Escape character is '^]'.