Cluster consists of: 1 haproxy, 3 postgres. EOFException appears for all client at about 208 of 800 connections (select count(*) used from pg_stat_activity;). What other limits could be causing the problem?
VM Haproxy
ss -tunap | grep 5000 |wc -l
# Output from 170 to 230 (of 4096 limit)
ss -tunap |wc -l
# Output from 340 to 510 (of 800 limit)
cat /etc/haproxy/haproxy.cfg
global
maxconn 4096
log 127.0.0.1 local2
defaults
log global
mode tcp
retries 2
timeout client 30m
timeout connect 8s
timeout server 30m
timeout check 5s
listen stats
mode http
bind *:7000
stats enable
stats uri /
listen postgres
bind *:5000
option httpchk
http-check expect status 200
default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions
server node1 192.168.19.8:5432 maxconn 200 check port 8008
server node2 192.168.19.9:5432 maxconn 200 check port 8008
server node3 192.168.19.10:5432 maxconn 200 check port 8008
Postgres
node-1 (replica)
free -hm
total used free shared buff/cache available
Mem: 3.8Gi 388Mi 1.6Gi 59Mi 1.9Gi 3.1Gi
Swap: 0B 0B 0B
node-2 (leader)
free -hm
total used free shared buff/cache available
Mem: 3.8Gi 587Mi 1.4Gi 102Mi 1.9Gi 2.9Gi
Swap: 0B 0B 0B
select count(*) used from pg_stat_activity;
# Output to 208, not more
show shared_buffers;
# Output 512MB
node-3
free -hm
total used free shared buff/cache available
Mem: 3.8Gi 334Mi 1.4Gi 60Mi 2.1Gi 3.2Gi
Swap: 0B 0B 0B
patronictl -c /etc/patroni.yml edit-config postgres
loop_wait: 10
maximum_lag_on_failover: 1048576
postgresql:
parameters:
max_connections: 800
shared_buffers: 512MB
pg_hba:
- host replication replicator 127.0.0.1/32 md5
- host replication replicator 192.168.19.8/24 md5
- host replication replicator 192.168.19.9/24 md5
- host replication replicator 192.168.19.10/24 md5
- host all all 0.0.0.0/0 md5
use_pg_rewind: true
use_slots: true
retry_timeout: 10
ttl: 30
EOFException at 800 of 800 connections limit