thank you in advance for any input.
We have a master server with 130 databases in a RDS Postgresql server version 14.8.
We have enabled 260 replication tasks, which have set 2 replication slots per database and 60 tables per replication slot, table sizes varies from 100 Mb to 100 Gb. But the average table size is around 100 Mb.
We generate approximately 100 MB daily on the target tables per database. The database is pretty much busy all the time with DBLoad ranging from 40% - 80%.
Our master database server parameters are as following:
max_worker_processes = 32
max_logical_replication_workers = 4 max_parallel_maintenance_workers = 2
autovacuum_max_workers = 3 max_parallel_workers_per_gather = 4
logical_decoding_work_mem = 64Mb
The instance has 16 VCpus 128Gb Mem:
We gather the information from pg_output using the AWS Services Database Migration Service (DMS) with the default postgres plugin "test_decoding.
I couldn't understand how the max_logical_replication_workers play a role on replication the data. In our case would max_logical_replication_workers be a constraint that restrict "4" workers replication per slot?
Besides, we noticed that the logical_decoding_work_mem is not being respected and there are wal_senders process in the databases using over 600 Mbs.
We have enabled the replication and saw the memory consumption and the DBLoad going through the roof, and when reducing the logical_decoding_work_mem we didn't see any improvement, so I am kind of lost.
Appreciate any guidance.
Cheers,