Unusally large incremental backups with pbbackrest

Question

I'm doing daily full backups of postgres database (v12.2) and hourly incremental backups with pgbackrest (v2.24) and crontab.

After an out-of-disk hickup (and after an event where a lot of activity was going on in the DB) I started investigating the cause of out of disk space and noticed that incremental backup sizes (1.5GB ~ 1.8GB) are not that different from full backups (2.2GB).

Commands that produce both type of backups:

sudo -Hu postgres pgbackrest --stanza=****** --type=full --log-level-console=info backup > /dev/null 2>&1
sudo -Hu postgres pgbackrest --stanza=****** --type=incr --log-level-console=info backup > /dev/null 2>&1

Backup folder looks like this:

1.5G    ./20210509-002301F_20210509-132302I
1.5G    ./20210509-002301F_20210509-012301I
2.2G    ./20210509-002301F
1.5G    ./20210510-002302F_20210510-012301I
1.8G    ./20210509-002301F_20210509-192301I
1.5G    ./20210510-002302F_20210510-032302I
1.5G    ./20210509-002301F_20210509-032302I
1.8G    ./20210510-002302F_20210510-072302I
1.5G    ./20210509-002301F_20210509-202301I
1.5G    ./20210509-002301F_20210509-092301I
1.8G    ./20210509-002301F_20210509-072302I
933M    ./backup.history
1.5G    ./20210509-002301F_20210509-142301I
1.1G    ./20210509-002301F_20210509-022302I
1.5G    ./20210510-002302F_20210510-082301I
1.8G    ./20210510-002302F_20210510-062302I
1.5G    ./20210509-002301F_20210509-212301I
1.5G    ./20210509-002301F_20210509-082301I
1.5G    ./20210510-002302F_20210510-042301I
1.5G    ./20210509-002301F_20210509-162302I
1.8G    ./20210509-002301F_20210509-182301I
1.8G    ./20210509-002301F_20210509-122301I
1.5G    ./20210509-002301F_20210509-232301I
1.8G    ./20210510-002302F_20210510-122301I
1.5G    ./20210509-002301F_20210509-042302I
1.8G    ./20210510-002302F_20210510-102302I
1.5G    ./20210509-002301F_20210509-062301I
1.8G    ./20210509-002301F_20210509-102302I
1.5G    ./20210510-002302F_20210510-022301I
1.8G    ./20210510-002302F_20210510-112302I
1.8G    ./20210510-002302F_20210510-092302I
1.5G    ./20210509-002301F_20210509-112302I
2.2G    ./20210510-002302F
1.8G    ./20210509-002301F_20210509-152302I
1.8G    ./20210509-002301F_20210509-172301I
1.5G    ./20210510-002302F_20210510-052302I
1.8G    ./20210509-002301F_20210509-222301I
1.5G    ./20210509-002301F_20210509-052302I
61G     .

According to the docs, incremental backups should be only a fraction on the full backup size and should, obviously, depend on full backups. That does not appear to be the case here. Is there something else I'm missing here?

Answer 1

I found a possible answer at pgBackRest github issues page:

"Incremental and differential backups only operate at the segment level. Segments in Postgres can be up to 1GB so even a small change can produce a 1GB delta."

From: https://github.com/pgbackrest/pgbackrest/issues/540#issuecomment-393304831