RocksDB. Backups take up more space than the database itself

Question

I encountered a problem in which the backup takes up more space than the database itself.

233G    rocks database size
651G    rocks backup size

Both the database engine and the backup engine are configured by default.

15M     rocks_backup/meta
2.5G    rocks_backup/private
649G    rocks_backup/shared_checksum

Please tell me if this is normal, and if it is not then please help me figure it out.

Answer 1

If you have multiple backups with major compactions in between (or obviously with different logical data) then your backup directory will be much larger, as each differently-compacted version of the data is stored there. Use PurgeOldBackups() to minimize the number of backups that are kept.

If you had bad timing in taking a backup, such as just before a major compaction that purged a lot of obsolete data, or before flushing a large memtable with a large WAL file (typically uncompressed), it is possible that a single backup could be larger than the DB itself (a later version of the DB itself). But it should not be possible for the backup of a single DB to be larger than the DB size (+ WAL files if not flushing for backup) at the start of creating the backup. (Excluding some minor overheads like the backup meta file.)

Answer 2

Shared your question in the Speedb Hive. Here's what we have for you:

In order to assist you we will need more information on the content of the shared_checksum directory ...

You can find the Speedb hive here and (once you've registered) the link to the thread with your question here, if you have more questions or need additional info