Extents reported in USN_RECORD_V4 struct are a lot bigger than actual changes

Question

We use USN change journal to track changes on SQL data files for performing incremental backups.

For big files (of size 5 TB), we have seen USN_RECORD_V4 have extents reporting that a big portion of file is modified but actual change on the file is very less.

I want to understand how and when USN_RECORD_EXTENT are added to a USN_RECORD_V4.

In what scenarios is it possible that data is not overwritten but there is an extent added in USN_RECORD_V4.

On one day, I observed, 943 records for a file of size 5 TB. Of these 943 records, there are three consecutive records for this file which had extents which corresponds to changed data of around 330 GB, 220 GB and 4300 GB. All other 940 records combined reported 70 GB data is changed.

But actual change of that file is only of 600 GB (We did a diff between file backups of the two versions of file at 16KB granularity). So, the three big records corresponds to some actual changes in the file but are reporting some very big extents which are not modified.

Due to this, our backup software is doing way more work than what it should.

Answer 1

how and when USN_RECORD_EXTENT are added to a USN_RECORD_V4.

Reason flags that identify reasons for changes that have accumulated in this file or directory journal record since the file or directory opened.

I suggest you could check the Reason of records.

In what scenarios is it possible that data is not overwritten but there is an extent added in USN_RECORD_V4.

According to the Doc: Change Journal Records

multiple changes to the same file may result in only one reason flag being added to the current record. If the same kind of change occurs more than once, the NTFS file system does not write a new record for the changes after the first.