git fails to stage files, show all files as duplicate but the char casing is not an issue

Question

In my case, I did simple one liner change to one of my file and wanted to commit my changes but noticed that commit -am "" did not add/commited the file.

After issuing git ls-files --stage, I see probably all files in my project showing as duplicates. Here is example of one of the files

100644 6314bd3f89d1b794c6d8c0fb9bb4aa492e2d510a 0   SquirrelFoH/Squirrel.FoH.ViewModels/UserLoginViewModel.cs
100644 6314bd3f89d1b794c6d8c0fb9bb4aa492e2d510a 0   SquirrelFoH/Squirrel.FoH.ViewModels/UserLoginViewModel.cs

Interestingly, some of the files showing ad duplicates are not modified by me at all, some are but nevertheless, they show as duplicate and as you can see below, the casing is not the issue like in other SO posts on here.

UPDATE

While this does not resolve my issue described above, it helped me to use git reset --hard HEAD~1 which resets HEAD pointer to 2nd last commit which is the commit that worked. I use --hard above to discard last commit since it caused the issue above for me. If you need to keep these changes, using --soft instead will reset HEAD to your commit before last commit and add changes in last commit to staging area.

git reset --hard HEAD~1
git reset --hard HEAD~2
git reset --hard HEAD~3
...

Above commands reset HEAD pointer 1, 2, 3, ... commits before last commit and discard any changes after. Use --soft instead of --hard if you dont want to discard these changes in which case these changes will be staged for you.

This is the situation I had. Below, last commit is commit A which is the one with the duplicates that started showing up after last pull of remote changes into my local branch. Commits B, C, ... are commits before commit A:

commit A
commit B - git reseat --hard HEAD~1
commit C 

, now my last commit is commit B which does not have the duplicates. I can now try to merge again and see if I will have the same issue as I had with commit A. As I mentioned this does not solve the issue but at least allows me to try to recreate it or continue my work and deal with the merge later.

Answer 1

You will have to check if the issue persists in Git 2.22.1 (Q3 2019)/ Git 2.25 (Q1 2020), as The data collected by fsmonitor was not properly written back to the on-disk index file (on Mac, Linux or Windows)

See commit b5a8169, commit d4c0a3a (24 May 2019) by Johannes Schindelin (dscho).
^{(Merged by Junio C Hamano -- gitster -- in commit 10432cc, 25 Jul 2019)}

mark_fsmonitor_valid(): mark the index as changed if needed

Without this bug fix, t7519's four "status doesn't detect unreported modifications" test cases would fail occasionally (and, oddly enough, a lot more frequently on Windows).

The reason is that these test cases intentionally use the side effect of git status to re-write the index if any updates were detected: they first clean the worktree, run git status to update the index as well as show the output to the casual reader, then make the worktree dirty again and expect no changes to reported if running with a mocked fsmonitor hook.

The problem with this strategy was that the index was written during said git status on the clean worktree for the wrong reason: not because the index was marked as changed (it wasn't), but because the recorded mtimes were racy with the index' own mtime.

As the mtime granularity on Windows is 100 nanoseconds (see e.g. https://learn.microsoft.com/en-us/windows/desktop/SysInfo/file-times), the mtimes of the files are often enough not racy with the index', so that git status call currently does not always update the index (including the fsmonitor extension), causing the test case to fail.

The obvious fix: if we change any index entry's CE_FSMONITOR_VALID flag, we should also mark the index as changed.
That will cause the index to be written upon git status, including an updated fsmonitor extension.

Side note: Even though the reader might think that the t7519 issue should be much more prevalent on Linux, given that the ext4 filesystem (that seems to be used by every Linux distribution) stores mtimes in nanosecond precision. However, ext4 uses current_kernel_time() (see https://unix.stackexchange.com/questions/11599#comment762968_11599; it is amazingly hard to find any proper source of information about such ext4 questions) whose accuracy seems to depend on many factors but is safely worse than the 100-nanosecond granularity of NTFS (again, it is horribly hard to find anything remotely authoritative about this question).
So it seems that the racy index condition that hid the bug fixed by this patch simply is a lot more likely on Linux than on Windows. But not impossible ;-)

---

With Git 2.25 (Q1 2020), fsmonitor is more robust, and removes an incorrect BUG() that should not trigger.

See commit 61eea52 (13 Nov 2019) by Junio C Hamano (gitster).
^{(Merged by Junio C Hamano -- gitster -- in commit aec3b2e, 01 Dec 2019)}

fsmonitor: do not compare bitmap size with size of split index

^{Reported-by: Utsav Shah
Helped-by: Kevin Willford
Helped-by: William Baker}

3444ec2e ("fsmonitor: don't fill bitmap with entries to be removed", 2019-10-11, Git v2.24.0-rc1 -- merge listed in batch #11) added a handful of sanity checks that make sure that a bit position in fsmonitor bitmap does not go beyond the end of the index.

As each bit in the bitmap corresponds to a path in the index, this is the right check most of the time.

Except for the case when we are in the split-index mode and looking at a delta index that is to be overlayed on the base index but before the base index has actually been merged in, namely in read_ and write_fsmonitor_extension().

In these codepaths, the entries in the split/delta index is typically a small subset of the entire set of paths (otherwise why would we be using split-index?), so the bitmap used by the fsmonitor is almost always larger than the number of entries in the partial index, and the incorrect comparison would trigger the BUG().

---

The index files can become corrupt under certain conditions when the split-index feature is in use, especially together with fsmonitor, which have been corrected with Git 2.41 (Q2 2023).

See commit 061dd72, commit be6b65b, commit 3b7a447, commit 3704fed (26 Mar 2023) by Johannes Schindelin (dscho).
^{(Merged by Junio C Hamano -- gitster -- in commit f315a8b, 04 Apr 2023)}

fsmonitor: avoid overriding cache_changed bits

^{Noticed-by: Jeff Hostetler
Signed-off-by: Johannes Schindelin}

As of e636a7b ("read-cache: be specific what part of the index has changed", 2014-06-13, Git v2.1.0-rc0 -- merge listed in batch #8), the paradigm cache_changed = 1 fell out of fashion and it became a bit field instead.

This is important because some bits have specific meaning and should not be unset without care, e.g. SPLIT_INDEX_ORDERED.

However, b5a8169 (mark_fsmonitor_valid(): mark the index as changed if needed, 2019-05-24, Git v2.23.0-rc0 -- merge listed in batch #2) (mark_fsmonitor_valid(): mark the index as changed if needed, 2019-05-24) did use the cache_changed attribute as if it were a Boolean instead of a bit field.

That not only would override the SPLIT_INDEX_ORDERED bit when marking index entries as valid via the FSMonitor, but worse: it would set the SOMETHING_OTHER bit (whose value is 1).
This means that Git would unnecessarily force a full index to be written out when a split index was asked for.

Let's instead use the bit that is specifically intended to indicate FSMonitor-triggered changes, allowing the split-index feature to work as designed.