Inspect git repo after using BFG repo-cleaner

Question

Very basic git question:

I uploaded some compromising information to Github and am using bfg to clean the repo. I followed the documentation and performed the following actions:

$ git clone --mirror git://example.com/some-big-repo.git
$ bfg --replace-text passwords.txt  my-repo.git

I received the following output:

Found 233 objects to protect
Found 9 commit-pointing refs : HEAD, refs/heads/experimental, refs/heads/master, ...

Protected commits
-----------------

These are your protected commits, and so their contents will NOT be altered:

 * commit 497fc1c8 (protected by 'HEAD')

Cleaning
--------

Found 80 commits
Cleaning commits:       100% (80/80)
Cleaning commits completed in 301 ms.

BFG aborting: No refs to update - no dirty commits found??

I'd like to see if the private information was cleared from my repo but I'm not sure how to check the files in the mirrored repo. Any ideas?

Answer 1

A quick way to check if a password is still in your history might be to use the 'git pickaxe', aka the -S option. Here's an example that checks for the string password1:

git log -Spassword1

However, from the output shown in your question, it looks like The BFG couldn't find any of the entries from passwords.txt in your repo (prompting the messsage 'no dirty commits found??' which you see at the end of the output), which is a bit strange if you're sure they're in there. Was this the first time you'd run the BFG on the repo? Perhaps it was the second time, and The BFG had already removed the passwords?

The passwords.txt file you give to The BFG should have one password per line, ie:

changeme
password1
password2

The BFG only looks at text files under 1MB by default. Are your passwords in some file that might appear to be binary, or bigger than 1MB?

Update: For seeing what's changed in a repo-clean-up, you could also try Eric S. Raymond's repodiffer (part of his reposurgeon project): http://www.catb.org/~esr/reposurgeon/repodiffer.html - you use it like this:

$ repodiffer old-repo-copy.git new-repo-copy.git

The script may take a while to run, but it will tell you precisely what has changed between those two repos.

Full disclosure: I'm the author of the BFG Repo-Cleaner.

Answer 2

Had the same issue for 2 hours on OSX. In my case the issues was the password.txt file (maybe corrupt or something). Make sure you create the .txt via terminal by using vi/nano. Don't create it with tools like TextEdit and change extension.

Answer 3

To double-check if specific sensitive data is not present in your repo anymore, you may use the following command:

git log --patch | grep --color=auto secret

or use less +/secret instead of grep to see it in context.

Answer 4

I wanted to leave an addition here, because I spent an hour trying to work out why I was getting the same error message:

BFG aborting: No refs to update

And I absolutely couldn't work out why.

And then I realised that BFG, like git, is case sensitive - and the file name I was typing in was all lower case, while the version in git had an upper case initial letter.

As I was on Windows, I hadn't thought about this - the file system there is not case sensitive.

Hopefully this will save someone else a couple of hours of pain!