How to fetch enough commits to do a merge in a shallow clone

Question

What I'm trying to do: test pull requests from github. I want to locally merge a pull request into master and run some tests on the result. Since the repository is huge, I do a shallow clone.

To be able to do the merge, I fetch more and more commits (git fetch with increasing --depth) until I have the merge-commit between master the pull request.

However, it doesn't work every time. It looks like I do not only need the merge-base, but also every commit in the master..merge_base range. I'm not sure however how to do that.

So, the question is: how do I fetch enough history to do the merge?

Answer 1

What you need (I think), in a catch-22 way, is 'git describe --all --first-parent' to tell you the depth of the given commit from the appropriate branch. However I'm not sure how to get that from Github before you do your shallow fetch ;-)

Answer 2

I recently hit this issue and found a way that saved a ton of time, get the merge-base commit from the API rather than get enough repo information downloaded to calculate it https://docs.github.com/en/rest/commits/commits#compare-two-commits

This api call amounts to

gh api \
  repos/my_github_user/my_repo/compare/main...my_pr_branch \
  | jq -r '.merge_base_commit.sha'"

Or curl if you prefer

curl -s -H "Authorization: token $GITHUB_TOKEN" \
  https://api.github.com/repos/my_user/my_repo/compare/main...my_pr_branch

here's more of what my github workflow / action looks like

      - name: Fetch merge base SHA from API
        run: |
          my_merge_base_cmd="gh api repos/github/github/compare/${{ github.event.pull_request.base.sha }}...${{ github.event.pull_request.head.sha }} | jq -r '.merge_base_commit.sha'"
          echo $my_merge_base_cmd
          my_merge_base=$(eval $my_merge_base_cmd)
          echo "MY_MERGE_BASE_SHA=$my_merge_base" >> $GITHUB_ENV

      - name: Fetch merge base SHA
        run: |
          echo $MY_MERGE_BASE_SHA
          git fetch \
            --no-tags \
            --prune \
            --progress \
            --no-recurse-submodules \
            --depth=1 \
            origin $MY_MERGE_BASE_SHA

      - name: Checkout merge base SHA
        run: |
          echo $MY_MERGE_BASE_SHA
          git checkout \
            --force \
            $MY_MERGE_BASE_SHA

Basically once I have the merge base SHA checked out I can build artifacts and compare them artifacts I built on the PR branch

If you need the commits between the merge-base and the PR, you'd have to shallow fetch them too, but at least using the merge base you know your ending point

Note, best to use the sha, not the ref for the API call to fetch merge base, as the ref can change what it points to (someone merging main into their PR branch for example), resulting in a race condition.

Answer 3

If you have the history from when feature branched from master but don't want the full history of master then you can estimate a branching date and use;

git fetch --shallow-since=<date> origin master

It's hard to use any other form of git fetch to do what you want (query the remote for the merge-base) because git fetch fetches refs. There might not be a ref that you're looking for.

You can automate the digging using the following script.

while [ -z $( git merge-base master feature ) ]; do     
    git fetch -q --deepen=1 origin master feature;
done