Run tests only for staged files: `git stash -k -u` and `git stash pop` will raise conflicts on partial staged files

Question

tl;dr

I want to run tests only with staged files before commit:

1. git stash save -k -u to stash unstaged/untracked files/changes before testing
2. run tests with staged files
3. git stash pop to restore changes/files at step 1.

The problem is using git stash pop will raise conflicts on the files with partial staged. Resolving the conflicts will lead to lose the partial staged/unstaged changes (you need to pick partial lines to staged again).

Update: If you want to know the shell script to run this procedure, please jump to the last section for more information.

NOTE: only adjacent lines (or close enough) with partial staged will cause this issue. For example, if you have 6 new line changes in a file:

1 +| a     (add to staged)
2 +| b     (add to staged)
3 +| c     (add to staged)
4  | d     (keep unstaged)
5  | e     (keep unstaged)
6  | f     (keep unstaged)

Now use git stash -k -u and then git stash pop will raise conflict.

Demonstrate the question

Git provides three phases for changes before commit: staged, unstaged and untracked.

Any changes will be added to unstaged. Before commit, you can pick some of lines or files and add them to staged by git add.

Now, after adding some of code to staged, I want to run tests with only staged files to make sure they were suitable for commit, so I need to stash unstaged and untracked changes (new files) by git stash -k -u and keep staged changes.

Say, for example, I have 3 file changes: file A is fully staged, file B is partial staged (some of code), and file C is a new file which is untracked.

[staged]
  file A
  file B (only stage some of code)
[unstaged]
  file B
[untracked]
  file C (new file)

After running git stash -k -u, all unstaged/untracked changes are stashed.

[staged]
  file A
  file B (only stage some of code)
[unstaged/untracked]
  <none, clean>

Here comes the problem. After running tests and then git stash pop, it raises conflicts on file B because it is partial staged. I'm sure that I did not change file B when stashing and testing.

I wonder how to auto-merge with git stash pop without any conflict just like before I stashed them.

My workflow

I think this is a very usual workflow

  development start
          |
[make changes (unstaged)] 
          |
(pick changes to staged for commit by `git add`)<---|
          |                                         |
          V                     (pick other changes to fulfill tests)
[partial staged/unstaged]                           |
          |                                         |
(stash unstaged changes by `git stash -k -u`)       |
          |                                         |
(run tests only with staged files for commit)       |
          |                                         | 
(restore stashed files by `git stash pop`)          |
          |                                         |
          |------------<if test failed>-------------| 
          |
    <if test success>
          |
[commit staged files by `git commit`]
          |
          V
keep development or next commit

I need a way to stash pop without losing staged/unstaged state for all changes. Keeping staged files is very important to commit or fulfill test by adding other changes.

Update with solution: a shell script to run the procedure

According to @torek's answer, I write a shell script to run tests with only staged files:

#!/bin/sh -e

# stash all unstaged changes
# (-k: unstaged files; -u: new added files; -q: quite)
echo '--------------------------------------------------------------'
echo '---- Stash all unstaged/untracked files (git stash -k -u) ----'
echo '--------------------------------------------------------------'
BEFORE_STASH_HASH=$(git rev-parse refs/stash)
git stash -k -u -q
AFTER_STASH_HASH=$(git rev-parse refs/stash)
if [ "$BEFORE_STASH_HASH" == "$AFTER_STASH_HASH" ]; then
  echo '\n\n---- Stash failed! Please check and retry. ----\n\n';
  exit 1;
fi;

# run test only with staged files
echo '-------------------'
echo '---- Run tests ----'
echo '-------------------'
<run your tests here> ||      #### <=== replace your test command here
(echo '\n\n---- Tests failed! Please fix it before commit. ----\n\n')

# restore all stashed changes
# http://stackoverflow.com/questions/41304610/
echo '-----------------------------------------------------------'
echo '---- Restore all stashed files (git stash pop --index) ----'
echo '-----------------------------------------------------------'
git reset --hard -q &&
git clean -df -q &&
git stash pop --index -q ||
(echo '\n\n---- Restore failed! Please check and fix it. ----\n\n')

Answer 1

This is not a full answer—see How to recover from "git stash save --all"? for more—but while this is an appealing process, and it will work once the bug in git stash gets fixed, it is a bit dangerous today.

If you have fixed the bug, or don't mind living slightly dangerously, :-) you can use this process:

1. Run git stash save -k -u and make sure it saves something (e.g., compare the results from git rev-parse refs/stash before and after).
2. Run your tests.
3. git reset --hard && git clean -df (optionally, including -q for both). The git reset --hard is needed only if the tests modify committed files, and the git clean is needed only if the tests create untracked files.
4. Run git stash pop --index. Note that the --index is critical here. You may wish to use -q as well.

Instead of save and pop --index, you might want to use create and apply --index and store your stash-bag under a different reference (that you manipulate and/or delete when done, in whatever way you like). Of course, if you're going to go this far, you might want to write your own modified git stash script that avoids the current one's bug in the first place.

---

There's a completely different, and to my mind simpler, approach to running tests:

1. Create an empty temporary directory.
2. Turn the current index into a tree, then read that tree into the temporary directory. (Or use git checkout-index to extract the index into the temporary directory. In either case, note the environment variables GIT_WORK_TREE and GIT_DIR, or the --git-dir and --work-tree arguments to the front end git command.)
3. Run the tests in the temporary directory.
4. Discard the temporary directory.

This avoids the git stash save bug and, with a slight modification to step 2, lets you test any revision. There are two obvious disadvantages: you need a place to store a temporary tree, and the temporary tree is not where the work-tree is. How much of a problem those are depends on your repository and your tests.

Answer 2

Currently, here is a workaround provided by @Paul. I write this here, but I'm still looking for a better solution.

These are the steps:

1. Use git stash -k -u to stash unstaged/untracked files but keep staged files for commit.
2. Run tests with staged files. If failed, go to step 3; if passed, go to step 4.
3. If failed, you need to restore unstaged/untracked files back and add new code to staged state. Here is the workaround for my problem:
   • Run git commit -m 'temp' to store staged files in a new commit temporally.
   • Run git stash pop to restore unstaged/untracked files. It still will raise conflict for adjacent line changes between commit (previous staged) and unstaged.
   • Run git checkout --theirs -- . && git reset to resolve the conflicts, if any, and reset unmerged state to unstaged state. Because previous staged files are now committed, git reset will not influence them.
   • Run git reset HEAD^ --soft to rollback files in the latest commit to staged state.
   • Now you have the same file state (staged/unstaged/untracked) as before step 1. You can add other code from unstaged to staged to fulfill the requirement for tests. Maybe there is one git command to achieve those steps above which I'm looking for.
4. If passed, just git commit with the message and start the next iteration of development.

Another approach is: (1) just commit staged files first, (2) run test on the latest commit, and (3) patch the commit, if needed, by git commit --amend until passing the tests. I would not prefer this approach, because any interruption to your work might make you forget to resume patching the commit for passing tests, and push or publish the wrong commit.

Answer 3

This flow works perfectly for me, including with partially staged files. I don't modify files in step 2.

# 1. Stash all changes, then discard all unstaged changes
$ git stash --include-untracked --keep-index

# 2. Run tests, run linter, check types, etc. Don't modify files. Commit if desired.

# 3a. If no commit was made, restore to initial state
$ git stash pop

# 3b. If a commit was made, restore unstaged changes
$ git checkout stash -- . ; git stash pop ; git reset

Useful aliases in ~/.gitconfig:

[alias]
  stash-unstaged = stash --keep-index --include-untracked

  restore-unstaged = "!git checkout stash -- . ; git stash pop ; git reset"

  # Shortcuts
  stun = stash-unstaged
  restun = restore-unstaged

(Note that the alias stash-unstaged isn't quite accurate, because the stash also includes the index. But it's a convenient shorthand.)

With aliases, the commands are quite simple:

1. git stash-unstaged

2. Run tests, linting, etc.

3. git stash pop or git restore-unstaged, depending on whether a commit was made