Combining test coverage with hpc

Question

I find hpc really confusing, even after reading several explanations and playing around quite a bit.

I have a library HML, and two test suites fileio-test and types-test, using HTF (I plan on moving to tasty). I'd like to run the two tests and then see the combined coverage from the two over the library.

At the moment I build the library using

cabal configure --enable-coverage
cabal build

And run the tests using

cabal configure --enable-coverage --enable-tests
cabal build
cabal test

hpc report --hpc-dir dist/hpc/vanilla/mix/fileio-test dist/hpc/vanilla/tix/fileio-test/fileio-test.tix

This shows me some coverage, but not the correct one. What I think is that it shows coverage, but only from one of the tests, and also includes coverage of the tests themselves.

I tried using

--hpc-dir dist/hpc/vanilla/mix/HML-0.1.0.0

But then hpc complains it can't find the module files it needs. I also tried combining coverage from the two tests with no luck.

Any pointers?

Answer 1

I am also attempting to call HPC directly and have a similar error.

Cabal 3.6 should be able to generate your HPC report for you without needing to call HPC like you describe. It can bypass that error. There is one caveat: https://github.com/haskell/cabal/issues/6440#issuecomment-1133542171

Add to cabal.project:

package *
  coverage: True
  library-coverage: True

then cabal test. The report should be somewhere in dist-newstyle.

---

Reading the verbose log from cabal test with the options above has shown the correct arguments to HPC. It solves the module error. Here is an example for Mustache: https://github.com/JustusAdam/mustache

Add to cabal.project:

package *
  coverage: True
  library-coverage: True

then cabal test -v all > foo.log.

In foo.log there should be a call to HPC such as:

~/.ghcup/ghc/8.10.7/bin/hpc markup \
dist-newstyle/build/x86_64-linux/ghc-8.10.7/mustache-2.4.0/t/unit-tests/hpc/vanilla/tix/unit-tests/unit-tests.tix \
'--destdir=dist-newstyle/build/x86_64-linux/ghc-8.10.7/mustache-2.4.0/t/unit-tests/hpc/vanilla/html/unit-tests' \
'--hpcdir=dist-newstyle/build/x86_64-linux/ghc-8.10.7/mustache-2.4.0/hpc/vanilla/mix/unit-tests' \
'--hpcdir=dist-newstyle/build/x86_64-linux/ghc-8.10.7/mustache-2.4.0/hpc/vanilla/mix/mustache-2.4.0' \
'--include=Text.Mustache' \
'--include=Text.Mustache.Types' \
'--include=Text.Mustache.Parser' \
'--include=Text.Mustache.Compile' \
'--include=Text.Mustache.Render'

Take that, replace markup with report, and remove --destdir, which gives something like:

hpc report dist-newstyle/build/x86_64-linux/ghc-8.10.7/mustache-2.4.0/t/unit-tests/hpc/vanilla/tix/unit-tests/unit-tests.tix \
'--hpcdir=dist-newstyle/build/x86_64-linux/ghc-8.10.7/mustache-2.4.0/hpc/vanilla/mix/unit-tests' \
'--hpcdir=dist-newstyle/build/x86_64-linux/ghc-8.10.7/mustache-2.4.0/hpc/vanilla/mix/mustache-2.4.0' \
'--include=Text.Mustache' \
'--include=Text.Mustache.Types' \
'--include=Text.Mustache.Parser' \
'--include=Text.Mustache.Compile' \
'--include=Text.Mustache.Render'

Pasting that into the terminal in the root of the Mustache project yields:

 59% expressions used (635/1069)
 28% boolean coverage (4/14)
       0% guards (0/6), 1 always True, 5 unevaluated
      50% 'if' conditions (4/8), 1 always False, 3 unevaluated
     100% qualifiers (0/0)
 41% alternatives used (34/82)
 56% local declarations used (13/23)
 63% top-level declarations used (48/76)