Observer Effect with Code Coverage in .NET

Question

I'm trying to implement code coverage for the Sentry .NET SDK.

• As this is an open source repository, I need to pick a code coverage tool that is freely available so that anyone can build the solution without having to license products.
• As we target multiple platforms, we have to be able to build this on Mac, Linux and Windows... so we can't use solutions that ship as extensions to Visual Studio

The possible solutions appear to be:

• microsoft.codecoverage
• altcover
• coverlet

I tried microsoft.codecoverage and altcover but both tools alter the results of unit tests. Specifically, instrumenting with either of those code coverage tools alters debug stack traces that get collected by Sentry (which causes our verify tests to fail). When tests are run with microsoft.codecoverage or altcover the CodeId in DebugImages is different.

Coverlet is the only one that doesn't suffer from this Observer Effect. However we've found our CI builds fail periodically when using Coverlet (e.g. here), with errors like Unable to read beyond the end of the stream. when calculating the coverage result.

Are there any other code coverage options out there for .NET or has anyone been able to successfully overcome the above problems with existing coverage tools?

Answer 1

I didn't find a solution the observer effect on the microsoft/altcover libraries but using coverlet.collector instead of coverlet.msbuild appears to be the solution to the difficulties with Coverlet (according to the docs on this known issue) and that also doesn't introduce any observer effects of it's own... so seems to be the best solution.