VSTest@2 Task is Updating / Overriding coverlet.runsettings and not respecting exclusions

Question

I am trying to add code coverage to a .NET Framework 4.8 solution in Azure DevOps. I have based my approach on the questions here and here and the coverlet docs, and am successfully getting code coverage results.

However this solution also contains a number of shared projects which have their unit tests in a different solution. I wish to exclude these projects, and the test projects themselves from the coverage report.

In my coverlet.runsettings file I have included the following lines based on the docs:

<Exclude>[coverlet.*.tests?]*,[*]Coverlet.Core*,[Company.*UnitTest]*</Exclude>
<Include>[Company.Application.*]*</Include>

However when the tests are run using the VSTest@2 I see the following in the logs

Provided settings file:
<?xml version="1.0" encoding="utf-8" ?>
<RunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="XPlat code coverage">
        <Configuration>
          <Format>cobertura</Format>          
          <Exclude>[coverlet.*.tests?]*,[*]Coverlet.Core*,[Company.*UnitTests]*</Exclude>
          <Include>[Company.Application.*]*</Include>
        </Configuration>
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>
</RunSettings>
Updated Run Settings:
<RunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="XPlat code coverage">
        <!-- As above -->
      </DataCollector>
      <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
        <!-- Lots of configuration ommitted for brevity -->
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>
  <RunConfiguration>
    <MaxCpuCount>0</MaxCpuCount>
    <BatchSize>1000</BatchSize>
    <ResultsDirectory>D:\a\_temp\TestResults</ResultsDirectory>
  </RunConfiguration>
</RunSettings>
**************** Starting test execution *********************

So it appears the VS test task is altering my runsettings on the fly and then not respecting the inclusions and exclusions values

My yaml is:

- task: VSTest@2
  displayName: 'Run Tests'
  inputs:
    testAssemblyVer2: '**\*UnitTests*.dll'
    searchFolder: '.\Output'
    codeCoverageEnabled: true
    runSettingsFile: .\Builds\coverlet.runsettings  

---

Can anyone suggest a setting or some other way to respect the inclusions and exclusions?

Answer 1

It is counterintuitive, but in order to VS test task does not alter runsettings on the fly, need to remove "codeCoverageEnabled: true" and add "runSettingsFile: .\Builds\coverlet.runsettings":

- task: VSTest@2
  displayName: 'Run Tests'
  inputs:
    testAssemblyVer2: '**\*UnitTests*.dll'
    searchFolder: '.\Output'
    runSettingsFile: .\Builds\coverlet.runsettings  

Code coverage still will be calculated because of runsettings file.

In order to exclude something, you can check https://learn.microsoft.com/en-us/visualstudio/test/customizing-code-coverage-analysis?view=vs-2022#sample-runsettings-file I copy pasted here in case the link is broken

<!--
About include/exclude lists:
Empty "Include" clauses imply all; empty "Exclude" clauses imply none.
Each element in the list is a regular expression (ECMAScript syntax). See /visualstudio/ide/using-regular-expressions-in-visual-studio.
An item must first match at least one entry in the include list to be included.
Included items must then not match any entries in the exclude list to remain included.
-->

            <!-- Match assembly file paths: -->
            <ModulePaths>
              <Include>
                <ModulePath>.*\.dll$</ModulePath>
                <ModulePath>.*\.exe$</ModulePath>
              </Include>
              <Exclude>
                <ModulePath>.*CPPUnitTestFramework.*</ModulePath>
              </Exclude>
            </ModulePaths>

            <!-- Match fully qualified names of functions: -->
            <!-- (Use "\." to delimit namespaces in C# or Visual Basic, "::" in C++.)  -->
            <Functions>
              <Exclude>
                <Function>^Fabrikam\.UnitTest\..*</Function>
                <Function>^std::.*</Function>
                <Function>^ATL::.*</Function>
                <Function>.*::__GetTestMethodInfo.*</Function>
                <Function>^Microsoft::VisualStudio::CppCodeCoverageFramework::.*</Function>
                <Function>^Microsoft::VisualStudio::CppUnitTestFramework::.*</Function>
              </Exclude>
            </Functions>

            <!-- Match attributes on any code element: -->
            <Attributes>
              <Exclude>
                <!-- Don't forget "Attribute" at the end of the name -->
                <Attribute>^System\.Diagnostics\.DebuggerHiddenAttribute$</Attribute>
                <Attribute>^System\.Diagnostics\.DebuggerNonUserCodeAttribute$</Attribute>
                <Attribute>^System\.CodeDom\.Compiler\.GeneratedCodeAttribute$</Attribute>
                <Attribute>^System\.Diagnostics\.CodeAnalysis\.ExcludeFromCodeCoverageAttribute$</Attribute>
              </Exclude>
            </Attributes>

            <!-- Match the path of the source files in which each method is defined: -->
            <Sources>
              <Exclude>
                <Source>.*\\atlmfc\\.*</Source>
                <Source>.*\\vctools\\.*</Source>
                <Source>.*\\public\\sdk\\.*</Source>
                <Source>.*\\microsoft sdks\\.*</Source>
                <Source>.*\\vc\\include\\.*</Source>
              </Exclude>
            </Sources>

            <!-- Match the company name property in the assembly: -->
            <CompanyNames>
              <Exclude>
                <CompanyName>.*microsoft.*</CompanyName>
              </Exclude>
            </CompanyNames>

            <!-- Match the public key token of a signed assembly: -->
            <PublicKeyTokens>
              <!-- Exclude Visual Studio extensions: -->
              <Exclude>
                <PublicKeyToken>^B77A5C561934E089$</PublicKeyToken>
                <PublicKeyToken>^B03F5F7F11D50A3A$</PublicKeyToken>
                <PublicKeyToken>^31BF3856AD364E35$</PublicKeyToken>
                <PublicKeyToken>^89845DCD8080CC91$</PublicKeyToken>
                <PublicKeyToken>^71E9BCE111E9429C$</PublicKeyToken>
                <PublicKeyToken>^8F50407C4E9E73B6$</PublicKeyToken>
                <PublicKeyToken>^E361AF139669C375$</PublicKeyToken>
              </Exclude>
            </PublicKeyTokens>