Ignore IgnoreAttribute

Question

We have MSTest tests which automatically run in hourly production. One of these tests is marked with [Ignore] attribute because it is not yet ready to run it in our production environment.
Now I want to start that test (only) on my local environment because my local environment is ready for that test.
I try to run that test by clicking on ReSharper's icon on the left side of test code or by clicking Run Selected Tests icon in Unit Test Sessions window and nothing happens.

I fix it currently by commenting out the [Ignore] line. But now I need to be aware to remove the comment characters (//) before checking-in the code.

Is there another way to temporarly run an [Ignore]'d test ?

Answer 1

Recently when I have encountered problems such as this, I add a new Build Configuration to the visual studio project named something such as "Local Developer Debug" and use the settings from the existing Debug configuration. Then I go to "Project -> MyProjectName Properties -> Build", make sure "Local Developer Debug" is the selected configuration and add "LOCALDEVBUILD" to "Conditional compliation symbols". This allows for the use of preprocessor directives to 'toggle' code at compile time:

#if (!LOCALDEVBUILD)
    [Ignore]
#endif

Not sure if this is what you're looking for... but it allows you to run/utilize specific code depending on the intentions of the build (via the build configuration)... With this method you can leave the test ignored for more 'official' builds, but still execute it at your leisure if you so desire.

Answer 2

I have another approach. This automatically removes [Ignore] attributes on the developer side.

1. Add Directory.Build.targets to your root tests folder

<Project>
    
    <PropertyGroup Condition="'$(CI)' == 'true'">
        <DefineConstants>$(DefineConstants);CONTINUOUS_INTEGRATION_BUILD</DefineConstants>
    </PropertyGroup>

</Project>

2. Add this code to local tests:

#if CONTINUOUS_INTEGRATION_BUILD
    [Ignore]
#endif

You can also check possible CI environment variables here: https://github.com/dotnet/reproducible-builds/blob/f41ae4c0a4400acce7797691d69134f2562b26b0/src/DotNet.ReproducibleBuilds/DotNet.ReproducibleBuilds.props#L32

Answer 3

You could also use a test category to mark the test methods you do not want to be included in your automated test, e.g.

[TestCategory("IgnoreOnBuild")]

Thus not using the [Ignore] attribute. And combine this with a filter in your build definition, under Basic -> Automated Test -> Test Source -> Test Case Filter:

TestCategory!=IgnoreOnBuild

Answer 4

It looks like there are other ways to enable/disable tests using the testrunconfig file. However, if you use the IgnoreAttribute it compiles in with the code and therefore cannot be easily toggled.

From the documentation:

The Ignore attribute for a unit test resides in the source file of the unit test, together with the other attributes and source code of the test. This means that, if you disable a unit test in the Test Manager window, and later run the test by using the MSTest.exe command-line utility, the test still runs. But if you mark a unit test with the Ignore attribute, compile the assembly, and then run the test by using MSTest.exe, the test does not run. It does not run because the Ignore attribute has become part of the test.

Answer 5

As @Lasse mentioned, it is possible to set [TestCategory("DockerEnvironmentRequired")]. In addition to that, you can set a filter in your local .runsettings that includes or excludes the tests with the specified TestCategory.

Example Test:

[TestClass]
[TestCategory("DockerEnvironmentRequired")]
public class MyTests
{
    [TestMethod]
    public async Task XXX_ShouldSuccess()
    {
        ...
    }
}

Example external build environment .runsettings:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
    <TestRunParameters>
        ...
    </TestRunParameters>
    <RunConfiguration>
        <TestCaseFilter>TestCategory!=DockerEnvironmentRequired</TestCaseFilter>
    </RunConfiguration>
</RunSettings>

Locally, you would remove the TestCaseFilter node in your .runsettings to run them.

Alternative with CLI: dotnet test --filter TestCategory!=DockerEnvironmentRequired