How to run multiple specflow projects through cmd?

Question

I have numerous .Net Framework class library projects, all with specflow and specrun.specflow nuget packages installed.

I'm able to run all these projects in the Test explorer of Visual Studio 2019 but I want to know if this can be run using cmd prompt.

I'm planning to automate by creating a batch file to run all the projects through cmd without having to go test explorer in VS 2019 and run them manually

Does anybody have any idea if this can be achieved? If possible, can you please share the commands needed to run them?

Edit 1:

Based on Greg Burghardt's suggestion, I did the following

1. I went to the path where vstest.console.exe is present(C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\Extensions\TestPlatform)
2. Opened cmd from that path and ran the cmd vstest.console.exe mytests.dll At first I got an error saying that the above dll was not found, so I pasted the dll to the same location and upon executing the same command again, I got the below message

No test is available in C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\Extensions\TestPlatform\mytests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

Additionally, path to test adapters can be specified using /TestAdapterPath command. Example /TestAdapterPath:.

Edit 2:

Instead of copying the dll and pasting it to the vstest.console.exe path location, I directly provided the path where the dll is residing and that ran all the tests that were present inside the dll so the cmd, will look something like

vstest.console.exe D:\Specflow\Dummy\bin\Debug\mytests.dll

Answer 1

Use the vstest.console.exe command line utility that comes with Visual Studio. This executable is buried deep inside the Visual Studio installation directory. You can find the exact path on your machine by searching Windows File Explorer for "vstest.console.exe" inside the installation folder for Visual Studio.

The basic command line arguments are:

path\to\vstest.console.exe path\to\tests.dll

That runs all tests in the DLL file generated by building a test project. There are a myriad of filtering options.

Run SpecFlow Tests By Tag

Running tests from the command line by SpecFlow tag is easy. Each tag becomes a [TestCategory] attribute, so just use the TestCategory filter:

path\to\vstest.console.exe path\to\tests.dll /TestCaseFilter:"TestCategory=SpecFlowTagName"

As an example, let's say you have this scenario:

Feature: Application Security
    In order to ...
    As a ...
    I want to ...

@SmokeTest
Scenario: Logging in
    Given "tester" is a registered user
    When the user logs in as "tester"
    Then the user should see their dashboard

The scenario above has one tag associated with it: SmokeTest. You can run this scenario, and any other scenario tagged with "SmokeTest" using this command:

path\to\vstest.console.exe path\to\tests.dll /TestCaseFilter:"TestCategory=SmokeTest"

Run SpecFlow Tests By Feature

Each feature is turned in to a test class. The feature title (not the file name. The text that comes after "Feature: ..." inside the feature file.) is turned in to a C# class name. Non alpha numeric characters are converted to "_" characters. Then the word "Feature" is appended to it.

Using this example feature:

Feature: Application Security
    In order to ...
    As a ...
    I want to ...

The feature title is Application Security, so the test class is named ApplicationSecurityFeature. Now you can run that whole feature file by full name:

vstest.console.exe tests.dll /TestCaseFilter:"FullyQualifiedName~ApplicationSecurityFeature"

Run SpecFlow Tests By Scenario

This is just a variation of running them by feature. Each scenario in a feature file becomes a test method. The title of the scenario is converted to a C# class name, replacing all non alpha numeric characters with "_".

Given this feature and scenario:

Feature: Application Security
    In order to ...
    As a ...
    I want to ...

Scenario: Logging in
    ...

The class name is "ApplicationSecurityFeature" and the test method name is LoggingIn. Again, run by fully qualified name:

vstest.console.exe tests.dll /TestCaseFilter:"FullyQualifiedName~ApplicationSecurityFeature.LoggingIn"

---

More information about the command line options for vstest can be found at Microsoft.com: VSTest.Console.exe command-line options