How can I run my testing application immediately after a build in TFS?

Question

I have a solution that uses the standard TFS build definition template. Unit tests are run when building the solution, but don't fail the build.

I also have a separate testing application that runs a suite of automated tests. These tests take too long to be run as part of the main build (since we need access to the built application in order to do manual testing), but I want to kick off this testing application automatically once a build has completed, and I want it to run against the built application. I don't know if it matters, but the testing application is also part of the solution, and references some of the application DLLs.

I am utterly confused as to how best to achieve this. Do I need to create a new build process template? Do I actually want to have this in a separate build?

Answer 1

I would not recommend doing this as part of a build and should instead be part of a release pipeline. Your automated integration tests should be run in an environment that is configured to collect data under test rather than to compile code.

Release Management for Visual Studio 2013 provides the ability to push your build output to a server and then execute whatever you want against it.

http://nakedalm.com/execute-tests-release-management-visual-studio-2013/

I configured a CodedIU test in the example above but really any test setup can be used. I also configured a lab (optional) to asses the environment while the tests were executing.

Answer 2

If you're using the 2013 Build Template (TfvcTemplate.12.xaml) then there is a spot to specify a Post-Test PowerShell script to run. Then you just write a powershell script that runs your custom testing app and call it from the build.

Answer 3

There is third option, in between the two. Use the Lab template to run your tests, see Automated build-deploy-test workflows. You end-up in having the 'normal' build that produce the packaged artifacts, then this other build using LabDefaultTemplate.xaml that picks from there and run your UI or integration tests.

IMHO all three suggestion are good, each with its pros and cons.