"Run Functional Tests" task and Specflow - App.config being ignored?

Question

I've created a Release definition in TFS 2015 update 2 as seen below:

I want to use this as part of my CI process. The build and unit tests are executed on a dedicated BizTalk build server for each check-in. Executed nightly, this (currently problematic) release should deploy to an "DevTest" BizTalk server and execute integration tests.

All is good up to the last "Run Functional Tests" task. So, the BizTalk applications are removed, then deployed to the target server in the correct order. The "IntegrationTests" assembly contains a few Specflow tests which make use of the Transmock framework (http://transmock.codeplex.com/). Steps for these tests are contained within an assembly called "Common.Commponents". This is referenced from a test assembly called "Finance.IntegrationTests.dll". The tests pass when run on my Dev VM. However, when executed by the "Run Functional Tests" task, I get the following error:

Test method Finance.IntegrationTests.ASC_BacsFeature.BacsFileMoved threw exception: NUnit.Framework.InconclusiveException: No matching step definition found for one or more steps.using System;using TechTalk.SpecFlow;namespace MyNamespace{ [Binding] public class StepDefinitions { [Given(@"an output path specified by sso app '(.)' and setting '(.)'")]public void GivenAnOutputPathSpecifiedBySsoAppAndSetting(string p0, string p1){ ScenarioContext.Current.Pending();} [Given(@"the inpu

I have set the App.config of my test project to use MsTest as seen below:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow"/>
  </configSections>
  <specFlow>
    <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
    <unitTestProvider name="MsTest"/>
    <stepAssemblies>
       <stepAssembly assembly="LCC.Integration.Common.Components" /> 
    </stepAssemblies>
  </specFlow>
<startup><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.1"/></startup></configuration>

It seems that when the tests are executed remotely, NUnit is used rather than MSTest.

The "Common.Components.dll" is successfully deployed to the GAC earlier in the release via GacUtil /I, so I'm not sure why it can't be found?

I've tried running the tests directly on the remote server. I RDP'd on, then opened a command window and ran "vstest.console Finance.IntegrationTests.dll". I then received warnings that the test input files could not be found, followed by errors because "Could not load file or assembly 'nunit.framework'".

I think the problem is, App.config of the Integration Test assembly is not being used when the tests are executed remotely. This explains why it tries to use NUnit rather than MSTest and also why it is unable to bind to the common steps - because their location is given in the node.

Does anyone know how I can make the App.config be used when testing remotely?

Answer 1

There were a couple of issues. I'd forgotten that the app.config would be compiled into finance.integrationtests.dll.config. References to dlls in the config needed to have the full strong name rather than just the name.