Visual Studio Test Runner doesn't find test and fails with: Some or all identity references could not be translated

Question

I'm trying to execute my tests with a .runsettings file, but the test run fails with the following message:

------ Run test started ------
Some or all identity references could not be translated.
Operation is not valid due to the current state of the object.
========== Run test finished: 0 run (0:00:00,2280287) ==========

The EventLog provides more details on the error:

Error Handler Exception: System.Security.Principal.IdentityNotMappedException: Some or all identity references could not be translated.
   at System.Security.Principal.NTAccount.Translate(IdentityReferenceCollection sourceAccounts, Type targetType, Boolean forceSuccess)
   at System.Security.Principal.NTAccount.Translate(Type targetType)
   at Microsoft.VisualStudio.Enterprise.WebInstrument.IISResetter.CreateSidFromUser(String userName)
   at Microsoft.VisualStudio.Enterprise.WebInstrument.IIS7Resetter.GetUserFromAppPool(ApplicationPool pool)
   at Microsoft.VisualStudio.Enterprise.WebInstrument.IIS7Resetter.get_Users()
   at Microsoft.VisualStudio.TraceCollector.CommonDataCollector.get_AllowedUsers()
   at Microsoft.VisualStudio.TraceCollector.CommonDataCollector.InitiateCollection()
   at Microsoft.VisualStudio.TraceCollector.CommonDataCollector.GetEnvironmentVariables()
   at Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector.GetEnvironmentVariables()
   at Microsoft.VisualStudio.TraceCollector.BaseDataCollector.Microsoft.VisualStudio.TestTools.Execution.ITestExecutionEnvironmentSpecifier.GetTestExecutionEnvironmentVariables()
   at Microsoft.VisualStudio.TestPlatform.TestRunnerService.DataCollectors.TestPlatformDataCollectorInfo.GetTestExecutionEnvironmentVariablesSync()
   at Microsoft.VisualStudio.TestPlatform.TestRunnerService.DataCollectors.DataCollectionPluginManager.GetEnvironmentVariables(Boolean& unloadedAnyCollector)
   at Microsoft.VisualStudio.TestPlatform.TestRunnerService.DataCollectors.DataCollectionPluginManager.LoadAndInitDataCollectors(DataCollectionRunSettings dataCollectionSettings, Boolean& allCollectorsLoaded)
   at Microsoft.VisualStudio.TestPlatform.TestRunnerService.DataCollectors.DataCollectionPluginManager.LoadDataCollectors(RunSettings testRunSettings, Boolean& allCollectorsLoaded)
   at Microsoft.VisualStudio.TestPlatform.TestRunnerService.WcfTestRunnerService.BeforeTestRunStart(String settingsXml, Boolean resetDataCollectors, Boolean isRunStartingNow)
   at SyncInvokeBeforeTestRunStart(Object , Object[] , Object[] )
   at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
   at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage41(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage3(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage2(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage11(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage1(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet) \r\n    at System.Security.Principal.NTAccount.Translate(IdentityReferenceCollection sourceAccounts, Type targetType, Boolean forceSuccess)
   at System.Security.Principal.NTAccount.Translate(Type targetType)
   at Microsoft.VisualStudio.Enterprise.WebInstrument.IISResetter.CreateSidFromUser(String userName)
   at Microsoft.VisualStudio.Enterprise.WebInstrument.IIS7Resetter.GetUserFromAppPool(ApplicationPool pool)
   at Microsoft.VisualStudio.Enterprise.WebInstrument.IIS7Resetter.get_Users()
   at Microsoft.VisualStudio.TraceCollector.CommonDataCollector.get_AllowedUsers()
   at Microsoft.VisualStudio.TraceCollector.CommonDataCollector.InitiateCollection()
   at Microsoft.VisualStudio.TraceCollector.CommonDataCollector.GetEnvironmentVariables()
   at Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector.GetEnvironmentVariables()
   at Microsoft.VisualStudio.TraceCollector.BaseDataCollector.Microsoft.VisualStudio.TestTools.Execution.ITestExecutionEnvironmentSpecifier.GetTestExecutionEnvironmentVariables()
   at Microsoft.VisualStudio.TestPlatform.TestRunnerService.DataCollectors.TestPlatformDataCollectorInfo.GetTestExecutionEnvironmentVariablesSync()
   at Microsoft.VisualStudio.TestPlatform.TestRunnerService.DataCollectors.DataCollectionPluginManager.GetEnvironmentVariables(Boolean& unloadedAnyCollector)
   at Microsoft.VisualStudio.TestPlatform.TestRunnerService.DataCollectors.DataCollectionPluginManager.LoadAndInitDataCollectors(DataCollectionRunSettings dataCollectionSettings, Boolean& allCollectorsLoaded)
   at Microsoft.VisualStudio.TestPlatform.TestRunnerService.DataCollectors.DataCollectionPluginManager.LoadDataCollectors(RunSettings testRunSettings, Boolean& allCollectorsLoaded)
   at Microsoft.VisualStudio.TestPlatform.TestRunnerService.WcfTestRunnerService.BeforeTestRunStart(String settingsXml, Boolean resetDataCollectors, Boolean isRunStartingNow)
   at SyncInvokeBeforeTestRunStart(Object , Object[] , Object[] )
   at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
   at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage41(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage3(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage2(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage11(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage1(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)

Removing the .runsettings file fixes the issue, but includes a number of assemblies in the coverage report which we want to exclude.

Answer 1

It turns out that when you include such a file, the ASP.NET Code Coverage collection activates. This in turn needs to reset the application pool to attach itself to it. (Hence the reference to IIS7Resetter). Turning off the ASP.NET code Coverage gathering features resolves the error:

<?xml version="1.0" encoding="utf-8"?>
<!-- File name extension must be .runsettings -->
<RunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
        <Configuration>
          <CodeCoverage>
            <!-- Include the below line in your .runsettings file-->
            <CollectAspDotNet>False</CollectAspDotNet>
          </CodeCoverage>
        </Configuration>
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>
</RunSettings>

But that didn't explain what was going wrong. Since the Stack Trace hints towards the App Pool user, we started investigating that. Turns out that setting the App Pool to LocalSystem or NetworkService actually resolves the issue as well. We had it configured to .\SomeLocalUser. Some more tinkering found the root cause, the code used to get the identity of the Application pool doesn't like the short notation. actually entering the machine name fixes it completely: MachineName\SomeLocaluser. I suspect that this is an actual bug in Visual studio and that it'll be fixed in a future timeframe, but until then these two options might help someone else figure out what's going wrong as well.

Note: It's probably best to set the CollectAspDotNet option to False by default anyway. As long as you're not running integration tests or CodedUI tests against your server, it speeds up your test run considerably.

See also:

• https://jessehouwing.net/visual-studio-vstest-fix-identity-references/
• http://msdn.microsoft.com/en-us/library/jj159530%28v=vs.110%29.aspx