XRay and XUnit reporter parses multiple test case attributes incorrectly

Question

I'm trying to run these selenium tests with multiple Tms attributes and member data. The aim is to generate XUnit report and import it to XRay test execution ticket.

[Theory]
[Tms(4045)]
[Tms(4081)]
[Trait("Category", "happyPath")]
[MemberData(nameof(TestData.DataForTest_4045), MemberType = typeof(MileagePreferencesTestData))]
[MemberData(nameof(TestData.DataForTest_4081), MemberType = typeof(MileagePreferencesTestData))]
public void Test_Should_CheckFieldsForAdvancedSettings(string actionType, RoutingPreferencesDto routingPreferences, VehiclePreferencesDto vehiclePreferences)
{
      ... some code here ...
}

However, xunit reporter does not interpret these multiple Tms attributes correctly and generates report like this:

<?xml version="1.0" encoding="utf-8"?>
<assemblies timestamp="10/02/2020 13:17:01">
  <assembly name="some_path" run-date="2020-10-02" run-time="13:17:01" total="2" passed="1" failed="1" skipped="0" time="52.537" errors="0"><errors />
    <collection total="2" passed="1" failed="1" skipped="0" name="Test collection for Project.SettingsManagement.SomeTestSuite" time="52.537">
      <test name="Project.SettingsManagement.SomeTestSuite.Test_Should_CheckFieldsForAdvancedSettings" type="Project.SettingsManagement.SomeTestSuite" method="Test_Should_CheckFieldsForAdvancedSettings" time="27.3424342" result="Fail">
        <output>[2020-10-02 13:17:03 INF] Start test execution </output>
        <failure>
          <message>OpenQA.Selenium.ElementNotInteractableException : element not interactable(Session info: chrome=85.0.4183.121)</message>
          <stack-trace>
                -some failure stack trace-
          </stack-trace>
        </failure>
        <traits>
          <trait name="Test" value="TEST-4045" />
          <trait name="Test" value="TEST-4081" />
          <trait name="Category" value="happyPath" />
        </traits>
      </test>
      <test name="Project.SettingsManagement.SomeTestSuite.Test_Should_CheckFieldsForAdvancedSettings" type="Project.SettingsManagement.SomeTestSuite" method="Test_Should_CheckFieldsForAdvancedSettings" time="25.1940956" result="Pass">
        <output>[2020-10-02 13:17:30 INF] Start test execution</output>
        <traits>
          <trait name="Test" value="TEST-4045" />
          <trait name="Test" value="TEST-4081" />
          <trait name="Category" value="happyPath" />
        </traits>
      </test>
    </collection>
  </assembly>
</assemblies>

, so, first, compiler runs TEST-4045 and generates a XML report block for both tests TEST-4045 and TEST-4081

   <traits>
    <trait name="Test" value="TEST-4045" /> // This test has been executed and has 'failed' status
    <trait name="Test" value="TEST-4081" />
    <trait name="Category" value="happyPath" />
   </traits>

, next, the compiler picks TEST-4081, and also runs it, however it generates another XML report block

   <traits>
    <trait name="Test" value="TEST-4045" />
    <trait name="Test" value="TEST-4081" /> // This test has been executed and has 'passed' status
    <trait name="Category" value="happyPath" />
   </traits>

Once the XML report is ready, I import it to XRay test execution ticket Import XML XUnit report to XRay test execution ticket

So, the problem is that wnen XRay parses the XML file it picks the first block related to TEST-4045 = - <trait name="Test" value="TEST-4045" /> and marks it with appropriate status, let's say PASSED.

Wnen parser moves to the next block related to TEST-4081, it also picks the first one <trait name="Test" value="TEST-4045" />, wipes the previous test result related to TEST-4045(PASSED) and sets the new one (FAILED) related to TEST-4081

This is how trait discoverer and tmsattribute code looks like

namespace Project
{
    public class TmsDiscoverer : ITraitDiscoverer
    {
        private const string Key = "Test";

        public IEnumerable<KeyValuePair<string, string>> GetTraits(IAttributeInfo traitAttribute)
        {
            var ctorArgs = traitAttribute.GetConstructorArguments().ToList();
            string value = FormattableString.Invariant($"TEST-{ctorArgs[0]}");
            yield return new KeyValuePair<string, string>(Key, value);
        }
    }

    [TraitDiscoverer("Project.TmsDiscoverer", "Project")]
    [AttributeUsage(AttributeTargets.Method, AllowMultiple = true)]
    public sealed class TmsAttribute : Attribute, ITraitAttribute
    {
        public TmsAttribute(object value)
        {
        }
    }
}

So, please, help me to figure out how to configure Tms attribute or TraitDiscoverer results import correctly, so that it would be nearly like this

    <traits>
    <trait name="Test" value="TEST-4045" />
    <trait name="Category" value="happyPath" />
   </traits>
   <traits>
    <trait name="Test" value="TEST-4081" />
    <trait name="Category" value="happyPath" />
   </traits>

Answer 1

Xray understands the "test" and "requirement" Trait, so you can use them directly in your code. So you could write something like:

    [Fact]
    Trait("test","CALC-2")]
    [Trait("labels","core UI")]
    public void PassingTest()
    {
        Assert.Equal(4, Add(2, 2));
    }

    [Fact]
    [Trait("requirement","CALC-1")]
    public void FailingTest()
    {
        Assert.Equal(5, Add(2, 2));
    }

where CALC-2 is the issue key of the Test and CALC-1 is the issue key of a "requirement"/story in Jira. These traits are optional, since Xray does autoprovisioning of Test entities, but the "test" Trait can be useful in case you want to enforce that a certain test method maps to a certain Test issue in Jira. In the docs, you can see a usage example.

This is how Xray processes the xUnit XML report.

So, I would recommend to use regular Traits and not the Tms ones you're using, if you can.