Cucumber tags as TestNG group in QAF and Spring boot environment

Question

I have 4 tests in a feature file with 2 different tags @first and @then. I would like @first tests to run first with parallelism and @then tests to run after all @first tests finished, with parallelism too.

The project is here : https://github.com/marcesso/qafTesting

@CucumberOptions(plugin = {"com.qmetry.qaf.automation.cucumber.QAFCucumberPlugin", "pretty", "html:target"},
        /*tags = {"@Ignore"},*/
        features = {"src/test/resources/my/custom/packagename/testing"})
public class RunnerTest extends AbstractTestNGCucumberTests {

    @Autowired
    private ObjectMapper objectMapper;


    @Test(description = "Runs Cucumber Scenarios", dataProvider = "scenarios", groups = {"first"})
    public void runScenarioFirst(PickleWrapper pickleWrapper, FeatureWrapper featureWrapper) throws Throwable {
        super.runScenario(pickleWrapper,featureWrapper);

    }
    @Test(description = "Runs Cucumber Scenarios", dataProvider = "scenarios", groups = {"then"}, dependsOnMethods =
            "runScenarioFirst")
    public void runScenarioThen(PickleWrapper pickleWrapper, FeatureWrapper featureWrapper) throws Throwable {
        super.runScenario(pickleWrapper,featureWrapper);
    }

    @Override
    @DataProvider(name = "scenarios", parallel = true)
    public Object[][] scenarios() {
        return super.scenarios();
    }

    @PostConstruct
    public void setUp() {
        objectMapper.registerModule(new JavaTimeModule());
    }

}

The issue is that all tests run twice (once per test method) and the "groups" attribute of @Test annotation does not filter tests as I expected (very bottom at https://qmetry.github.io/qaf/latest/gherkin_client.html)

There is no parallelism at all too.

I tried to filter pickles in tests methods but tests that did not match the condition were displayed as passed even if not run

if(pickleWrapper.getPickle().getTags().contains("@first")) {
            super.runScenario(pickleWrapper,featureWrapper);
}

Answer 1

In above example of RunnerTest, GherkinClient from qaf doesn't come in to picture because you are using cucumber runner. GherkinScenarioFactory and BDDTestFactory2(with qaf 2.1.15+) are GherkinClient implementation of QAF. When you are using any of them, you don't need above RunnerTest class. BDDTestFactory2 is preferred over GherkinScenarioFactory and it supports additional syntax features on top of standard gherkin syntax.

When you are using cucumber runner (RunnerTest class in your case), tags are not considered as TestNG groups. If you want to use cucumber runner to run feature file you need to handle it using cucumber options. AFAIK, What you are looking for can't be achieved with single class when using cucumber runner.

When you are using qaf you can use BDD2Factory instead of cucumber test class. You can provide xml configuration considering scenario as TestNG test case. You can mix and match different configuration options supported by TestNG same as executing Test written in java. In your case it may look like below:

<suite name="QAF Demo" verbose="0" parallel="false" data-provider-thread-count="10">

<test name="First"  parallel="methods"  thread-count="5">
   <groups>
      <run>
        <include name="first" />
      </run>
   </groups>
   <classes>
      <class name="com.qmetry.qaf.automation.step.client.text.BDDTestFactory2" />
   </classes>
</test>

<test name="second"  parallel="methods"  thread-count="5">
   <groups>
      <run>
        <include name="then" />
      </run>
   </groups>
   <classes>
      <class name="com.qmetry.qaf.automation.step.client.text.BDDTestFactory2" />
   </classes>
</test>
</suite>

You also can utilize metadata filter. Groups/Tags are also considered as meta data of testcase by qaf. So instead of :

<groups>
  <run>
    <include name="first" />
  </run>
</groups>

you can simply provide include parameter as below:

   <parameter name="include" value="{'groups': ['first']}" />

I want to reiterate here that above features only available when you are running using BDDTestFactory2 but not available when using cucumber runner. Refer using qaf-bdd-runner