We have a parameterised test class and certain tests are skipped in a particular case. Almost all the tests need to setup which is added as part of the @Before method. We are using maven-failsafe-plugin with a rerun count of 3. Suppose for parameter A, the test setup function fails, then failsafe will try to rerun it. On rerun, the setup succeeds, but the test itself is skipped due to Assume.assumeTrue( assumption violation ). Here, the test is marked as failure, instead of marking it as flake. Is there any way we can tag the test as flake instead of failure.
@Before
public void setup() {
// code which can fail
}
@Test
public void testSomeMethod() {
Assume.assumeTrue(...) // This method is skipped for parameter A
}
In the fail-safe report, there is a line
<!-- a skipped test execution in re-run phase -->
which looks to be coming from https://github.com/apache/maven-surefire/commit/47aebbb6d7faad3db837fe5fe93d8ffb11bf37c8#diff-c634b539151b4b76bde50d93d5a6cd2b0f51aa411281398cc295cc721bdfa1d5R278. The plugin has fixed the issue with xml structure issue https://issues.apache.org/jira/browse/SUREFIRE-1556, but rerun in this particular case is useless, since this always will be reported as failure, if it failed in setup the first time. Is there a better way to handle this and consider skipping test on rerun as a PASS and proceed?