JUnit5 @BeforeAll Run Twice

Question

I have a Gradle based Java project that contains some unit test classes based on JUnit4 and some classes based on JUnit5. When I run all those tests locally via Gradle/Eclipse, they always pass but when these are run via Jenkins, a few of the tests fail (different tests/classes every time).

I am also using JMockit and Mockito frameworks in some of those unit tests.

One thing I have noticed is that for some Junit5-based unit tests that fail, the @BeforeAll annotated function gets executed twice as we have a check in that function to error in case we initialize a variable twice.

Adding some code of the Junit5 based class, that fails with the initializationError as the @BeforeAll runs twice causing an Exception...

    import mockit.Expectations;
    import mockit.Mocked; 
    .....
    @Mocked
    private static ClientManager mockClientManager;
        
    @Mocked
    Client mockClient;
    ....
    @BeforeAll
    public static void setMockClientManager() throws Exception {
        new Expectations() {{           
            mockClient.setActive(true);
            //putClient fails we can only have max of 1 activeClient, since this function setMockClientManager runs twice for some reason, it throws an exception
            mockClientManager.putClient(activeClient);
            //          
            mockClientManager.getActiveClient(); result = activeClient;
        }};
    }

    @Test
    public void someTestMethod() throws Exception {
        //The test functions never gets to run since the @BeforeAll failed/errored...
    }

Any idea what could be causing this issue?

Thanks!

P.S: Also after trying different options, I have noticed if we use the flag forkEvery=1 in the test task of Gradle, then the build pass. Still would like to find out why the build fails (for random classes) when a single process (which is default Gradle behaviour) is used to running the tests.

Answer 1

Ooh, there's a lot going on here. :)

First, from experience, I would not recommend comingling JMockit and Mockito in the same test-class. Likewise, comingling JUnit4 and JUnit5 in the same file is also likely bad. A particular test class should be either mockito or jmockit, and should be entirely junit4 or junit5. You can choose different permutations in different test classes. Hence having 2 test classes - ApplicationMockitoTests and ApplicationJMockitTests that both verify various behavior of Application.java. Your example looks like a JMockit+JUnit5 (OK, so far), but your description indicates you might be comingling.

Putting Expectations inside of BeforeAll seems curious. BeforeAll is usually used to setup one or more tests for execution. Meanwhile, Expectations block is usually used to declare what you anticipate will be called by the test or the behavior you want. I can sort of see where you are going, but it seems goofy.

I suspect your real issue is that you appear to be using the expectations (inside the before-all) as an attempt to configure your mock object. That's not how it works. Expectations works by saying "when a method is called, this is the result I expect". You can add in the concept of "I expect this method to be called N number of times" as well. So, invoking set-methods (including the put-method) is probably not doing what you think. Hence, I believe your real-code invokes getActive() and you want it to get a value of true. Rather than doing the setActive(true) as you do, simply put "getActive()" into your expectation, and then do a "result=true". Force the value you want (in Expectations) rather than attempting to put the value into the mock. Alternatively, you can call setActive(..) on a mock, but you would do that outside the Expectations block, not inside.