Is it a bug using mockito-kotlin on Spring Batch?

Question

I'm experiencing very weird situation AFAIK.

I'm writing an test for Spring Batch Processor which is StepScope using mockito-kotlin. Below is an example.

// Processor to test
@StepScope
@Component
class MyProcessor(
    private val myService: MyService
) {
    fun process(item: Item) {
        // doSomething
        myService.foo(arg1, arg2)
    }
}

// test
@SpringBootTest
@TestConstructor(autowireMode = TestConstructor.AutowireMode.ALL)
class MyProcessorTest(
    private val myProcessor: MyProcessor,
    @MockBean private val myService: MyService
) {
    @Test
    fun testA() {
        given { myService.foo(any(), any()) } willThrow { IllegalArgumentException("myException") }
        myProcessor.process(Item(...))
    }

    @Test
    fun testB() {
        given { myService.foo(any(), any()) } willReturn { "SomeValue" }
        myProcessor.process(Item(...))
    }
}

I ran the above code, and testA was followed by testB, and then testB failed with IllegalArgumentException("myException") even I expected myService.foo() will return "SomeVale" normally

Here is what I found:

• stacktrace was referencing the line of given clause.
• every test-cases has same instance for myService which is mock-class

I've created another test class to reproduce this situation, but failed. Mocking normal component(not step-scoped) behaved as I expected. Why is this happening ? Is this just a bug ? or expected case ?

edit)

Below is my own SpringBootTest(without Spring Batch) to debug.

@Component
class MyService {
    fun foo(arg1: String, arg2: String) {
        // doSomething
    }
}

// test
@SpringBootTest
@TestConstructor(autowireMode = TestConstructor.AutowireMode.ALL)
class MyProcessorTest(
    @MockBean private val myService: MyService
) {
    @Test
    fun testA() {
        given { myService.foo(any(), any()) } willThrow { IllegalArgumentException("myException") }

        myService.foo(arg1, arg2) // throw exception as expected
    }

    @Test
    fun testB() {
        given { myService.foo(any(), any()) } willReturn { "SomeValue" }
        myService.foo(arg1, arg2) // return value as expected
    }
}

Similar test-case, but it works well as I expected. Even confused, I changed my stubbing method from BDD-style stubbing(given().willReturn()) into doReturn().when(mock).myMethod(), then it works well ! So I thought this should be a bug, right ?

Answer 1

You are just about to jump to one of the most hideous parts of Spring. What you see is a feature, not a bug.

The problem is that the context is cached between the 2 tests. The mocked bean is cached on both tests, and the stubbing is preserved. If both tests are run in random order (which JUnit should do), you should see that the 2nd test passes randomly.

One approach to deal with this is to reset the mocks between runs, but this makes the maintenance of the tests harder. The other is that you stop being able to run tests concurrently.

@BeforeEach
fun resetMocks() {
  myService.reset()
}

Another approach, but worse for me, is to use @DirtiesContext which creates a fresh context for each test. The problem with this solution is that tests become considerably slower (creating a new spring context is crazily expensive). If you have a few hundred tests, your build can easily take 5-10 minutes.

To share my opinion, there's no good solution to this problem other than to ditch Spring altogether.