How to write a unit test for a method that does a retry with back off? (Using FailSafe in Java)

Question

This is the method that I would like to test:

void someMethodThatRetries(){
Failsafe.with( retryPolicy ).get(() -> callServiceX());
}

Retry policy looks like this :

this.retryPolicy = new RetryPolicy()
                .retryIf( responseFromServiceXIsInvalid() )
                .withBackoff( delay, MAX_DELAY, TimeUnit.MILLISECONDS )

This method calls a service X and retries the call to service X on a certain condition(response from X does not have certain values). Each retry is done with a delay and backoff. Test Looks like this :

@Test
    public void retriesAtMostThreeTimesIfResponseIsInvalid() throws Exception {

// Code that verifies that ServiceX got called 3 times. Service is called using a stub, and I am verifying on that stub 

}

I am writing a test that verifies that service X gets called 3 times(Maximum allowed retries are 3) when the condition is met.

Because of the Delay and Back-off the unit test takes too much time. How should we write test in this case?

One solution that I thought of is to do a separate test on the RetryPolicy that it should retry 3 times, and a separate test for the fact that it retries when condition is met.

How should I do it?

Answer 1

I'd say that you should aim at unit-testing the functions callServiceX and responseFromServiceXIsInvalid, but apart from that you are in the realm of integration-testing and subsystem-testing (aka component-testing). Everything with an algorithmic nature here is hidden behind the FailSafe and RetryPolicy classes and methods - your code is just calling them.

Therefore, many of the bugs that your code might contain lie in the interaction with / proper use of these external classes. For example, you might have messed up the order of arguments delay and MAX_DELAY - you would find this only with integration testing.

There are also potential bugs on unit-testing level, like, the value of delay might not match the specified time unit. The hassle of checking this with unit-testing in these circumstances would be too big in my eyes. Check this in a re-view, or, again, use subsystem-testing to see if the durations are as you expect.

Some additional warning: When doing integration-testing and subsystem-testing, be sure to keep the focus on the bugs you want to find. This will help you to avoid that in the end you are effectively testing the FailSafe and RetryPolicy classes - which hopefully have been tested already by the library developers.