to make the application fault tolerant, wanted to combine Retry mechanism along with CircuitBreaker of resilience4j.
I'm trying to achieve this "When in a single CircuitBreaker call, the Retry failures exceeds the CircuitBreaker failuretRateThreshold, then the CircuitBreaker should trip to OPEN state, but it is not happening.
RetryConfig retryconfig = RetryConfig.<String>custom()
.intervalFunction(intervalWithCustomExponentialRandomBackoff)
.maxAttempts(10)
.ignoreExceptions(IOException.class)
.retryOnException(e -> e instanceof HttpClientErrorException)
.build();
RetryRegistry retryregistry = RetryRegistry.of(retryconfig);
Retry retry = retryregistry.retry("serviceA",retryconfig);
CircuitBreakerConfig config = CircuitBreakerConfig
.custom()
.slidingWindowType(CircuitBreakerConfig.SlidingWindowType.COUNT_BASED)
.minimumNumberOfCalls(5)
.slidingWindowSize(10)
.failureRateThreshold(50.0f)
.permittedNumberOfCallsInHalfOpenState(3)
.waitDurationInOpenState(Duration.ofSeconds(30))
.writableStackTraceEnabled(false)
.recordException((Throwable e) -> {
if (e instanceof HttpClientErrorException) {
HttpClientErrorException httpException = (HttpClientErrorException) e;
return (Arrays.asList(HttpStatus.GATEWAY_TIMEOUT, HttpStatus.BAD_REQUEST).contains(httpException.getStatusCode()));
} else if (e instanceof HttpServerErrorException) {
return true;
} else {
return false;
}
})
.build();
CircuitBreakerRegistry registry = CircuitBreakerRegistry.of(config);
CircuitBreaker circuitBreaker = registry.circuitBreaker("serviceA");
String url = BASE_URL + "b";
Supplier<String> retrySupplier = () -> restTemplate.getForObject(
url,
String.class
);
Supplier<String> decoratedCircuitSupplier = CircuitBreaker.decorateSupplier(circuitBreaker, Retry.decorateSupplier(retry, retrySupplier));
decoratedCircuitSupplier.get() // this will call the other service (serviceB). Here as the serviceB is down, the response code will be 504.
excepectation as per my understanding: the Retry will do retries for 10 calls, but all of them fail with 504. As the CircuitBreaker also considers 504 as exception, CircuitBreaker should change to OPEN state upon 5 failures of the calls based on configuration.
what is happening: Retry keeps calling serviceB for 10 times, and CircuitBreaker remains in CLSOED state only.
https://resilience4j.readme.io/docs/getting-started-3#aspect-order quoting from the documentation
The Resilience4j Aspects order is the following: Retry ( CircuitBreaker ( RateLimiter ( TimeLimiter ( Bulkhead ( Function ) ) ) ) ) so Retry is applied at the end (if needed). If you need a different order, you must use the functional chaining style instead of the Spring annotations style or explicitly set aspect order using the following properties:
so the code is working as designed, once all the retries are exhausted only then the circuit breaker will flip its state.