Micronaut backpressure

Question

We are looking for the best way to limit the number of concurrent connections accepted by an HTTP endpoint exposed with Micronaut 2.0.1. The endpoint accepts image files, writes them to disk and creates thumbnails. With too many requests we are running out of memory.

What we have found so far are the settings for Netty thread pools. We are in doubt though whether these can let us enforce a specific limit. They seem to have more to do with Netty event loop thread pools?

Is there a well-established way to implement back pressure like that with Micronaut and Netty?

Answer 1

There's no out-of-the-box configuration that enforces concurrency limits like you want. You likely want to implement a HttpServletFilter that keeps track of the number of currently executing requests and respond with status 429 Too Many Requests if the number exceeds a configured limit. Something similar to the below code should work (NOTE: not tested)

​@Override
​public Publisher<MutableHttpResponse<?>> doFilter(HttpRequest<?> request, ServerFilterChain chain) {
       ​return Flowable.fromCallable(() -> atomicRequestCounter.incrementAndGet())
                      .subscribeOn(Schedulers.computation())
                      .flatMap(numConcurrentRequests -> {
                          if (numConcurrentRequests > MAX_ALLOWED_CONCURRENCY) {
                              return Flowable.just(HttpResponse.status(HttpStatus.TOO_MANY_REQUESTS));
                          } else {
                              return chain.proceed(request);
                          }
                       })
                       ​.doOnNext(res -> atomicRequestCounter.decrementAndGet());
}

If you really want to plug this into the Netty layer, take a look at ChannelPipelineCustomizer. You can apply the same principle as the above and add a handler into the pipeline that keeps track of concurrent connections. Note that the advantage of the filter-based method is that it allows you to enforce limits to specific URIs/resources whereas if you enforce at the Netty level, it affects your entire application.