Spring Integration MDC for Async Flow and Task Executors

Question

I have a flow that starts with a poller and hands off the message to several async flows downstream using task-executors to execute in parallel for a given dataset. A downstream aggregator completes the flow and notifies the poller that the flow is complete.

I would like to track every execution of the poller by using MDC so that the logs can be mapped to a particular execution of the flow.

I started by adding MDC to the poller thread (using Advice), however with this approach there could be a couple of issues:

1. How do I stamp the MDC on the executor thread when the async hand off happens?
2. Since executor uses a a thread pool, do I need to clear the MDC before the thread returns to the pool? Will there be any side effects?

Another approach would be to add MDC to the Message header and set it manually on the new thread during the async handoff. How to do that? For example, if I turn on the debug logs, the MDC should be stamped right from the beginning of the new thread execution and not from the point where my logic starts in the service activator. How to set this on the task-executor thread (and probably also remove before returning to the pool) using XML configuration? Something like an MdcAwareThreadPoolExecutor seen here. Also, I would not want the MDC logic to be spread across all the async handoff endpoints, may be there is some generic way to configure it?

Is there a better way to achieve this? Any known solutions?

Answer 1

I would like to track every execution of the poller by using MDC so that the logs can be mapped to a particular execution of the flow.

It is fully sound as "you would like to track the message journey in your flow". As you noticed there is the way to set some message header. So, why just don't map your logs by this specific header?

You can take a look into Message History pattern how to gather the whole path for the message, so then in logs you can track it back looking into message headers.

See here: https://docs.spring.io/spring-integration/docs/5.3.2.RELEASE/reference/html/system-management.html#message-history

If you really still insist on the MDC, then you definitely need to take a look into some MDCDelegatingExecutorDecorator. Some sample you can borrow from Spring Security and its DelegatingSecurityContextExecutor`: https://docs.spring.io/spring-security/site/docs/5.4.0/reference/html5/#concurrency