Camel route consuming ActiveMQ "Classic" message with unexpected delay in a random way

122 views Asked by At

We have an integration application based on Apache Camel and IBM MQ message broker. Part of our development work we need to be able to run this application locally on our developer laptops. The issue is that most of us have MacBook machines with Apple M1 processors and these cannot run IBM MQ message broker docker container. It seems M1 processors are not supported yet by IBM.

One solution we came up with was to fire up an EC2 in AWS have IBM MQ installed in there and open a ssh connection channel to that. Apart from the inconvenience of having to keep your EC2 compliant having one EC2 for each individual developer becomes a bit of an expense so we decided to use ActiveMQ "Classic" 5.11.1 as local message broker.

Based on some settings we configured our application to either create an ActiveMQ connection factory or an IBM MQ one and also use either an ActiveMQ or a JMS camel component which we then added to the Camel context with the same key so routes definition stays the same regardless of which message broker we use.

  var jmsComponent = useActiveMq ? activeMqComponent(cf) : new JmsComponent(new JmsConfiguration(cf));
  jmsComponent.setTransactionManager(platformTransactionManager);
  camelContext.addComponent(key, jmsComponent);

Everything works as expected and we are actually very happy with the end result. To test the functionality we used the ActiveMQ admin interface and manually injected some messages in some of our queues. In most of the cases the messages get picked up by the application instantly (as it should) but we observed that after a while it just takes forever (minutes) for a message to be picked up. We could not notice any particular pattern around this behaviour but we noticed that when starts happening it goes like this consistently.

We did not notice such a behaviour when running against an IBM MQ message broker but we cannot say it is because of ActiveMQ broker or just the ActiveMQ admin interface or the way we configured our routes. When restarting the application all messages get consumed in no time. All our routes are transacted.

I am a bit out of ideas and hopefully someone with more experience with ActiveMQ and or Camel can have some suggestions.

Initially I was using 5.16.5 which comes as a transitive dependency from spring-boot-starter-activemq but our CI server rejected because it has three vulnerabilities. Forcing the app to use 5.17.3 changed nothing.

NOTE 1

I replaced one of the Camel routes with a @JmsListener spring component and the random delays still happening which makes me think this issue has nothing to do with Apache Camel.

NOTE 2

I made the change below and it fixed the problem. The change consisted in using a fake cached connection factory with a cache size of 1. After making that change the messages were consumed off the queue as soon as they were made available with no exception. It would still be good to have an explanation about what caused the issue when cache size was 500:

private CachingConnectionFactory cachedConnectionFactory(ConnectionFactory mqQueueConnectionFactory) {
    var cachingConnectionFactory = new CachingConnectionFactory();
    cachingConnectionFactory.setTargetConnectionFactory(mqQueueConnectionFactory);
    cachingConnectionFactory.setReconnectOnException(true);
    // We don't want a cached connection factory when using ActiveMQ
    // The easiest way to achieve this is without complicating too
    // much our implementation is by setting the cache size to 1
    cachingConnectionFactory.setSessionCacheSize(useActiveMq ? 1 : 500);
    cachingConnectionFactory.setExceptionListener(e -> log.error("ErrorResponse making connection with MQ", e));
    return cachingConnectionFactory;
}
0

There are 0 answers