Is there a size limit on a RabbitMQ message header?

Question

I plan on storing stacktraces in the RabbitMQ message header. Do message headers have a size limit?

Answer 1

RabbitMQ is using by default the AMQP version 0.9.1. According to the AMQP protocol specification at page 31, a field-table can contains 4 billions entries:

long-uint = 4*OCTET

field-table = long-uint *field-value-pair

A long-string entry can have a length up to 4 billions octet:

long-string = long-uint *OCTET ; length + content

An AMQP message is split in 3 parts, header, properties, message. In the properties part you could put application-specific information. So the properties may contain your stacktrace.

By the way, headers and properties must be keept as small as possible because there is performance penalty

Answer 2

The answers so far seem to indicate that it would be no problem to cram a stacktrace in a header.

As John insinuates in his answer previously, some changes were made a couple of years a go that will cause an IllegalArgumentException to be thrown when the header size exceeds frame_max (by default this is indeed 128kB).

(The source code I'm referring to can be found here!)

As a (fun) side note, this was done to prevent an issue whereby messages that had single huge header that exceeded max frame size would result in the client creating and transmitting the huge header frame to server and, as a result, the server shuts the connection down with frame_too_large error and that breaks all open channels!

In order to include a stacktrace in a header, you could increase the header size, or set it to 0 for 'unlimited', but you should be aware that this isn't particularly advisable in most situations (larger values may improve throughput while smaller values may improve latency).

Answer 3

The answers so far seem to indicate that it would be no problem to cram a stacktrace in a header. However, I actually tried this, and ran into this problem:

Caused by: java.lang.IllegalArgumentException: Content headers exceeded max frame size: 163475 > 131072
    at com.rabbitmq.client.impl.AMQCommand.transmit(AMQCommand.java:115) ~[amqp-client-5.7.3.jar:5.7.3]

Seems to me that there somewhere is a 128kB limit being enforced (by default).

Answer 4

Since you're using RabbitMQ I'm guessing that you're using the standard protocol, which is AMQP. In that case you shouldn't add something like stacktraces in the header, since according to the AMQP specification 3.2.1 the header is used for standard delivery details:

The header section carries standard delivery details about the transfer of a message through the AMQP network. If the header section is omitted the receiver MUST assume the appropriate default values (or the meaning implied by no value being set) for the fields within the header unless other target or node specific defaults have otherwise been set.

From what I found the specs doesn't mention anything about a specific size however, so it might be possible to cram a stacktrace in there if you want to :)

Answer 5

There is a limit of 4k bytes for headers when using amqplib client for nodejs. When this is exceeded you will get the exception "Frame size exceeds frame max" and cosumer dies.

https://github.com/squaremo/amqp.node/issues/462