Maximum message/header length

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Maximum message/header length

Sandy Pratt
We recently encountered a situation where our unit tests were sending a known-bad string of nonsense as a message header.  We were surprised to find that this apparently caused our ~6 month old build of qpidd to crash upon receiving the message.

I tested our most recent build of qpidd  and found that while the broker (correctly) did not crash, there is still some upper limit on the length of message headers.  In the test below, I tried to send a test message with a header ~67kB in length, and received the following error on the broker side:

2010-01-26 14:29:13 error cluster(10.72.40.166:23674 READY) aborting connection 10.72.40.166:23674-4805: illegal-argument: Not enough data for field table. (qpid/framing/FieldTable.cpp:204)
2010-01-26 14:29:13 error cluster(10.72.40.166:23674 READY) aborting connection 10.72.40.166:23674-4805: framing-error: Framing version unsupported (qpid/framing/AMQFrame.cpp:113)

Without looking at the code, I presume the upper limit to be 2^16 bytes.

I have a few questions:


1)      Are max message and header lengths documented anywhere?

2)      Does the client do anything to validate the message and header length?  We're using the Java JMS client FWIW.

It's obviously sort of absurd to be sending headers this long, but I thought I'd bring it up anyway.

Thanks,

Sandy


Reply | Threaded
Open this post in threaded view
|

Re: Maximum message/header length

aconway.rh
On 01/26/2010 05:37 PM, Sandy Pratt wrote:

> We recently encountered a situation where our unit tests were sending a known-bad string of nonsense as a message header.  We were surprised to find that this apparently caused our ~6 month old build of qpidd to crash upon receiving the message.
>
> I tested our most recent build of qpidd  and found that while the broker (correctly) did not crash, there is still some upper limit on the length of message headers.  In the test below, I tried to send a test message with a header ~67kB in length, and received the following error on the broker side:
>
> 2010-01-26 14:29:13 error cluster(10.72.40.166:23674 READY) aborting connection 10.72.40.166:23674-4805: illegal-argument: Not enough data for field table. (qpid/framing/FieldTable.cpp:204)
> 2010-01-26 14:29:13 error cluster(10.72.40.166:23674 READY) aborting connection 10.72.40.166:23674-4805: framing-error: Framing version unsupported (qpid/framing/AMQFrame.cpp:113)
>
> Without looking at the code, I presume the upper limit to be 2^16 bytes.
>
> I have a few questions:
>
>
> 1)      Are max message and header lengths documented anywhere?
>
> 2)      Does the client do anything to validate the message and header length?  We're using the Java JMS client FWIW.
>
> It's obviously sort of absurd to be sending headers this long, but I thought I'd bring it up anyway.
>

Headers have to fit in one frame so they are restricted to the max-frame-size
which is 64k by default. It can be negotiated smaller when connecting but it
can't be bigger than 64k - it's a 16 bit length.

Message body is only limited by available memory (and allocated journal space on
disk if you're persisting them) The body is sent as a sequence of frames with an
"end of message" mark on the last one.

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Maximum message/header length

Rafael Schloming
Alan Conway wrote:

> On 01/26/2010 05:37 PM, Sandy Pratt wrote:
>> We recently encountered a situation where our unit tests were sending
>> a known-bad string of nonsense as a message header.  We were surprised
>> to find that this apparently caused our ~6 month old build of qpidd to
>> crash upon receiving the message.
>>
>> I tested our most recent build of qpidd  and found that while the
>> broker (correctly) did not crash, there is still some upper limit on
>> the length of message headers.  In the test below, I tried to send a
>> test message with a header ~67kB in length, and received the following
>> error on the broker side:
>>
>> 2010-01-26 14:29:13 error cluster(10.72.40.166:23674 READY) aborting
>> connection 10.72.40.166:23674-4805: illegal-argument: Not enough data
>> for field table. (qpid/framing/FieldTable.cpp:204)
>> 2010-01-26 14:29:13 error cluster(10.72.40.166:23674 READY) aborting
>> connection 10.72.40.166:23674-4805: framing-error: Framing version
>> unsupported (qpid/framing/AMQFrame.cpp:113)
>>
>> Without looking at the code, I presume the upper limit to be 2^16 bytes.
>>
>> I have a few questions:
>>
>>
>> 1)      Are max message and header lengths documented anywhere?
>>
>> 2)      Does the client do anything to validate the message and header
>> length?  We're using the Java JMS client FWIW.
>>
>> It's obviously sort of absurd to be sending headers this long, but I
>> thought I'd bring it up anyway.
>>
>
> Headers have to fit in one frame so they are restricted to the
> max-frame-size which is 64k by default. It can be negotiated smaller
> when connecting but it can't be bigger than 64k - it's a 16 bit length.
>
> Message body is only limited by available memory (and allocated journal
> space on disk if you're persisting them) The body is sent as a sequence
> of frames with an "end of message" mark on the last one.

This isn't a protocol limitation. I believe the JMS client will send
multiple header frames if the headers are large. I would guess from this
that the broker isn't defragmenting multiple header frames properly.

--Rafael


---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Maximum message/header length

Gordon Sim
On 01/26/2010 10:49 PM, Rafael Schloming wrote:
> I believe the JMS client will send
> multiple header frames if the headers are large. I would guess from this
> that the broker isn't defragmenting multiple header frames properly.

Correct, the c++ framing does not handle multi-frame header or command
segments at present (it does handle multi-frame body segments).

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:[hidden email]