Subject: [hybi] max frame size. was: consensus call on ability to announce max frame size

From: Greg Wilkins <[email-masked]>
Date: Sun, 31 Jul 2011 10:39:30 +1000

Changing topic to avoid debate in consensus call thread.

On 30 July 2011 17:46, John Tamplin <[email-masked]> wrote:
> I see no mention of maximum message size in this consensus call, only
> maximum frame size.

Sorry if that muddied the water. But I just wanted to be crystal clear
we are talking about frames not messages.

On 30 July 2011 17:31, Willy Tarreau <[email-masked]> wrote:
> I'd also argue that HTTP has been living for a decade with support for
> chunked encoding which does not set a limit on chunk size nor on message
> size and has not had any trouble with that.

But HTTP also does not have a chunk-too-large error code.
Implementations are forced to deal with large chunks.  This is often
no too difficult because stream APIs are the norm for HTTP.   Large
sizes are harder to handle in the datagram API of websocket.

On 30 July 2011 23:34, Thomson, Martin <[email-masked]> wrote:
> Given that there is no constraint on message size, this capability is pointless.  Any peer that receives this indication does not need to change their behaviour in any meaningful way.  Messages of any size can be sent regardless of what frame size is supported.

Note sure if you are talking about max message or max frame size?

But a peer that does not respect a max frame indication and does not
change it's behaviour will then just receive frame-too-large errors in
response.   If we allow frame-too-large errors, then surely we need a
way to communicate what is not too large.

Also, while I think a max frame size should be optional to declare, I
think that once it is declared it should be mandatory for senders to
respect that limit.

> That said, such a negotiation might be crucial to an effective multiplexing scheme.  I suggest that this be deferred and included as part of the multiplexing work.

I think it is more than just MUX that needs this.  Because of the
datagram style API of websockets, applications will not be stream
based (blocked reading content), but will be called back when frames
and/or messages are received.  Thus there will be implementations that
have fixed buffer sizes and will not work with large frames and/or
messages.      For large messages, there is not much the
implementations can do to influence the applications, so I don't see a
limit being worthwhile.  But for a max frame size, it is pretty simple
for implementations to fragment messages to respect a limit.

regards

---

From: Diogo Pereira <[email-masked]>
Date: Sun, 31 Jul 2011 18:49:02 +0100

> for implementations to fragment messages to respect a limit.

Can't the receiver just fragment large frames into smaller ones and
pass those to the application? That way it only needs to buffer as
much as it wants to.

-- 
Diogo

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 1 Aug 2011 09:04:47 +1000

> pass those to the application? That way it only needs to buffer as

Sure it can.    But semantically websocket endpoints are meant to
handle entire messages and are not set up to stream messages or even
to handle them frame by frame.   Specifically the APIs offered to
application are message based and not stream based, so there is little
point calling such and application with a frame (what is it going to
do with 2 bytes of a 6 bytes UTF-8 character?).

So implementations that receive large frames normally have to: a)
buffer them in memory; b) buffer them somewhere else eg disk; c) send
a frame-to-large error.

Solution a) can be asking for a DOS attack, so even with flexible
buffers, eventually most implementations will implement some limit and
go to c) Solution b) is much the same, except the limit is likely to
be much larger. Going directly to solution c) is the simplest and we
should expect simple implementation to exist.

In essence the limit is really a max message size limit, but there is
little good pre-warning what that limit is, because message size is
governed by applications and many/most applications will not know in
advance how large their message are or can be.   So telling them a
limit is not very helpful as they will not be able to act on it.  If
an app that needs 128k messages is run on infrastructure that has a
64k message limit, then it will just be an error (probably a 1003?).

If however, there is a frame size limit, then there is good reason to
advertise that, because implementations can act on it and do the
fragmentation before sending.  Note that the reason for a small frame
size limit is might be because of a desired to allow efficient MUX or
control frame interleaving rather than buffer limits, so false
fragmentation by the receiver is not that useful.

cheers

---

From: Diogo Pereira <[email-masked]>
Date: Mon, 1 Aug 2011 01:03:27 +0100

> do with 2 bytes of a 6 bytes UTF-8 character?).

Then the problem is the message size, not the frame size.
Maximum message size negotiation would solve this problem, as would a
stream-based or frame-based API. Maximum frame size negotiation would
achieve nothing, since one could still send e.g. a 1 GB message in
64-byte frames.

--
Diogo