Subject: Re: [hybi] consensus call on ability to announce max frame size

From: Bruce Atherton <[email-masked]>
Date: Mon, 01 Aug 2011 13:27:42 -0700

On 31/07/2011 8:37 AM, Paul Colomiets wrote:
> Of course maximum frame size is something different. It asks sender to 
> split longer messages to chunks of specified size. Thinking more about 
> this, I think I'm opposed to the idea of specifing frame size. The 
> problem with frame size limit, is that it gives lazy developers a 
> false impression that they can set buffer size to the maximum frame 
> size. In majority cases it's simply not true. And it's not very hard 
> to split message into frames in the intermediary either. +1 for 
> announcing message size -0 for announcing frame size 

This makes no sense to me. The application controls both the client and 
the server portions of the messages being sent. Why would anyone need a 
maximum message size? If you don't want them to be bigger than a certain 
maximum, then just don't send any that are larger.

Letting the other end know about the maximum frame size you can handle 
makes much more sense. The server implementation and the client 
implementation know nothing about each other other than that they both 
speak Websockets protocol. They have no clue what the maximum frame size 
is that they can safely send unless they can signal each other.

I don't understand what you mean about lazy programmers setting buffer 
size to maximum frame size. If my Websockets implementation broadcasts 
that it can handle a maximum frame size of 8K, then it probably will 
have buffers that are 8K. That only makes sense. How that gets assembled 
into messages is an implementation detail. Perhaps a stream is used to 
pass the message to the application layer, perhaps it isn't. Likely it 
depends on the size of messages that the application wants to send and 
the application can determine how it receives messages from the lower 
layers.

So to the point of the thread topic, I'm in favour of allowing the 
sending of a maximum frame size.

---

From: Paul Colomiets <[email-masked]>
Date: Fri, 5 Aug 2011 22:24:40 +0300

Hi Bruce,

On Mon, Aug 1, 2011 at 11:27 PM, Bruce Atherton <[email-masked]> wrote:
>This makes no sense to me. The application controls both the client
> and the server portions of the messages being sent. Why would
>anyone need a maximum message size? If you don't want them to
>be bigger than a certain maximum, then just don't send any that are larger.

Not everywhere applications are controlled well at both sides. There are
large number of open-source or otherwise duplicate installations for the
same application, with different infrastructure and network configuration
(e.g. switch from apache to nginx or to lighttpd and you will have another
kinds of limits, switch from one hosting provider to another, and again
you have another limits). This is where max-message-size comes handy.

Yes, it would be unfortunate if intermediaries could affect
max-message-size. But given that a lot of intermediaries will need to
inspect content (for fraud control, or something like that). They would
probably implement buffering of whole message, just to have
implementaion of inspection routines simpler (which in turn means
stronger). So probably they will also limit thier max message size.
And it's better to be aware of that. Otherwise intermediary getting to
big message can do only one thing: drop a connection. Which would
be pain both for users and for application writers to debug.

> I don't understand what you mean about lazy programmers setting buffer size
> to maximum frame size. If my Websockets implementation broadcasts that it
> can handle a maximum frame size of 8K, then it probably will have buffers
> that are 8K. That only makes sense.
>
The problem is slightly wider. If implementation broadcasts maximum frame size
of 1Mb or 1Gb do you still allocate buffer to this size? No. So don't
forget to limit
it's maximum value. If most of your message sizes are 512 bytes and other party
set a limit of 1K, will you have a buffer of 1K. If your message
volume is  very low,
then yes. But probably no, you will set it to 8K or 16K and will
read/write several
messages at once.

Then, how many applications would have to buffer whole
message? I think about 90%. Based on the similar experience with http (Yes,
really most of http frontends buffer whole http request before sending to a
backend).

-- 
Paul

---

From: Gabriel Montenegro <[email-masked]>
Date: Fri, 5 Aug 2011 21:46:20 +0000

Hi,

We're still deliberating on this point, but hope to declare a final consensus shortly (within a few days).

Chairs

From: [email-masked] [mailto:[email-masked]] On Behalf Of Gabriel Montenegro
Sent: Friday, July 29, 2011 23:33
To: Server-Initiated HTTP
Subject: [hybi] consensus call on ability to announce max frame size

At the meeting and on the mailing list leading up to it, the chairs saw consensus for the following action:

                Add ability to announce max frame size by  both client and server

We're confirming on the mailing list, and will declare this consensus call final by next Friday Aug 5.

Thanks,

Gabriel and Salvatore
HyBi co-chairs

---

From: Bruce Atherton <[email-masked]>
Date: Sat, 06 Aug 2011 12:33:57 -0700

On 05/08/2011 12:24 PM, Paul Colomiets wrote:
> Hi Bruce,
>
> On Mon, Aug 1, 2011 at 11:27 PM, Bruce Atherton<[email-masked]>  wrote:

I understand what you are saying, but this is still an application-level 
specific issue. If someone purchases an HTML WYSIWYG editor with a 
Websockets interface and plops it into their web page, then the author 
of the component cannot control that the server sends it only messages 
that it can understand. But if it never expects to receive messages 
larger than, say, 100 bytes, it can send a "Message too large" message 
if one arrives that is larger than that.

Having said that, I appreciate that for an empty subprotocol the 
implementation probably does need to worry about how it buffers the data 
in messages since it likely has to provide the default message 
implementation. I think that issue is what is causing people to believe 
that "Frame too large" makes no sense without "Message too large". The 
difference is that a Websockets implementation can do somethng about the 
former, but the latter requires application intervention.

> of 1Mb or 1Gb do you still allocate buffer to this size? No.

Maybe you don't. My point was not that you have to do it, but that 
matching buffer size and maximum frame size is a legitimate thing to do.

> If most of your message sizes are 512 bytes and other party

Absolutely.

> really most of http frontends buffer whole http request before sending to a

You are probably right, but when you are working at the application 
level you can decide whether that is a sensible decision to make or not.