Subject: Re: [hybi] max frame size. was: consensus call on ability to announce max frame size

From: Piotr Kulaga <[email-masked]>
Date: Wed, 3 Aug 2011 16:55:58 +0000

Can you please explain why we may need a size limit on a frame? It does not really makes sense to me - if a frame is too big for you for some reasons, act as intermediary and split single frame into multiple ones (assuming you understand extensions, which should be very common). 

I see some benefits in having a limit on a message size i.e. you can easily provide a message based APIs. However, as you mentioned stream APIs are a norm in HTTP world and personally I do not see a reason for any limits on message/frame size.

Kind regards,
Piotr

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Greg Wilkins
Sent: Saturday, July 30, 2011 5:40 PM
To: Hybi
Subject: [hybi] max frame size. was: consensus call on ability to announce max frame size

Changing topic to avoid debate in consensus call thread.

On 30 July 2011 17:46, John Tamplin <[email-masked]> wrote:
> I see no mention of maximum message size in this consensus call, only 
> maximum frame size.

Sorry if that muddied the water. But I just wanted to be crystal clear we are talking about frames not messages.

On 30 July 2011 17:31, Willy Tarreau <[email-masked]> wrote:
> I'd also argue that HTTP has been living for a decade with support for 
> chunked encoding which does not set a limit on chunk size nor on 
> message size and has not had any trouble with that.

But HTTP also does not have a chunk-too-large error code.
Implementations are forced to deal with large chunks.  This is often
no too difficult because stream APIs are the norm for HTTP.   Large
sizes are harder to handle in the datagram API of websocket.

On 30 July 2011 23:34, Thomson, Martin <[email-masked]> wrote:
> Given that there is no constraint on message size, this capability is pointless.  Any peer that receives this indication does not need to change their behaviour in any meaningful way.  Messages of any size can be sent regardless of what frame size is supported.

Note sure if you are talking about max message or max frame size?

But a peer that does not respect a max frame indication and does not change it's behaviour will then just receive frame-too-large errors in
response.   If we allow frame-too-large errors, then surely we need a
way to communicate what is not too large.

Also, while I think a max frame size should be optional to declare, I think that once it is declared it should be mandatory for senders to respect that limit.

> That said, such a negotiation might be crucial to an effective multiplexing scheme.  I suggest that this be deferred and included as part of the multiplexing work.

I think it is more than just MUX that needs this.  Because of the datagram style API of websockets, applications will not be stream based (blocked reading content), but will be called back when frames and/or messages are received.  Thus there will be implementations that have fixed buffer sizes and will not work with large frames and/or
messages.      For large messages, there is not much the
implementations can do to influence the applications, so I don't see a limit being worthwhile.  But for a max frame size, it is pretty simple for implementations to fragment messages to respect a limit.

regards
_______________________________________________
hybi mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/hybi

---

From: Scott Ferguson <[email-masked]>
Date: Wed, 03 Aug 2011 10:18:42 -0700

> I see some benefits in having a limit on a message size i.e. you can easily provide a message based APIs. However, as you mentioned stream APIs are a norm in HTTP world and personally I do not see…

+1

Also, as was already pointed out, since any mux extension will need a 
max frame size, it will be better to defer frame-size limits to the mux 
discussion.

Otherwise, there will be multiple frame-size limitations floating 
around: one that's properly designed for mux, and one that's half-baked.

-- Scott

> On 30 July 2011 17:46, John Tamplin<[email-masked]>  wrote:
> On 30 July 2011 17:31, Willy Tarreau<[email-masked]>  wrote:
> On 30 July 2011 23:34, Thomson, Martin<[email-masked]>  wrote:

---

From: Bruce Atherton <[email-masked]>
Date: Thu, 04 Aug 2011 12:01:30 -0700

The proposal is not to have a maximum frame limit in Websockets. The 
proposal is to allow a Websockets implementation to announce the maximum 
frame size it is able to handle. An implementation that can handle any 
frame size does not need to make any announcement at all.

This announcement would not only allow the other endpoint to modify the 
size of frames it can send, it would allow an intermediary to determine 
the maximum size it can coalesce into a single frame.

I don't understand the statement that you can split the frame. If it is 
too big for the implementation to handle, then it is too big to handle 
and the behaviour will be to drop it and send back a "Frame too large" 
error message. Better to let the other end know ahead of time rather 
than waste time sending data that will be dropped on the ground.

On 03/08/2011 10:18 AM, Scott Ferguson wrote:
>> Can you please explain why we may need a size limit on a frame? It 
>> does not really makes sense to me - if a frame is too big for you for 
>> some reasons, act as intermediary and split single frame into 
>> multiple ones (assuming you understand extensions, which should be 
>> very common).
>>
>> I see some benefits in having a limit on a message size i.e. you can 
>> easily provide a message based APIs. However, as you mentioned stream 
>> APIs are a norm in HTTP world and personally I do not see a reason 
>> for any limits on message/frame size.
>
> +1
>
> max frame size, it will be better to defer frame-size limits to the 
> mux discussion.
>

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 04 Aug 2011 12:25:24 -0700

On 08/04/2011 12:01 PM, Bruce Atherton wrote:
> proposal is to allow a Websockets implementation to announce the 
> maximum frame size it is able to handle. An implementation that can 
> handle any frame size does not need to make any announcement at all.
>
> This announcement would not only allow the other endpoint to modify 
> the size of frames it can send, it would allow an intermediary to 
> determine the maximum size it can coalesce into a single frame.

This isn't an important distinction.

Or, rather, it illustrates the problem with defining a max-frame-size 
concept now.

Mux will need a max frame size for flow-control reasons. It's an 
unavoidable concept.

If the base WebSockets defines a max-frame-size concept, that definition 
will either conflict with the mux definition (and confuse everyone), or 
constrain the mux definition, possibly in an unfortunate manner. This is 
a problem of premature specification.

>
> I don't understand the statement that you can split the frame. If it 
> is too big for the implementation to handle, then it is too big to 
> handle and the behaviour will be to drop it and send back a "Frame too 
> large" error message. Better to let the other end know ahead of time 
> rather than waste time sending data that will be dropped on the ground.

Well, that wasn't my comment, but it should be easy to understand.

If the receiver can only handle 8k frames because it has a goofy 
frame-based API, and the sender sends a 16k frame, the receiver can 
easily pass two 8k frames to the goofy frame-based API by reading the 
first 8k from the socket and passing the first frame, and then reading 
the second 8k from the socket and passing the second frame.

Since there is no message limit, sending two frames within the 8k limit 
to the goofy frame-based API will never be too big to handle.

-- Scott

>>> does not really makes sense to me - if a frame is too big for you 
>>> for some reasons, act as intermediary and split single frame into 
>>> easily provide a message based APIs. However, as you mentioned 
>>> stream APIs are a norm in HTTP world and personally I do not see a 
>>> reason for any limits on message/frame size.
>>
>> +1
>>

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 4 Aug 2011 21:38:07 +0200

Hi Scott,

> a problem of premature specification.

I would see it differently : if max-frame-size is not defined now,
implementations will not even consider there may exist a limit and
it will be harder to make them conform to future limits imposed by
mux. That's why I think that we should add provisions for a max-frame-size
in the spec and more importantly the adequate action to take when
this frame size is abused.

Regards,
Willy

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 04 Aug 2011 14:06:53 -0700

> this frame size is abused.

Since mux might decide a max-window-size is the appropriate concept and 
not care about a max-frame-size, that might not be helpful.

Getting mux right will be hard enough without constraints from the base 
WebSocket protocol. I'd prefer to give the mux group the flexibility to 
do the right thing.

At least on the server, a mux implementation will want to be tightly 
integrated with a lower level, if there is a distinct lower level. So 
I'm not sure that giving pre-mux hints to implementations will help much.

Since I think I've made my point, I'll stop here. My main point is a 
speculative constraint (like helping mux before it's defined) tend to 
cause more trouble it they solve. So if there's no compelling need for a 
feature, it should be postponed until the problem is better understood.

-- Scott

---

From: Piotr Kulaga <[email-masked]>
Date: Thu, 4 Aug 2011 21:16:43 +0000

I agree with Scott. So far the only explanation why we may need max-frame-size is to help mux extension. I would really refrain from defining something that may (or may not) be used by future extensions - we will be bloating the spec without any measurable benefit.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Scott Ferguson
Sent: Thursday, August 04, 2011 2:07 PM
To: Willy Tarreau
Cc: [email-masked]
Subject: Re: [hybi] max frame size. was: consensus call on ability to announce max frame size

On 08/04/2011 12:38 PM, Willy Tarreau wrote:
> implementations will not even consider there may exist a limit and it 
> will be harder to make them conform to future limits imposed by mux. 
> That's why I think that we should add provisions for a max-frame-size 
> in the spec and more importantly the adequate action to take when this 
> frame size is abused.

Since mux might decide a max-window-size is the appropriate concept and not care about a max-frame-size, that might not be helpful.

Getting mux right will be hard enough without constraints from the base WebSocket protocol. I'd prefer to give the mux group the flexibility to do the right thing.

At least on the server, a mux implementation will want to be tightly integrated with a lower level, if there is a distinct lower level. So I'm not sure that giving pre-mux hints to implementations will help much.

Since I think I've made my point, I'll stop here. My main point is a speculative constraint (like helping mux before it's defined) tend to cause more trouble it they solve. So if there's no compelling need for a feature, it should be postponed until the problem is better understood.

-- Scott

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 5 Aug 2011 07:23:43 +0200

> I agree with Scott. So far the only explanation why we may need max-frame-size is to help mux extension. I would really refrain from defining something that may (or may not) be used by future exten…

I understand both of your points. At the very least we must ensure that we
can define how a WS agent must react when it receives too large a frame or
message, so that the other end is aware that its frame or message was rejected
because it's too large.

This is something important, because once we deploy mux, there will very likely
be some trade-offs with minor but existing impacts on both ends. If we don't
define *now* what type of abnormal condition might be encountered, it will be
a lot harder to build something that does not break every assumption that was
made before.

With HTTP, we encounter implementation mistakes due to initially uncovered
areas (eg: 1xx is followed by another response) which caused 101 to sometimes
be mishandled. With WS, we took care for using a stricter definition of the
protocol and even reserve bits for future use. Still some types of failures
will happen and implementations must be ready to deal with them.

Regards,
Willy

---

From: Bruce Atherton <[email-masked]>
Date: Fri, 05 Aug 2011 08:14:00 -0700

> Mux will need a max frame size for flow-control reasons. It's an 

We seem to be talking past each other. I am not talking about the 
application layer or any extensions, I am talking about the core 
Websockets implementation. Pretend Mux will never exist. The core 
protocol still should allow implementations to communicate the limits 
they can handle for frame size rather than sending a bunch of data that 
will end up being wasted.

What is the downside to allowing that?

>
> If the base WebSockets defines a max-frame-size concept, that 
> definition will either conflict with the mux definition (and confuse 
> everyone), or constrain the mux definition, possibly in an unfortunate 
> manner. This is a problem of premature specification.

Base Websockets would NOT be defining a max-frame-size concept (other 
than the 63 bit payload length) for the protocol, it would just allow an 
implementation to specify it. It would be allowing implementations to 
communicate about limitations that might affect the other end before 
work is wasted. This is no different than other kinds of limitations 
that implementations might send each other, like what extensions they 
can handle or the version of the protocol they speak.

Even if Mux ends up being defined to have a maximum frame size of 32K, 
an implementation that can only handle an 8K frame size can never work 
with a frame larger than that.

> the second 8k from the socket and passing the second frame.

Again, we are not talking about the application layer. There is no API 
in this discussion, goofy or not. We are talking about a Websockets 
implementation that chooses an 8K buffer size to accept frames, and 
chooses not to buffer multiple blocks for a frame because they do not 
want to allow the other side to launch a denial of service attack.

How the server exposes the data in the buffer to the application is 
outside of the scope of this specification. Maybe it is a goofy frame 
API, maybe a stream, maybe through lazy messages. Whatever it is, it is 
not our problem.

---

From: Piotr Kulaga <[email-masked]>
Date: Fri, 5 Aug 2011 16:02:50 +0000

I do not understand your argument. Assuming that there is no goofy API layer, why would you like to buffer anything? If you are not buffering, then you do not care if the frame is 8K or 8M. Exposing frame boundaries to the upper layer does not make sense, as the boundaries can be changed by intermediaries, so they do not carry any information for upper layers (as opposed to messages, which boundaries must be preserved).

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Bruce Atherton
Sent: Friday, August 05, 2011 8:14 AM
To: Scott Ferguson
Cc: [email-masked]
Subject: Re: [hybi] max frame size. was: consensus call on ability to announce max frame size

> Mux will need a max frame size for flow-control reasons. It's an 

We seem to be talking past each other. I am not talking about the application layer or any extensions, I am talking about the core Websockets implementation. Pretend Mux will never exist. The core protocol still should allow implementations to communicate the limits they can handle for frame size rather than sending a bunch of data that will end up being wasted.

> manner. This is a problem of premature specification.

Base Websockets would NOT be defining a max-frame-size concept (other than the 63 bit payload length) for the protocol, it would just allow an implementation to specify it. It would be allowing implementations to communicate about limitations that might affect the other end before work is wasted. This is no different than other kinds of limitations that implementations might send each other, like what extensions they can handle or the version of the protocol they speak.

Even if Mux ends up being defined to have a maximum frame size of 32K, an implementation that can only handle an 8K frame size can never work with a frame larger than that.

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 5 Aug 2011 18:19:50 +0200

> I do not understand your argument. Assuming that there is no goofy API layer, why would you like to buffer anything? If you are not buffering, then you do not care if the frame is 8K or 8M. Exposin…

There may be a few reasons to buffer a bit when some transformation are
added later (eg: for deflate-frame). But I agree that in the general
case, there will be more uses of message buffering than of frame buffering.

Maybe we should try to determine usefulness the other way around : assuming
that there is no frame limit, any client, server or intermediary will be
subject to very large frames. What type of issues could we face ? I'm
currently excluding the mux case since we've all identified it and some
people think that it the max-frame-size will probably not help there
(though I don't necessarily agree).

That way if we find some valid cases we'll know why we need max-frame-size,
and if we find no valid case, let's forget about it.

Regards,
Willy

---

From: Bruce Atherton <[email-masked]>
Date: Fri, 05 Aug 2011 09:22:06 -0700

> I do not understand your argument. Assuming that there is no goofy API layer, why would you like to buffer anything? If you are not buffering, then you do not care if the frame is 8K or 8M. Exposin…

A stream API is still an API. I am not saying that there is no API, I am 
saying that what it is is not the business of this spec.

---

From: Scott Ferguson <[email-masked]>
Date: Fri, 05 Aug 2011 09:28:08 -0700

On 08/05/2011 08:14 AM, Bruce Atherton wrote:
> API, maybe a stream, maybe through lazy messages. Whatever it is, it 
> is not our problem.

A stream based receiver can use a fixed 8k buffer to implement 
arbitrary-length messages and frames without the need for 
max-frame-size. That's how streams work.

There's no unlimited buffering required. The implementation processes 
the message one 8k chunk at a time. If a frame is 64k, the 
implementation processes the data as 8 8k chunks.

-- Scott

---

From: Diogo Pereira <[email-masked]>
Date: Fri, 5 Aug 2011 17:43:33 +0100

On Fri, Aug 5, 2011 at 16:14, Bruce Atherton <[email-masked]> wrote:
> application layer or any extensions, I am talking about the core Websockets
> implementation. Pretend Mux will never exist. The core protocol still should
> allow implementations to communicate the limits they can handle for frame
> size rather than sending a bunch of data that will end up being wasted.

But why would an implementation be unable to handle a large frame?
Again, an implementation does not have to buffer the entire frame,
unless it needs to buffer the entire message, in which case the
max-frame-size limit does not help.

> Again, we are not talking about the application layer. There is no API in
> this discussion, goofy or not. We are talking about a Websockets
> implementation that chooses an 8K buffer size to accept frames, and chooses
> not to buffer multiple blocks for a frame because they do not want to allow
> the other side to launch a denial of service attack.

If you don't want to buffer more than 8K at a time, read 8K at a time
and pretend (creating a new header if needed) that those 8K are a
single, complete frame. What exactly is wrong with this?

--
Diogo

---

From: Bruce Atherton <[email-masked]>
Date: Fri, 05 Aug 2011 09:52:03 -0700

On 05/08/2011 9:28 AM, Scott Ferguson wrote:
>> Again, we are not talking about the application layer. There is no 
>> API in this discussion, goofy or not. We are talking about a 
>> Websockets implementation that chooses an 8K buffer size to accept 
>> frames, and chooses not to buffer multiple blocks for a frame because 
>> they do not want to allow the other side to launch a denial of 
>> service attack.
>>

And if the frame size is 16TB and the implementation doesn't provide a 
stream-based API?

Again we are talking past each other. My point is that if someone wants 
to write a client or server that can only handle frames that are no more 
than some arbitrary length, that should be a choice that they get to make.

Why do you want to limit what an implementer can do?

---

From: John Tamplin <[email-masked]>
Date: Fri, 5 Aug 2011 12:58:43 -0400

> And if the frame size is 16TB and the implementation doesn't provide a

Then they are going to have to buffer the entire message, whether it is sent
in a 16TB frame or tons of 1-byte frames.  Either they are providing a
steam-based API to their clients in which case they can fragment it as they
read it or they are providing just a message-based API to their clients, in
which case they have to buffer the message anyway and can simple store the
received frame data into the message as it is received.

> Again we are talking past each other. My point is that if someone wants to
> write a client or server that can only handle frames that are no more than
> some arbitrary length, that should be a choice that they get to make.
>

They can right now, and if they get a frame larger than they like they
return a "frame too large" error.

> Why do you want to limit what an implementer can do?

It increases complexity of senders -- every sender has to be prepared to
throttle down the size of what it sends to an arbitrary value.  You could
argue they are going to have to be able to do that when MUX is added anyway
and having them code their apps for it now will improve the ability to
deploy MUX, but is it a good idea to speculatively add it to the spec?

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Bruce Atherton <[email-masked]>
Date: Fri, 05 Aug 2011 10:31:48 -0700

On 05/08/2011 9:43 AM, Diogo Pereira wrote:
> On Fri, Aug 5, 2011 at 16:14, Bruce Atherton<[email-masked]>  wrote:

I don't know why. But I can make stuff up if you like.

Maybe I write a server designed to create messages that have data that 
is not constant. I might have a message of effectively infinite length 
that buffers the last 8 frames worth of data and the application 
displays it as a rolling graph. Or maybe I use Websockets for sending a 
huge amount of data out of which I pull only a few tidbits (using an 
extension) that goes into my message. Or maybe I am marshalling and 
unmarshalling domain objects and can deal with the fact that only a 
portion of the object is initialized at a time. Or maybe a whole lot of 
things I can't think of right now.

Please don't argue with these individual examples. The point is that 
there may be good reasons that people want to write a Websockets 
implementation that does not conform with what you are imagining.

> single, complete frame. What exactly is wrong with this? -- Diogo 

It forces an implementation never to do things across an entire frame. 
That rules out a lot of potential extensions.

Perhaps it would help if I gave an example of my implementation 
architecture. In it, frames are the domain of the client and server, and 
messages are the domain of the application. The class used for 
implementing messages is linked to the protocol as defined by the 
Sec-Websocket-Protocol line and if it is anything other than the default 
(empty) protocol the application provides the class.  The client and 
server get the frame, run it through any frame-based extensions, and 
pass it into the message class. Whether the message class buffers the 
whole thing, parses it, or just counts how many characters are present 
is unknown to the Websockets implementation. That is an 
application-level concern.

Now, you may not like my architecture. You may think it is stupid. But 
why on earth are you trying to forbid me to have it if I want to?

---

From: Bruce Atherton <[email-masked]>
Date: Fri, 05 Aug 2011 10:45:21 -0700

On 05/08/2011 9:58 AM, John Tamplin wrote:
> On Fri, Aug 5, 2011 at 12:52 PM, Bruce Atherton <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     And if the frame size is 16TB and the implementation doesn't
>     provide a stream-based API?
>
>
> Then they are going to have to buffer the entire message, whether it 
> is sent in a 16TB frame or tons of 1-byte frames.  Either they are 
> providing a steam-based API to their clients in which case they can 
> fragment it as they read it or they are providing just a message-based 
> API to their clients, in which case they have to buffer the message 
> anyway and can simple store the received frame data into the message 
> as it is received.

Or the server can simply return an error message that says that the 
frame is too large. Or the server can let the other end know ahead of 
time not to send any frames that are bigger than a certain maximum.

>     Again we are talking past each other. My point is that if someone
>     wants to write a client or server that can only handle frames that
>     are no more than some arbitrary length, that should be a choice
>     that they get to make.
>
>

Exactly. And that is wasted effort sending data that will never be used. 
>From my understanding,  cell phone networks go to heroic lengths to 
avoid sending more than they have to, such as resending TCP fragments to 
avoid resending a whole packet. So why not make it easy for everyone to 
avoid dropping data on the ground?

> It increases complexity of senders -- every sender has to be prepared 
> to throttle down the size of what it sends to an arbitrary value.  You 
> could argue they are going to have to be able to do that when MUX is 
> added anyway and having them code their apps for it now will improve 
> the ability to deploy MUX, but is it a good idea to speculatively add 
> it to the spec?

They have to do that anyway if they receive a frame too large error and 
still want to communicate. The only difference is that they will have to 
guess at what the right size frame might be, and may try several times 
before they get it right.

---

From: John Fallows <[email-masked]>
Date: Fri, 5 Aug 2011 11:17:03 -0700

Limiting message size would implicitly limit frame size but not the
converse, so introducing a way to announce an upper bound on acceptable
frame size doesn't prevent sending large messages.

If the message is expected to be delivered as a whole to the endpoint, then
frame size would seem to irrelevant to the amount of memory being allocated
to buffer all frames composing the message.

If the message is being considered part of a stream, such that message
boundaries are not deemed significant to the protocol running over
WebSocket, then limiting the frame size still doesn't seem that much more
useful than limiting the message size.

However, this is all largely from the perspective of the WebSocket
endpoints.  From the perspective of WebSocket intermediaries, there may be
an equivalent need to specify maximum frame size so that resources at the
intermediary can be managed effectively without interfering with the message
size observed by the WebSocket endpoints.

In short, we can probably benefit from the ability to announce
max-frame-size for intermediaries and max-message-size for the endpoints.

Kind Regards,
John Fallows

On Fri, Aug 5, 2011 at 9:58 AM, John Tamplin <[email-masked]> wrote:

> On Fri, Aug 5, 2011 at 12:52 PM, Bruce Atherton <[email-masked]>wrote:
>
> Then they are going to have to buffer the entire message, whether it is
> sent in a 16TB frame or tons of 1-byte frames.  Either they are providing a

-- 
>|< Kaazing Corporation >|<
John Fallows | CTO | +1.650.960.8148
444 Castro St, Suite 1100 | Mountain View, CA 94041, USA

---

From: Diogo Pereira <[email-masked]>
Date: Fri, 5 Aug 2011 20:44:48 +0100

Bruce,

Intermediaries can change the fragmentation of a message. An endpoint
can act as an intermediary and do the same. It's not that complicated!

If your implementation or application depends on the fragmentation
surviving intact from one endpoint to the other, you are doing it
wrong.

An extension can negotiate a maximum frame size if it needs to, so
this doesn't rule anything out.

For what it's worth, I don't see any problem with your implementation.
Just remember to fragment frames that are too large.

--
Diogo

On Fri, Aug 5, 2011 at 18:31, Bruce Atherton <[email-masked]> wrote:
>>> Websockets
> Maybe I write a server designed to create messages that have data that is
> not constant. I might have a message of effectively infinite length that
> buffers the last 8 frames worth of data and the application displays it as a
> rolling graph. Or maybe I use Websockets for sending a huge amount of data
> out of which I pull only a few tidbits (using an extension) that goes into
> my message. Or maybe I am marshalling and unmarshalling domain objects and
> can deal with the fact that only a portion of the object is initialized at a
> time. Or maybe a whole lot of things I can't think of right now.
>
> Please don't argue with these individual examples. The point is that there
> may be good reasons that people want to write a Websockets implementation
> that does not conform with what you are imagining.
>
>> If you don't want to buffer more than 8K at a time, read 8K at a time and
>> pretend (creating a new header if needed) that those 8K are a single,
>> complete frame. What exactly is wrong with this? -- Diogo
>
> It forces an implementation never to do things across an entire frame. That
> rules out a lot of potential extensions.
>
> messages are the domain of the application. The class used for implementing
> messages is linked to the protocol as defined by the Sec-Websocket-Protocol
> line and if it is anything other than the default (empty) protocol the
> application provides the class.  The client and server get the frame, run it
> through any frame-based extensions, and pass it into the message class.
> Whether the message class buffers the whole thing, parses it, or just counts
> how many characters are present is unknown to the Websockets implementation.
> That is an application-level concern.
>
> Now, you may not like my architecture. You may think it is stupid. But why
> on earth are you trying to forbid me to have it if I want to?
>

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 5 Aug 2011 22:08:59 +0200

> Just remember to fragment frames that are too large.

Reading all this, I remembered a use case that was evocated when working
on masking. At one point it was proposed to have the mask at the end of
the frame. While this proposal was not acceptable, the same principle
might apply to future extensions. For instance, an extension might
require that each frames are digitally signed. For obvious streaming
reasons, the signature would lie at the end of the frame. And intermediary
or server might then be configured to only deliver the frame if the
signature is valid. And intermediaries would probably be required not to
fragment such frames.

That said, as time passes, I'm thinking that once 1.0 is shipped, we'll
have a broader amount of users with many new expectations, and that a
version 1.1 will come soon after to help interoperability and to stop
geniuses everywhere from inventing square wheels. This 1.1 could very
well introduce new features that depend on new extensions, and it will
likely be adopted quickly. All this to say that if we can't agree on the
max-frame-size for 1.0, it might not be too late to introduce it with a
real use in 1.1 because we can expect users to upgrade.

Regards,
Willy

---

From: Bruce Atherton <[email-masked]>
Date: Fri, 05 Aug 2011 13:12:21 -0700

On 05/08/2011 12:44 PM, Diogo Pereira wrote:
> Bruce,
>
> Intermediaries can change the fragmentation of a message.

An intermediary cannot do so unless it understands all of the 
extensions. If one of the extensions requires the intermediary to buffer 
an entire frame, then that limits the size of frame that the 
intermediary can operate on.

> It's not that complicated!

It is perhaps a bit more complicated than you realize.

> surviving intact from one endpoint to the other, you are doing it

It would if I relied on that. Thankfully I don't.

But I do rely on the extensions being applied correctly on any 
refragmentation of a message that is sent. Fortunately the protocol 
allows me to rely on that.

Note that I do not claim that it is a requirement that a max frame size 
make it all the way from one end point to another. If an intermediary 
wants to consume it, fine. If they want to replace it with one of their 
own, that is fine too. So long as the frame size that I can handle is 
followed on the hop from that intermediary to me, I'm good.

> Just remember to fragment frames that are too large.

On outgoing frames I can control that. Incoming frame size is controlled 
by whoever wrote the implementation that is sending them. I have no 
control over that.

---

From: Diogo Pereira <[email-masked]>
Date: Fri, 5 Aug 2011 22:17:07 +0100

On Fri, Aug 5, 2011 at 21:12, Bruce Atherton <[email-masked]> wrote:
> An intermediary cannot do so unless it understands all of the extensions. If
> one of the extensions requires the intermediary to buffer an entire frame,
> then that limits the size of frame that the intermediary can operate on.

If an extension requires an intermediary to do the impossible then
that is a problem with the extension. I don't understand all this talk
about extensions. Extensions can have a mechanism for negotiating
max-frame-size, or anything else that they might need.

> On outgoing frames I can control that. Incoming frame size is controlled by
> whoever wrote the implementation that is sending them. I have no control
> over that.

If you're receiving a frame you must understand the extensions, so you
can probably control fragmentation. If the extension does not allow
for this (e.g. Willy's example) and also does not have a mechanism for
negotiating maximum frame size, then I would argue that that is,
again, a problem with the extension.

--
Diogo

---

From: Bruce Atherton <[email-masked]>
Date: Fri, 05 Aug 2011 17:09:57 -0700

On 05/08/2011 2:17 PM, Diogo Pereira wrote:
> On Fri, Aug 5, 2011 at 21:12, Bruce Atherton<[email-masked]>  wrote:

Huh? Who said an intermediary had to do anything impossible? And I've 
never discussed anything about extensions needing a maximum frame size. 
I think you've missed the point of what I have been saying. Let me try 
again.

I have shown how my architecture passes frames into a Message object 
that processes it in some way, perhaps buffering the data or perhaps 
discarding it or perhaps doing something in between. You have suggested 
I could just pass in buffer-sized blocks to the Message object 
regardless of frame size. Doing that would rule out any extension that 
operates on an entire frame, since such an extension would need the 
entire frame in memory at the same time.

Just to be clear since this seems to be confusing, the extension does 
not have a maximum size of frame it can handle, the core Websockets  
implementation does.

Is the ability to indicate in a header the maximum frame size that I can 
handle really so awful for you that you don't want to let me work with 
frames? I really don't get the intensity of the opposition. This seems 
so trivially easy to support and is an easy win for some scenarios. I 
don't get the downside.

> can probably control fragmentation.

??? You've completely lost me here. I can't control the incoming 
fragmentation without being able to let someone know what size I can handle.

> If the extension does not allow for this (e.g. Willy's example) and 
> also does not have a mechanism for negotiating maximum frame size, 
> then I would argue that that is, again, a problem with the extension. 
> -- Diogo

---

From: Diogo Pereira <[email-masked]>
Date: Sat, 6 Aug 2011 15:54:15 +0100

On Sat, Aug 6, 2011 at 01:09, Bruce Atherton <[email-masked]> wrote:
> I have shown how my architecture passes frames into a Message object that
> processes it in some way, perhaps buffering the data or perhaps discarding
> it or perhaps doing something in between. You have suggested I could just
> pass in buffer-sized blocks to the Message object regardless of frame size.
> Doing that would rule out any extension that operates on an entire frame,
> since such an extension would need the entire frame in memory at the same
> time.

No, I'm suggesting you pass buffer-sized *frames*. If no extensions
have been negotiated, you can simply read an arbitrary amount of
payload, create a new frame header with the correct payload length,
and you end up with a frame, an entire frame. A frame that you can
handle exactly as you would handle any other frame.

If extensions have been negotiated, this may not be possible because
of some extension requirement. That is a problem that should be solved
by that extension, either by providing a way for endpoints to announce
a maximum frame size, or, if possible, by redesigning the extension in
a way that facilitates fragmentation.

The base protocol does not have this limitation, and that is why it
does not need max-frame-size. It's not awful, it's just unnecessary,
and I think there is already enough unnecessary complexity in the
protocol.

--
Diogo

---

From: Bruce Atherton <[email-masked]>
Date: Sat, 06 Aug 2011 10:09:09 -0700

On 06/08/2011 7:54 AM, Diogo Pereira wrote:
> On Sat, Aug 6, 2011 at 01:09, Bruce Atherton<[email-masked]>  wrote:
> No, I'm suggesting you pass buffer-sized *frames*.

It's the same thing. I'm not bothered about whether I handle Frame 
objects or a buffer and a bit of state.

Look, I'm tired of this conversation. Here's the thing. Unless you can 
convince every implementer of Websockets to do things your way, then it 
just doesn't matter. There will be frame size limits in some 
implementations, which means that everyone has to deal with it. Good 
luck with the lectures, because you haven't convinced me to change my 
implementation.

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 9 Aug 2011 13:50:20 +1000

On 7 August 2011 03:09, Bruce Atherton <[email-masked]> wrote:
>  There will be frame size limits in some implementations,

Exactly!  This is a choice between have a frame size limit or not.
All implementations will have a frame size limit declared or not.
There might even conceivably be implementations that can handle 2^64
frames - but that will not be in a browser running on your i-phone any
time soon!

So the choice is not between having a limit and not having a limit.

The choice is between declaring what your limit is or keeping it a secret.

cheers

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 9 Aug 2011 00:40:20 -0700

> frames - but that will not be in a browser running on your i-phone any time
> soon!

You seem to suggest that having max. frame size would help browsers, but it won't.

Since the current JS API for WebSockets is message based, and a message can be
composed of an infinite number of frames, even a max. frame size won't help.

Either you have max. _message_ size negotiated or have a streaming API.

I am certainly not advocating for max. message size ... but then, the question
of ws browser API is out of scope wrt to the ws _protocol_.

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 09 Aug 2011 11:16:40 +0100

>a way that facilitates fragmentation.
You are advocating that the problem is solved for each extension. This 
is not a good design, IMHO.

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 09 Aug 2011 11:24:37 +0100

> max-frame-size. That's how streams work.

I don't think this is true, especially in presence of extensions like 
per frame compression.

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 09 Aug 2011 11:30:25 +0100

>But why would an implementation be unable to handle a large frame?
Because it might be running on a device with limited memory, such as a 
cell phone.

>unless it needs to buffer the entire message,
>
That is incorrect. For cases where a frame needs to be unencrypted, 
decompressed, etc, it is very likely that the whole frame is needed to 
perform the operation.

>in which case the
This only works in absence of any extension. We already have written 
down extensions which violate that.

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 09 Aug 2011 11:31:12 +0100

Bruce Atherton wrote:

>> fragment it as they read it or they are providing just a 
>> message-based API to their clients, in which case they have to buffer 
>> the message anyway and can simple store the received frame data into 
>> the message as it is received.
>

Exactly.

>>     wants to write a client or server that can only handle frames
>>     that are no more than some arbitrary length, that should be a
>>     choice that they get to make.
>>
>>
>> They can right now, and if they get a frame larger than they like 
>> they return a "frame too large" error.
>

+1.

> avoid sending more than they have to, such as resending TCP fragments 
> to avoid resending a whole packet. So why not make it easy for 
> everyone to avoid dropping data on the ground? 
>
>> to throttle down the size of what it sends to an arbitrary value. 
>>  You could argue they are going to have to be able to do that when 
>> MUX is added anyway and having them code their apps for it now will 
>> improve the ability to deploy MUX, but is it a good idea to 
>> speculatively add it to the spec?
>
> They have to do that anyway if they receive a frame too large error 
> and still want to communicate. The only difference is that they will 
> have to guess at what the right size frame might be, and may try 
> several times before they get it right.

Agreed.

---

From: John Tamplin <[email-masked]>
Date: Tue, 9 Aug 2011 14:48:35 -0400

> decompressed, etc, it is very likely that the whole frame is needed to

Typically, these APIs are stream-oriented already.  For example, zlib takes
some chunk of input data and when its output buffer is full you write it
out.  You can use whatever buffer size is convenient for you, and it
certainly doesn't have to be the entire size of the frame or message.

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 9 Aug 2011 11:52:04 -0700

> Because it might be running on a device with limited memory, such as a cell
> phone.

where is the problem?

> decompressed, etc, it is very likely that the whole frame is needed to

for crypto, use a stream cipher .. i.e. AES-CTR

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 10 Aug 2011 10:51:43 +1000

> You seem to suggest that having max. frame size would help browsers, but it won't.

It is not about "having" a max frame size. All implementation will
have a max frame size even if that is 2^64.   This is about knowing
what the max frame size is.

> composed of an infinite number of frames, even a max. frame size won't help.

This does not solve the problem of too large messages.  That is a
separate issue.

> Either you have max. _message_ size negotiated or have a streaming API.

Streaming does not help as frame boundaries can be arbitrary.  In the
simplest case, what is a JS application going to do with 3 bytes of a
6 byte UTF-8 character? An extra layer of buffering?

What if there is an extension that puts a digital signature on every
frame, so that data can be integral but not confidential.    Data in
such a frame cannot be streamed to an application until the entire
frame is received and the signature verified.

regards

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Wed, 10 Aug 2011 10:03:30 +0200

> frame is received and the signature verified.

All this stuff seems a design aberration for me. Layers separations
are not clear. Frames should occur at "transport level" (whichever
that is being already in application layer), while data arriving to
the application (the WebSocket user) should be an entire message (if
the design is message-oriented). WS user should not be disturbed with
framing stuff.

Please confirm me the following:

When a WS client receives a frame (containing a fragment of a message):

a) would such frame be sent to the JS API? (In this case JavaScript
could receive even non valid UTF-8 bytes, and of course, the JS code
would not know how to handle such incomplete data.

b) or would the WS client wait until the last frame (bit
"more_frames=0" or whatever) arrives to send the entire message to the
JS API?

I assume b) is correct (in any case?).

Thanks.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: John Tamplin <[email-masked]>
Date: Wed, 10 Aug 2011 04:13:55 -0400

> I assume b) is correct (in any case?).

The current JS API, which is discussed in WHATWG not here, only exposes a
message-oriented API that deals with complete messages.  If there is demand,
I suspect a streaming API will be added but initially it will always be case
b.

Many other implementations do provide a streaming API, and are intended for
use in the server or in non-browser clients.  In any case, the design of an
API for WebSockets is not in the scope of this WG, though of course
implementation considerations may inform choices made in the wire protocol.

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Wed, 10 Aug 2011 10:39:45 +0200

> implementation considerations may inform choices made in the wire protocol.

I understand, thanks.

Just a question please: in case of a streaming API, how will be
indicated the end of a full message? amybe by sending a special
control sequence to the API consumer after the full message has been
sent to it?

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 10 Aug 2011 20:31:08 +0100

>>Because it might be running on a device with limited memory, such as a cell
Because in a presence of an extension a frame might have to be consumed 
in its entirety before any payload data can be extracted from it. Which 
means that the the receiver needs to have enough memory for the whole frame.

>for crypto, use a stream cipher .. i.e. AES-CTR
I think you are making assumptions that that would always be the case. 
There is no such restriction in revision 10 at the moment.

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 10 Aug 2011 20:38:00 +0100

John Tamplin wrote:

> <[email-masked]> wrote:
>
>     That is incorrect. For cases where a frame needs to be
>     unencrypted, decompressed, etc, it is very likely that the whole
>     frame is needed to perform the operation.
>
> Typically, these APIs are stream-oriented already.  For example, zlib 
> takes some chunk of input data and when its output buffer is full you 
> write it out.  You can use whatever buffer size is convenient for you, 
> and it certainly doesn't have to be the entire size of the frame or 
> message. 

Ok, this might work for the per-frame compression, but there is 
currently no requirement that all extensions must support streaming. 
Greg's example of an extension with digital signature is a good one. The 
data is not even valid until the signature can be verified. If the 
signature fails to verify, what to do with the data which was already 
delivered to the application?

A requirement to work with partial frames would put additional 
restrictions on the design of future extensions.
Either way, this needs to be clarified in the draft.

---

From: John Tamplin <[email-masked]>
Date: Wed, 10 Aug 2011 16:10:45 -0400

> control sequence to the API consumer after the full message has been

That will be up to the API.  One I know of in Java gives an InputStream (or
Reader for text messages) to the client that signals EOF on the end of the
message.

---

From: Bruce Atherton <[email-masked]>
Date: Wed, 10 Aug 2011 15:19:50 -0700

On 10/08/2011 12:38 PM, Alexey Melnikov wrote:
>> write it out.  You can use whatever buffer size is convenient for 
>> you, and it certainly doesn't have to be the entire size of the frame 
>> or message. 
>
> Greg's example of an extension with digital signature is a good one. 
> The data is not even valid until the signature can be verified. If the 

Or an extension that detects whether a frame payload is plain ASCII or 
binary by checking for the high bit being set on every byte, and 
modifying the opcode as a result. I'm sure there are many other examples 
where a stream interface would not be appropriate.

---

From: Scott Ferguson <[email-masked]>
Date: Wed, 10 Aug 2011 15:33:03 -0700

> delivered to the application?

Those hypothetical extensions need to negotiate their own limits. 
Frame-size isn't the only resource limitation an extension might have.

Even extensions that have frame-size limits might have different notions 
of the limit, like hop-to-hop vs end-to-end, or dynamic MTU-discovery, 
or windowing limits, or whatever complicated flow-control MUX ends up 
requiring.

If the extension negotiates itself, it can define the right frame-size 
limit. If the base spec defines the limit, the extension is stuck.

A one-size-fits-all frame-size limit in the WebSocket spec isn't going 
to be the right answer for extensions.

-- Scott

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Thu, 11 Aug 2011 07:16:53 +0100

> Frame-size isn't the only resource limitation an extension might have.

Right on dude.  That also works for extensions in series with 
conflicting demands while leaving the base spec layer clean of this 
specialized detail.

-Andy