Subject: [hybi] WebSocket Frame Size and latency

From: "Bob Gezelter" <[email-masked]>
Date: Wed, 10 Aug 2011 02:48:38 -0700

Andy,

Indeed. As I noted in my comment yesterday, the only absolute impact of
frame size is on the latency of what can come after a frame.
Realistically speaking, not interfering with PING/PONG and other
administrative traffic requires a latency measured in fractions of a
second. This translates into frame sizes of order 10**4 on present day
LAN interconnects, more like 10**3 on lower speed interconnections.

Multiplexing extensions, which I would prefer not to be extensions, but
part of the base specification, extend this problem in the context of
providing a degree of fairness to the different channels (e.g., an
extremely long frame, >10**8 bytes, does not create an effective,
non-removable blockage in the channel.

- Bob Gezelter, http://www.rlgsc.com

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Wed, 10 Aug 2011 11:02:18 +0100

On 08/10/2011 10:48 AM, Somebody in the thread at some point said:

Hi -

> LAN interconnects, more like 10**3 on lower speed interconnections.

Right.  I agree that sounds reasonable at the moment.

There was a thread a few months ago where I recall proposing much 
smaller max frame size for that reason, the bigger frame sizes are 
worthless.  After some to-ing and fro-ing and calculation about what 
next-gen Ethernet would consider OK 2^24 or 2^32 seemed future-proof 
enough.  We can actually send and test 2^32 byte frames in a minute or 
two on real hardware today; nobody can test the max frame size as it stands.

But there was no reaction (no defense of 2^63 either) and 2^63 is still 
sitting there in the spec.

> extremely long frame,>10**8 bytes, does not create an effective,

Yeah that boggles my mind.  There will be a lot of additional traffic 
doing the work of "TCP ACK" for each subchannel.  I can see why you 
think it should be in the base spec and don't disagree, but it's so 
difficult to get traction here I fear we will be into 2015 before it's 
actually agreed that way and it's better to take what we have to the 
bank now.

-Andy

---

From: Tobias Oberstein <[email-masked]>
Date: Thu, 11 Aug 2011 01:11:09 -0700

> LAN interconnects, more like 10**3 on lower speed interconnections.

Good that you bring that up, since I think it directs this whole frame
size discussion to more fruitful grounds. Here is my cut.

Indeed, control messages can be viewed as the messaging on a "control
channel".
When looked at this in that way, even the base WebSockets protocol has
already two channels implicitly built in.

When having a data link shared my more than 1 channel (virtual circuit),
the question of data link occupation time appears.

This again, is in part then related to frame size. But I think choosing
an optimal frame size is not so simple as it first appears.

There is a tradeoff between the duration a link is occupied on average by
channels, and the efficiency of data transfer. I.e. when you allow channels
to occupy only for very short periods, a channel can not monopolize
bandwidth, latency decreases, but overhead increases, and thus effective
bandwidth.

Another issue: I.e. ADSL links have vastly different bandwidths in downlink
versus uplink direction.

More importantly, mobile links have not only also asymmetric bandwidths,
but also time varying bandwidths.

If you put only those aspects (and there may be more) together, you'll want
to have frame size dynamically adjusted based on currently available link
bandwidth and acceptable link occupation time (latency). Per direction.
The acceptable link occupation time might be application dependent.

Announcing a max. frame size in HTTP headers during initial handshake, being
fixed for both directions and the whole lifetime of the connection does not
fit the bill.

> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Thu, 11 Aug 2011 10:55:37 +0100

> fixed for both directions and the whole lifetime of the connection does not

The guys who like frame mtu plan to create implementations which break 
if you send them legal frames bigger than some size they liked when they 
wrote the code.

So they will create an implementation which cannot cope with frames 
above a certain size.  It's that "I'm gonna break" size they want to 
formally warn peers about so they won't send frames above that size.

The standard should not wink at that broken behaviour in the base 
definition, if some extension wants to be like that then it is up to the 
peer extension to enforce a fragmentation workaround and that ugliness 
all belongs to the extension.

Optimizing frame size for throughput, overhead and latency is related 
but different issue, you could still decide to fragment at some more 
optimal, smaller limit dynamically even with this frame mtu stuff.

Really big frames are evil from a latency standpoint and it is legal to 
block the channel for literally many years as it stands, that can't be 
right either.

-Andy

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 11 Aug 2011 23:33:01 +1000

On 11 August 2011 19:55, "Andy Green (林安廸)" <[email-masked]> wrote:
> The guys who like frame mtu plan to create implementations which break if
> you send them legal frames bigger than some size they liked when they wrote
> the code. So they will create an implementation which cannot cope with frames above a certain size.

Can you please try to be a little less insulting and dismissive of the
views of others that you don't agree with.  Your representation of the
motivations of "the guys" advocated a declaration of max frame limit
are incorrect.

You appear to be arguing against the existence of frame size limits,
when what is being proposed is simply the declaration a limit IF it
exists, which it may do for a number of reasons (including but not
limited to simple implementations) and that is allowed to exist by the
existence of the frame-too-large close status.

I would understand your position better if you were advocating the
removal of frame-too-large error, but you've also acknowledged that
there are reasons to limit frame size, but that it should be
extensions that do it.     However I find it a confused situation to
advocate that multiple extensions will be required to implement frame
size limitation mechanisms that will ultimately be policed by a
frame-too-large error mechanism in the base protocol - which does not
know what the frame limit is.

---

From: Tobias Oberstein <[email-masked]>
Date: Thu, 11 Aug 2011 07:04:58 -0700

> advocate that multiple extensions will be required to implement frame size
> limitation mechanisms that will ultimately be policed by a frame-too-large

their mechanisms may be different .. as I tried to suggest in the other post,
an extension may want to dynamically adjust the frame size as a function of
current uplink/downlink bandwidth and delay

a mechanism which fixes a max. frame size during handshake for both
uplink and downlink for the whole duration of the WS connection is just _one_
conceivable

if that one mechanism would be baked into the base spec, an extension
that wants to dynamically adjust as above would be limited in what it can do,
for no reason, and to the disadvantage of that extension

> error mechanism in the base protocol - which does not know what the frame
> limit is.

the frame too large "error" should be removed from the base spec

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Thu, 11 Aug 2011 15:10:43 +0100

On 08/11/2011 02:33 PM, Somebody in the thread at some point said:
> On 11 August 2011 19:55, "Andy Green (林安廸)"<[email-masked]>  wrote:

Not sure why you feel insulted by that observation since --->

> existence of the frame-too-large close status.

"... planning to create implementations which break if you send them 
legal frames bigger than some size they liked when they wrote the code" 
seems to sum that up fairly.

I mean I'm bald Greg, I got over people noticing that a while back it's 
just a fact.

> removal of frame-too-large error, but you've also acknowledged that

It's a bit of a 'dark art' of argumentation to try to leverage the 
existence of an error already into the spec as the basis for this frame 
mtu stuff ^^

> there are reasons to limit frame size, but that it should be

Actually I heard exactly one reason, which is this putative frame 
signing thing.

> know what the frame limit is.

The base protocol deliberately averts its head from details that go on 
under the clothes of extensions.  Extensions that are constructed with 
arbitrary frame size limitations fall under that same modesty, it's the 
business of the extension peers to manage that problem that particular 
extension decides to introduce.

Many times the outcome of discussions here is push it into an extension 
or let an extension take care of it.  Seems this is another one because 
the only reason found for wanting it involves a characteristic of a 
particular proposed extension which is able to solve its requirement in 
the same extension.

-Andy

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 11 Aug 2011 10:49:12 -0600

This is your Area Director speaking. I would very much like to issue an
IESG ballot for our base document in the near future. The WG needs to
decide this matter in order for us to proceed. See below for details.

> know what the frame limit is.

Apparently there is a disconnect in the specification: as currently
written, an implementation can reject a frame because it is too big
(error code 1004), but it can't signal its maximum frame size.

Therefore, as your Area Director, I am requesting a consensus call on
the following alternative.

1. Remove error code 1004 "Frame Too Large".

or

2. Retain error code 1004, add an OPTIONAL way for an entity to signal
its maximum frame size, and specify that an implementation SHOULD NOT
send frames larger than the maximum frame size signalled by its peer.

Please reply to this consensus call by the close of business next
Wednesday, August 17, 2011. When you reply, please indicate your
preference ("1" or "2") and provide a brief reason, if appropriate.

Thank you.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Bruce Atherton <[email-masked]>
Date: Thu, 11 Aug 2011 10:40:21 -0700

> preference ("1" or "2") and provide a brief reason, if appropriate.

I very strongly prefer 2.

No matter what the WG decides, I will be closing connections if a frame 
is larger than the maximum that I can deal with. My implementation is 
only intended to be used in situations where the vast majority of 
messages will be small, and if there is ever a message larger than 64K 
it will either be a bug or an attack. That means I will never need 
frames that are bigger than that under any circumstances.

Although I could implement in a way to allow very large frames, this 
would be a violation of YAGNI, overengineering for a case that should 
never happen. Doing the simplest thing possible that will work requires 
that I match my maximum frame size with my buffer size.

I will deal with any frame I receive no matter what size it is within 
the range of the payload length, but dealing with it may well mean 
closing the connection. This is in complete conformance with the spec 
(whether there is a specific error code for closing the connection or 
not), so there is no chance that this would be a broken implementation.

To me, the question is how other implementations will be interoperable 
with mine. Option 2 makes it easy. Option 1 makes it difficult. Since 
option 2 allows all behaviour to be completely optional, I see no reason 
not to adopt it into the spec.

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Thu, 11 Aug 2011 18:53:07 +0100

> send frames larger than the maximum frame size signalled by its peer.

1) Remove the error code.

> preference ("1" or "2") and provide a brief reason, if appropriate.

The only valid use that has been given for frame mtu scheme involves one 
proposed extension, and those extension peers can arrange to meet their 
restrictions and perform any negotiation at that level without any 
footprint in the base spec.

Introducing it at the base spec instead is inviting people to make 
fragile implementations by buffering whole frames and adds needless 
complexity to the protocol and testing.

-Andy

---

From: Tobias Oberstein <[email-masked]>
Date: Thu, 11 Aug 2011 11:22:12 -0700

> 1. Remove error code 1004 "Frame Too Large".

+1, since it's consequent and removes a source of confusion and misdirection.

There are conceivable use cases for having

- no frame size limits (where 2^63 serves practically infinite)
- max. frame size negotiated per connection, up&down, fixed for duration of connection
- dynamic/adaptive frame size, different for up/down
- non-negotiated, fixed size

and so on ..

Use cases will determine requirements and by that mechanisms. Mechanisms will differ
and may contradict each other.

Constraints on frame size should be the business of extensions. Not the base spec.

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 12 Aug 2011 09:20:53 +1000

As it stands, my preference is for #2, as I believe that resource
limited implementations like what Bruce Atherton have described are
valid and they need a defined error code to indicate the reason of the
close.

However, if we added a message-too-large error, then I would support
option #1 (removal of frame-too-large).

The significant difference between frame-too-large and
message-too-large is that ws implementations can do something about
the frame too large error, which essentially represents a miss
negotiation of protocol configuration.   The message-too-large error
is fatal as far as the protocol is concerned, as there is nothing an
implementation can do to reduce the size of the application messages,
other than report the reason for the failure.   In the absence of
extensions, I suspect that implementations like Bruce's are really
limited by message size and not frame size.

My own implementation support both a max frame size or streaming data
to the application layer.   If it is accumulating frames into a
message, it also supports a maximum message size.  Currently if that
maximum message size is exceeded it is miss-using the 1004
frame-too-large close code to close the connection, because there is
no other reasonable alternative.

regards

---

From: Roberto Peon <[email-masked]>
Date: Thu, 11 Aug 2011 16:32:09 -0700

+1 message to large
-=R

On Thu, Aug 11, 2011 at 4:20 PM, Greg Wilkins <[email-masked]> wrote:

> if

---

From: Roberto Peon <[email-masked]>
Date: Thu, 11 Aug 2011 16:32:30 -0700

er, message-*too-large.
-=R

---

From: "Howard Wang(Hao)" <[email-masked]>
Date: Fri, 12 Aug 2011 01:33:24 +0000

Option 2

Though prefer to include more robust optional approaches to acquire the reason for failure.

or

Thank you.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Bruce Atherton <[email-masked]>
Date: Thu, 11 Aug 2011 19:39:58 -0700

> limited by message size and not frame size.

Not for me, although I can imagine it might be for others. If a message 
exceeds the maximum size for a particular subprotocol on the sending 
side it is never sent to the Websockets layer. If it is too large on the 
receiving side then the application sends a "Message too large" message 
and all further frames for that message are discarded.

I'd be willing to bet that "Message too large" is less of an issue than 
you think for other implementations, too. Messages are often not all the 
same size. The applications that will be using my implementation have 
80% of messages less than 4K, and 95% are less than 8K. If it were 
called for I could have the odd outlier message in the multimegabyte 
range and it wouldn't significantly impact the total memory needed for 
messages. But make the buffers for frames all multiple megabytes and 
there are serious scaling issues. So "Message too large" is probably far 
less of a concern than "Frame too large" in many implementations like 
mine that buffer frames.

As Bob said message size and frame size are orthogonal. Even though 64K 
is my maximum message size, I do not want to have to create a buffer 
that size for every incoming frame, it would be terrible for both 
scaling and latency reasons. That is one of my concerns if 
max-frame-size does not make it into the spec. I may have to increase my 
buffer size far beyond what is sensible to the requirements just to deal 
with implementations that insist on sending every message in its own 
individual frame, or that happen to choose a size bigger than the one 
chosen as optimal for my situation. If a major browser chose to allow 
frame sizes of 64K and I couldn't send a max-frame-size, I'd be forced 
to adopt the complexity of streaming from connection to message.

> no other reasonable alternative.

There is no reasonable alternative at the Websockets layer, and I 
understand why you would want a "message too large" close status. If you 
can handle it at the application layer instead, though, it is better. If 
the application wants to close the connection it can and if it wants to 
say why it is closing the connection it can send a message before it 
closes it. Or it can arrange for the message to be resent in a smaller 
form if it has a mechanism for doing so. Or it can ignore dropped 
messages if that is appropriate. Only the application knows for sure.

But I get that for many implementations, maximum message size is a 
little leaky into the Websockets layer and having the ability to signal 
it at that layer could be desirable. Please don't drop maximum frame 
size to get it, though. They serve different needs.

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 12 Aug 2011 13:43:35 +1000

Bruce,

I agree that frame sizes and message sizes are different things.

The reason my implementation has a message size limit is that
applications have the option of letting the implementation aggregate
byte data arriving into a text message, thus forcing the
implementation to buffer the entire message before delivery to the
application.     Thus I have a max message size (expressed in
characters) to be set to control that buffering.   If the message is
too large, then I cannot defer to the application to send a
message-too-large error, because the application never sees the
inbound message.  I thus am currently sending a 1004 frame-too-large.
 It would be good to have a proper error code for that - although I
guess I could add the option to simply discard the message with a
local warning.

Note that applications are also free to use an alternate API to
consume bytes as they arrive, in which case no limit is needed or
applied.

Separately to message limits, I have the ability to limit frame sizes
to the buffer sizes or let the parser create fake fragments for large
frames that are streamed to the aggregation layer.  There is no reason
in my current implementation to actually limit the frame size, but the
capability is there so that future extensions that may need to process
entire frames or that have latency concerns can restrict frame size.
If we remove the frame-too-large error code, then I will probably
remove the option to limit frame sizes to a single buffer and the
extensions will have to fend for themselves.

So reading about your example, you appear to have the ability to
receive large messages in small buffers.  You appear to be saying that
you want the frame size restriction so that you can keep the small
buffers.       But I'm not understanding why you need entire frames to
fit within those small buffers?  Unless you are running extensions
that need to process entire frames, you should be able to process
large frames in small buffers (in my case if I receive a frame larger
than my buffer, I pass it to the frame aggregation/handling layer as a
fake smaller frame and when the remaining data is received in the same
buffer it is passed on as a fake continuation frame).  Can you explain
a bit more why you want to process the entire frame at once?

cheers

On 12 August 2011 12:39, Bruce Atherton <[email-masked]> wrote:
> exceeds the maximum size for a particular subprotocol on the sending side it
> is never sent to the Websockets layer. If it is too large on the receiving
> side then the application sends a "Message too large" message and all
> further frames for that message are discarded.
>
> I'd be willing to bet that "Message too large" is less of an issue than you
> think for other implementations, too. Messages are often not all the same
> size. The applications that will be using my implementation have 80% of
> messages less than 4K, and 95% are less than 8K. If it were called for I
> could have the odd outlier message in the multimegabyte range and it
> wouldn't significantly impact the total memory needed for messages. But make
> the buffers for frames all multiple megabytes and there are serious scaling
> issues. So "Message too large" is probably far less of a concern than "Frame
> too large" in many implementations like mine that buffer frames.
>
> As Bob said message size and frame size are orthogonal. Even though 64K is
> my maximum message size, I do not want to have to create a buffer that size
> for every incoming frame, it would be terrible for both scaling and latency
> reasons. That is one of my concerns if max-frame-size does not make it into
> the spec. I may have to increase my buffer size far beyond what is sensible
> to the requirements just to deal with implementations that insist on sending
> every message in its own individual frame, or that happen to choose a size
> bigger than the one chosen as optimal for my situation. If a major browser
> chose to allow frame sizes of 64K and I couldn't send a max-frame-size, I'd
> be forced to adopt the complexity of streaming from connection to message.
>
> There is no reasonable alternative at the Websockets layer, and I understand
> why you would want a "message too large" close status. If you can handle it
> at the application layer instead, though, it is better. If the application
> wants to close the connection it can and if it wants to say why it is
> closing the connection it can send a message before it closes it. Or it can
> arrange for the message to be resent in a smaller form if it has a mechanism
> for doing so. Or it can ignore dropped messages if that is appropriate. Only
> the application knows for sure.
>
> But I get that for many implementations, maximum message size is a little
> leaky into the Websockets layer and having the ability to signal it at that
> layer could be desirable. Please don't drop maximum frame size to get it,
> though. They serve different needs.
>
>

---

From: Tobias Oberstein <[email-masked]>
Date: Thu, 11 Aug 2011 22:47:36 -0700

> frame sizes of 64K and I couldn't send a max-frame-size,

Both Firefox and Chrome currently send a message - independent of size -
always _unfragmented_ (that is fragment size = message size)

I have tested this with messages up to 8MB.

Both will receive a 8MB message arbitrarily fragmented (tested with frame
sizes from 64 bytes - 8MB)

>  I'd be forced

There is no complexity in doing streaming. On the contrary: it is easy,
good design, and you can layer buffering to any amount/unit on top.

Note that you can't layer streaming a implementation which insists on
buffering.

In any case: having a specific implementation choice be the reason to
decide about protocol questions appears weird.

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Fri, 12 Aug 2011 06:58:30 +0100

> valid and they need a defined error code to indicate the reason of the

The choice in the consensus call is a false dichotomy... in fact we can 
keep the error code and not implement the rest of the crap in #2 into 
the spec if we like, they're not mutually exclusive.  This is the "dark 
art of argumentation" I referred to.  Peter, did you come up with that 
either-or yourself?

I wrote libwebsockets targeting weak ARM9 platforms and it was developed 
both on a PC and a couple of ARMv5-based devices, so I have some 
experience from the angle of embedded websockets implementation.

In fact a websocket peer experiences a websocket frame as a part of, a 
singular, or a series of TCP/IP packets.  The only other footprint of a 
frame per se is the state of how much is left to come and where it fits 
inside the message.  So, other than what an implementation must be able 
to do with the message payload, frame processing footprint does not 
extend beyond max packet size and a few bytes of state.

Because frames can legally be a factor of 2^16 and more beyond the 
resources of any credible peer for the next 10 years, it's a red herring 
to talk about "resource limited implementatons" and "frames".  If you 
take the wrong approach to processing frames, ALL peers are "resource 
limited implementations" in the face of 2^63 frames.

-Andy

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 12 Aug 2011 17:43:46 +1000

On 12 August 2011 15:58, "Andy Green (林安廸)" <[email-masked]> wrote:
> The choice in the consensus call is a false dichotomy... in fact we can keep
> the error code and not implement the rest of the crap in #2 into the spec if
> we like, they're not mutually exclusive.

This is the only option that I'm strongly opposed to.

It is ridiculous to allow an implementation to say: "The frame you
sent is too large, but I'm not going to tell you what size I will
accept".

If we are to allow a limit, then we have to provide a way to
communicate what that limit is.   This can be static via a handshake
header or dynamic via some control messages, but it can't be
telepathic!

---

From: "Simon Pieters" <[email-masked]>
Date: Fri, 12 Aug 2011 10:29:40 +0200

> preference ("1" or "2") and provide a brief reason, if appropriate.

I prefer 1.

I think 2 is unnecessary complexity. The more overall complexity the  
protocol has, the less likely it is that it will be implemented  
interoperably.

-- 
Simon Pieters
Opera Software

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Fri, 12 Aug 2011 10:01:25 +0100

On 08/12/2011 08:43 AM, Somebody in the thread at some point said:
> On 12 August 2011 15:58, "Andy Green (林安廸)"<[email-masked]>  wrote:

The aggregate message can be "too large", but not a frame unless your 
frame parser is broken (or your signing extension peer ignored the frame 
size limit agreed / implied).

You may say, well, let's say he sends me a message that's too big in one 
frame, what about then?  We could tell him, right?  Yeah you could tell 
him your message limit but he can't react at the frame level, and you're 
telling him about your message capability not frame.  If he sends you 
the message in multiple smaller frames the message is still too big, if 
he cuts it into multiple messages it's not one message any more.

So frame mtu isn't doing anything useful.

-Andy

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 12 Aug 2011 11:09:42 +0200

> preference ("1" or "2") and provide a brief reason, if appropriate.

I prefer #2, because given all the reluctance that was exposed in this
thread, I'm fairly sure than when we later *need* to implement frame
size limitation for whatever extension that needs it (eg: compression
or digital signature), it will not be possible at all because some
lazy implementations will have major design limitations that will make
them incompatible with shorter frames.

I personally do not see what becomes complex or difficult in deciding
that a message that is larger than a *known* frame size limit should
be emitted in multiple frames. It's just a matter of repeating a frame
header from time to time.

I would also add that framing has been there for a very long time now,
and I suspect that a number of implementations have silently decided
that they did not need it, which would explain why suddenly forcing
them to emit frames is annoying. If we're at a point where framing is
supposed not to be supported at all by some implementations, we probably
have more serious issues to solve than to decide whether or not we
should announce a frame size.

Regards,
Willy

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Fri, 12 Aug 2011 10:32:56 +0100

> or digital signature), it will not be possible at all because some

Compression won't care, it can pull in or spill a subblock at a time 
unrelated to actual frame size or like zlib work at bit level.

Atomic frame signing is a special hypothetical case but even that's not 
useful compared to message signing, since it can be attacked by, eg, 
MITM setting FIN on a signed packet that was not actually the last one, 
frame header not being signed payload and subject to meddling.

> header from time to time.

Since only non-base extensions care, let the extension peers be 
responsible for the chopping-up according to that particular extension's 
limitations.

Bear in mind extension implementations are intimately specific to the 
framework they're using.  If the guy writing the signed frame extension 
for websocket framework X finds frame handling is broken, he will get it 
fixed in websocket framework X so his extension can work.

> them to emit frames is annoying. If we're at a point where framing is

Interoperability tests which Greg and I have written (eg, fraggle test 
in libwebsocket) are the answer to keeping implementations honest or 
shown to be broken.  Adding in bloat to the spec that doesn't actually 
do anything useful won't do what you're hoping here.

-Andy

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 12 Aug 2011 12:16:08 +0200

> unrelated to actual frame size or like zlib work at bit level.

if it's frame-based compression, then yes it will matter.

> frame header not being signed payload and subject to meddling.

I don't agree. Signing of frames is very useful because it saves
intermediaries from buffering the whole *message*. Instead, they can
check each frame's signature and forward them as they're checked. It
then makes a lot more sense to validate a few kB at a time and let them
pass than to store a large block to disk, validate its signature then
let it go.

Imagine someone who would make use of WS for telephony. It would be
a well-suited use of digital signatures and you'd want the frames to
be signed, not the message (which is the whole conversation).

> responsible for the chopping-up according to that particular extension's 

I have no issue with that. My concern is that if there is so much reluctance
in adopting something as trivial as setting a limit on emitted frames size,
surely there is an underlying reason which will surface again when we'll want
to design those extensions. My fears now are that 1 frame == 1 message for
many years to come and that MUX and other possibly useful extensions will
not be possible just because of a few broken implementations which will
heavily rely on the assumption above.

> fixed in websocket framework X so his extension can work.

Except that application live for many years and are not changed every
few months.

> do anything useful won't do what you're hoping here.

But why are you constantly saying that setting a limit is "bloat" Andy ?
It's just a matter of storing a value and use it as a check when messages
are to be framed. We're not speaking about *forcing* every agent to send
that large frames, but the opposite, ask that nobody along the path sends
frames larger than what the whole path reliably supports without efforts.

The bloat would come from supporting any frame size because some
implementations will have to do ugly things for that.

Regards,
Willy

---

From: Salvatore Loreto <[email-masked]>
Date: Fri, 12 Aug 2011 13:42:14 +0300

> art of argumentation" I referred to.  Peter, did you come up with that 
(as individual)

if I understood you correctly you state that an implementation should be
ready
to receive buffer of any size (till 2^63) so I don't see in this case
the rational to keep the error code 1004 "Frame too large"

in your implementation in which case/scenario do you send this error code,
if you do.

cheers
/Sal

-- 
Salvatore Loreto
www.sloreto.com

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 12 Aug 2011 12:58:37 +0200

> header or dynamic via some control messages, but it can't be

Of course. Which is the problem in indicating maximun frame size
during the handshake negotiation?? that's in fact a *negotiation*
process!
The HTTP request could have a X-WebSocket-Frame-Limit header
indicating the maximum frame size the client can receive, and the 101
response could contain same header indicating maximun frame size the
server accepts. This is the way rather than ugly reactions after the
limit has been reached.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: "Len Holgate" <[email-masked]>
Date: Fri, 12 Aug 2011 12:45:29 +0100

> send frames larger than the maximum frame size signalled by its peer.

Well, I for one have found the discussion useful and have changed my mind
somewhat since the start...

My preference is 1.

Frame size should not be an issue except for extensions which require
complete frames before they can be validated or manipulated; so any frame
limit should be part of the extension's specification - though this may
result in multiple ways to specify frame limits in different extensions.

Message size is more of an issue, but only for APIs that provide a message
based interface which can only present complete messages to the user of the
API; if such an API allowed the application layer to supply the message
buffers then the issue is passed on to the application level code.
Furthermore, message size limits are of no interest to APIs that provide a
stream based interface. As such, message size limits are an application
level concept and should not appear in this protocol spec. 

Possibly reworking section 1.2 (especially)

"A message is a complete unit of data at an application
   level, with the expectation that many or most applications
   implementing this protocol (such as web user agents) provide APIs in
   terms of sending and receiving messages.  The WebSocket message does
   not necessarily correspond to a particular network layer framing, as
   a fragmented message may be coalesced, or vice versa, e.g. by an
   intermediary."

And 

"The WebSocket protocol uses this framing so that specifications that
   use the WebSocket protocol can expose such connections using an
   event-based mechanism instead of requiring users of those
   specifications to implement buffering and piecing together of
   messages manually."

To acknowledge that handling of large messages is most likely best done with
a streaming interface rather than a message based interface (or removing the
suggestion that the websocket protocol handler will provide complete
messages) may mean that people wont forever be misunderstanding the design
intentions and suggesting message and/or frame limits or implementing the
protocol in such a way that they have inherent hidden limits.

Len Holgate
http://www.lenholgate.com
http://www.serverframework.com

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Fri, 12 Aug 2011 12:52:41 +0100

> in your implementation in which case/scenario do you send this error code,

We don't send it.  There's no valid use for it IMO.

However if the problem is solved just by Greg having it for his
extension to use, and without whole frame mtu, an error code reserved
for that is so cheap and self-contained, I'm happy to see him have it.

The vote though assumes that we have to be against the error code and
the frame mtu implementation, or for both; actually I don't care about
an error code getting reserved unnecessarily if it makes people happy,
and do care about whole frame mtu business being introduced.

-Andy

---

From: Tobias Oberstein <[email-masked]>
Date: Fri, 12 Aug 2011 05:12:16 -0700

> Of course. Which is the problem in indicating maximun frame size during the
> handshake negotiation?? that's in fact a *negotiation* process!

one problem is: value is fixed for lifetime of session .. can't be dynamic/adaptive

> The HTTP request could have a X-WebSocket-Frame-Limit header indicating
> the maximum frame size the client can receive, and the 101 response could
> contain same header indicating maximun frame size the server accepts. This
> is the way rather than ugly reactions after the limit has been reached.

---

From: "Simon Pieters" <[email-masked]>
Date: Fri, 12 Aug 2011 14:26:41 +0200

> and do care about whole frame mtu business being introduced.

FWIW I agree with Andy here. I don't mind keeping the error code.

-- 
Simon Pieters
Opera Software

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 12 Aug 2011 14:30:37 +0200

> one problem is: value is fixed for lifetime of session .. can't be dynamic/adaptive

Why should it be adaptative?

During the handshake the client tells its maximum frame size and the
server tells the same to the client. After that, they can use
whichever frame size (being always lower than the given limits by the
remote).

If you want such a feature to be re-negotiable after WS has been
established, then the design of the protocol is bad as the negotiation
takes places during the WS handshake (or should just "some" things be
negotiated during handshake and others later but using WS transport
protocol?).

Giving an example of a *correctly* designed protocol, in SIP a session
is established via an INVITE method in which session parameters
(audio, video, codecs....) are negotiated (an also in the response to
the INVITE). Later during the session (different RTP/MSRP streams) any
endpoint can re-nogitiate session parameters by sending a new INVITE
(AKA "re-INVITE") with the purpose of adding more streams (i.e. adding
video to an audio call), modifying existings streams or whatever. This
is robust and well designed.

But in WS it seems that any new (or not new) feature requires a hack.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Fri, 12 Aug 2011 13:41:37 +0100

> pass than to store a large block to disk, validate its signature then

Assuring the correctness of all the frames you received is not the same 
as assuring the correctness of the message, since especially with 
streamed content like a phone call I can truncate your message as a MITM 
by messing with the frame's unsigned FIN bit.  If the problem is missing 
frames you can't determine that except from the larger context of what 
the message ought to have been.

> be signed, not the message (which is the whole conversation).

I agree frame signature extension is pretty interesting.  But it has the 
above vulnerability if there is not also any message digest or signature 
that makes it alone not usable AFAICT.  That's why I say message 
signature is more useful, although I guess I could put it better.

> I have no issue with that. My concern is that if there is so much reluctance

OK that's good.

> heavily rely on the assumption above.

Well, I do hear you and was arguing that the 2^63 frame limit on the 
standard encourages that thinking, whereas a smaller frame limit, still 
enough to dilute the frame header overhead would enforce people to have 
to deal with the FIN framing stuff because they're seeing multiframe 
messages coming in the real world.

However trying to 'fix' that approach by defining a frame mtu that 
otherwise doesn't do anything useful seems to me it'll get worked around 
by stuff declaring frame limit of 2^63 if they want to take the attitude 
you're worrying about.  So it still doesn't serve any purpose.

> Except that application live for many years and are not changed every

Well, they would be deploying the new extension at that time. 
Otherwise, they continue to not care it doesn't support what the 
extension they don't have needs.

> But why are you constantly saying that setting a limit is "bloat" Andy ?

I do not hear about it doing anything useful, yet it wants to be added 
to the spec.

In that case, we are adding bloat to the spec.

> implementations will have to do ugly things for that.

We should push that code into extensions that want it, not the base spec 
that does not need it.  Otherwise we can flood the base spec with this 
and that which one day might be needed by a fantasy extension and there 
are zero people here who want to start down that road.

-Andy

---

From: Tobias Oberstein <[email-masked]>
Date: Fri, 12 Aug 2011 05:51:50 -0700

> Why should it be adaptative?

example: a muxing extension wants to dynamically adjust it's frame size
depending on up/downlink bandwidth/delay to target a tradeoff between
latency/efficiency.

would it be announced a fixed, max. frame size of N during handshake,
the range within which frame size can be varied is limited by N, which might be
smaller than what it wants to do at some point during session lifetime.

the extension might simply want to adapt the frame size it fragments any
message into during sending based on it's estimate of current
uplink bandwidth/delay.

when the decision about current send frame size is always made by the sending
peer, there is no need to communicate that current frame size between the
peers, nor is there a use of a "max. frame size".

if the decision is made by server for itself and client, the server will need to 
communicate the current frame size the client is supposed to use during
sending to the client.

note, that this "current frame size" is not a max. frame size, but just the frame
size a peer is fragmenting any message into

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 12 Aug 2011 15:12:14 +0200

> depending on up/downlink bandwidth/delay to target a tradeoff between

This stuf should be handled at much more lower layer, within WS
protocol as *pure* transport protocol. Letting "extensions" to change
such size is, IMHO, a bad design. I don't think that current spec has
layers well defined.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Fri, 12 Aug 2011 14:43:46 +0100

On 08/12/2011 02:12 PM, Somebody in the thread at some point said:
> 2011/8/12 Tobias Oberstein<[email-masked]>:

You have to bear in mind the extensions negotiate that they have a peer 
implementation of the same extension to talk to.  So your client 
"sign-frames-1.0" extension will always have a compliant 
"sign-frames-1.0" peer on the server side that "speaks its language".

In the case that there's a sequence of extensions operating, the 
ordering of the sequence of extensions can be important that they match.

By leaving it to the extension "layer", each extension peer pair which 
has a fragment size constraint can explicitly or implicitly configure 
the fragmentation to happen at that layer, also any other constraints or 
special needs they have can privately occur at that layer.  Nobody else 
needs to know in terms of other extensions or the base protocol.

As a side effect, that same-extenion - same-extension constraint stuff 
can be adaptive, but it's not really the reason to put it there.

Since in the case of buffering frames it's only really the signing frame 
extension that has a legit reason to do it IMO, it's righteous that the 
code to deal with it should fall inside that extension.

-Andy

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 12 Aug 2011 15:58:54 +0200

2011/8/12 "Andy Green (林安廸)" <[email-masked]>:
> By leaving it to the extension "layer", each extension peer pair which has a
> fragment size constraint can explicitly or implicitly configure the
> fragmentation to happen at that layer, also any other constraints or special
> needs they have can privately occur at that layer.  Nobody else needs to
> know in terms of other extensions or the base protocol.

Ok, can my HTTP client (a script in Ruby/Python/whatever) decide the
payload size of the Ethernet message sent to my local network?

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Fri, 12 Aug 2011 15:15:54 +0100

On 08/12/2011 02:58 PM, Somebody in the thread at some point said:
> 2011/8/12 "Andy Green (林安廸)"<[email-masked]>:

It can certainly decide in what size packets to issue stuff.  What 
happens after that, before and after your Ethernet adapter in terms of 
packet coalescing or fragmentation is out of its control.

That's the same deal here, except that intermediaries should understand 
the extensions involved (and follow any private mtu negotiation) if 
they're going to change fragment sizes.

-Andy

---

From: Tobias Oberstein <[email-masked]>
Date: Fri, 12 Aug 2011 07:17:55 -0700

> > the fragmentation to happen at that layer, also any other constraints
> > or special needs they have can privately occur at that layer.  Nobody
> > else needs to know in terms of other extensions or the base protocol.
> 

No. So what?

Can the torrent protocol transfer fragments of a file? You bet.

Does that mean torrent is "badly designed" because it uses fragments, where
we already have fragments on the TCP layer, packets on IP, frames on Ethernet, ..?

The concept of "chunk" exists on multiple layers and each layer may have it's
own "chunk" concept, use and rules for it.

Onions are good;)

---

From: Bruce Atherton <[email-masked]>
Date: Fri, 12 Aug 2011 08:44:51 -0700

> entire frames or that have latency concerns can restrict frame size.

Sounds like you have a good general purpose solution. I started down 
that road myself, making my solution work both streamily and not in a 
variety of ways. Then I realized that I was solving a problem that I did 
not have because my implementation is not intended to be a general one, 
and ripped much of it out.

> Unless you are running extensions
> that need to process entire frames,

That's part of it. There are three levels of extensions: stream (which 
will hopefully only have deflate-stream as an example), frame, and 
message. Frame-based extensions are useful for communicating between the 
Websocket implementations, and sometimes it can be useful to have them 
work with the payload. Willy gave one example, I gave another concerning 
changing the opcode based on the contents of the payload. Yet another 
would be modifying the payload in some way and thus needing to adjust 
the payload length. I don't currently have any of these at the moment, 
but I am thinking about them.

The response I have gotten is that such extensions are invalid or that 
they should set their own length limits. Why they are invalid is beyond 
me. Since they have no length limits other than what the underlying 
implementation provides, it makes no sense for them to come up with an 
arbitrary limit, particularly since the implementation may change the 
amount it buffers without mentioning it to the extension.

>   you should be able to process

Thanks, but I've been down this road a few times and it doesn't work 
out, because there is always a workaround for any objection.

I start out with a nice clean design that meets my needs but for the 
lack of being able to declare a maximum frame limit. Someone suggests a 
different implementation, like "why don't you aggregate the data from 
smaller buffers into frames?" I come up with reasons I don't like it, 
for example I don't want to copy data multiple times before reaching the 
message level. I'll then get responses like, "You don't have to worry, 
the cost is negligible" and "You don't have to copy, you can maintain 
the frame as a collection of buffers" and "STREAMIFY". I'll respond to 
each suggestion. If I maintain the buffers as elements of a frame, I 
bloat the amount of data needed to store the frame. If I streamify I 
lose the ability to model the spec. The streaming guys continue to 
insist that streaming is the one true implementation, completely 
ignoring anything I've written. Meanwhile, I look back and see my nice 
simple design disappearing over the horizon, all for a lack of a maximum 
frame size that would hurt nobody.

---

From: Scott Ferguson <[email-masked]>
Date: Fri, 12 Aug 2011 08:53:23 -0700

> 1. Remove error code 1004 "Frame Too Large".

1) remove the error code (more specifically, replace with "Message Too 
Large")

No compelling use-cases have been given for the max-frame-size. The main 
result would be to allow and encourage limited implementations. Given 
past headaches in working around broken peers, it's a bad idea for the 
spec to actively encourage them.

Replacing it with a "Message Too Large" would be a good idea, primarily 
as an application-layer error. Applications need a way of rejecting 
large messages. (Some WebSocket-layer implementations like the browser 
API might need to reject large messages.)

-- Scott

---

From: Scott Ferguson <[email-masked]>
Date: Fri, 12 Aug 2011 09:10:07 -0700

On 08/12/2011 03:16 AM, Willy Tarreau wrote:
> to design those extensions.

Well, my opposition is because it's adding complexity primarily to 
support broken receivers, The brittleness of a broken receiver which 
relies on a sender to work around its limitations is a bad thing. Given 
past pain in dealing with broken peers, I don't see the value of 
encouraging it in the spec.

Plus, the max-frame-size makes the whole sendfile() argument for the 
massive frames pointless, and if we're really limiting frame size, we 
should have used a fixed 16-bit frame length. :)

> My fears now are that 1 frame == 1 message for

Huh?

Dynamic and stream senders send messages in multiple frames. For 
example, My Resin implementation uses a fixed 8k sending buffer while 
accepting any size frame. Any stream-based implementation should have 
similar behavior.

Plus, the complexity of MUX is far beyond just the frame-size. Adding a 
frame-size doesn't get you anywhere near implementing MUX.

-- Scott

---

From: Bruce Atherton <[email-masked]>
Date: Fri, 12 Aug 2011 09:47:21 -0700

> depending on up/downlink bandwidth/delay to target a tradeoff between

Size limits on extensions have nothing to do with limitations of the 
underlying implementation. Why do people keep conflating the two issues?

Let me try an analogy. A plane has a maximum cargo capacity. A 
particular airline may have a calculation setting maximum cargo based on 
calculations to minimize fuel costs. If this is less than the plane's 
capacity then it acts as the maximum amount of cargo that is carried. 
But if the calculation comes out to more than the plane's maximum, the 
plane cannot suddenly carry more cargo. The maximum is set by the lesser 
of the two values.

This analogy can be adaptive as well. Perhaps the calculation changes 
based on whether there is a headwind. That changes one of the values, 
but the maximum that the plane can carry stays constant.

I know, I know. Streamify and the plane can carry an unlimited cargo. 
That is a different issue, and one that I am really tired of hearing 
because I don't want the plane to carry an unlimited cargo. The point is 
that a maximum frame size does not stop a Mux extension from adjusting 
its frame size based on circumstances, within the limits of the 
underlying implementation.

---

From: Bruce Atherton <[email-masked]>
Date: Fri, 12 Aug 2011 10:01:39 -0700

> sizes from 64 bytes - 8MB)

Ouch. So all that work I went to having separate threads between layers 
so that control frames could sneak in in the middle of a longer message 
was totally wasted?

How very sad.

---

From: Tobias Oberstein <[email-masked]>
Date: Fri, 12 Aug 2011 10:01:21 -0700

> don't want the plane to carry an unlimited cargo. The point is that a maximum
> frame size does not stop a Mux extension from adjusting its frame size based
> on circumstances, within the limits of the underlying implementation.

Don't know if it makes sense, but I will try once more, because I think we should at least
listen to each other on this list.

Implementation A has a fixed, built-in max. frame size of 64k.
It wants to announce that, and make any other implementation obey to that max.

Implementation B has no limit on frame size, but wants to scale it dynamically
within - lets say 8k - 1M. It wants to do that when it speaks some special extension.

What happens is: B can only use range 8k - 64k, since A imposes the max. of 64k.

What if implementation C has a limit of 125 bytes, because that is the max. payload
of control messages, and C decided for whatever reasons, that it would be simpler
to make that the limit.

Now, A, B and C will need to obey 125.

How can this be resolved?

---

From: Bruce Atherton <[email-masked]>
Date: Fri, 12 Aug 2011 11:35:20 -0700

> listen to each other on this list.

I agree that there is far too little of that going on.

> What happens is: B can only use range 8k - 64k, since A imposes the max. of 64k.

Almost. Remember that we are talking about maximum frame size, not 
actual frame size. B can use any frame size 0K to 64K.

But let's change the scenario a bit to match what you are saying. Let us 
say that extension B requires sending a minimum of 8K in each frame. 
Then you would be right that the range will always be 8K to 64K.

> How can this be resolved?

Implementation C is not compatible with extension B because it does not 
provide the minimum frame size that B requires. So when extensions are 
negotiated, A would ask to use extension B and C would refuse to use it. 
Then communication between the two would either continue without it, or 
the application level in A may shut down the connection if it required 
the use of B.

---

From: Patrick McManus <[email-masked]>
Date: Fri, 12 Aug 2011 14:40:01 -0400

> so that control frames could sneak in in the middle of a longer message 

If there was a use case we wanted to support that required fragmenting
on transmit, then we would fragment. I expect mux will be that case. It
is a mistake to read Tobias's comments as a statement that firefox won't
ever send fragments.

---

From: Tobias Oberstein <[email-masked]>
Date: Fri, 12 Aug 2011 12:42:01 -0700

> Then you would be right that the range will always be 8K to 64K.

Ok, I was a bit sloppy with the B example. B is supposed to be able to
send and receive frames of any size in the range 0 - 2^63.

However, when it sends a _message_ of lets say 1M, it wants to fragment that
message into frames of length C, where it wants C depend on network
conditions, even varied over time, and varied in a range say R = 8k - 1M.

In this case, when B talks to A, that range R will clamped to the max.
that A has announced, and when that is less than 1M, then B is limited
in comparison to what it would do when talking to another peer of type B.

==

In the spirit of listen, two questions rgd. your implementation.

a) Suppose you want to send a 1M message. In what frame sizes do you
fragment?

b) Suppose you talk to another peer that has a very small max. frame size
limit (say 125 .. or even 1 byte). What will be the effect on your
implementation (i.e. in terms of efficiency)?

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Sat, 13 Aug 2011 09:37:36 +0100

> underlying implementation. Why do people keep conflating the two issues?

Because emails keep coming which look like they confused the general and 
common limitation on message length found in every real browser, and 
frame processing.  We are after all talking about an error calling 
itself *frame* too big, which is not possible -->

Because frame processing is performed packet by packet, with the single 
exception of this proposed frame signing extension, actually the frame 
layer just deals with a packet at a time.

In the case of the exception, since he always has a peer extension of 
the same type negotiated, that peer extension can fragment frames to 
satisfy the extension buffering limitation.

So, there's *nothing* in the frame layer that can run up against the 
concept of "frame too big", it just means more packets can be expected 
to continue the next frame, a packet at a time.

And yet, we discuss an error and a whole proposed mtu workaround 
mechanism for the base spec dealing with the concept of "*frame* too big".

"Message too big" is a genuine issue that can only be dealt with by 
hanging up the connection.  Using the controlled close stuff with a 
reason code sounds like what should happen then.  And nothing stops the 
peer deciding that the message has gotten too big dynamically, eg, on 
malloc fail for the next chunk or some variable limit without any 
negotiation action needed.  That reflects the reality that the peer may 
indeed dynamically find itself in an OOM situation and have to close 
what it otherwise intended to accept randomly.

-Andy

---

From: Bruce Atherton <[email-masked]>
Date: Mon, 15 Aug 2011 15:58:26 -0700

> is a mistake to read Tobias's comments as a statement that firefox won't

Thanks for the clarification.

Can I ask for a bit more info, though?

1) If you received a close message with a "Frame Too Large" status, 
would the Firefox implementation ever retry the connection with a 
smaller frame size?

2) If the spec included the language about maximum frame size that is 
proposed (the "MAY" and "SHOULD NOT"), would Firefox  honour a request 
for a  particular maximum frame size?

---

From: Bruce Atherton <[email-masked]>
Date: Mon, 15 Aug 2011 16:15:33 -0700

On 13/08/2011 1:37 AM, "Andy Green (林安廸)" wrote:
> Because emails keep coming which look like they confused the general 
> and common limitation on message length found in every real browser, 
> and frame processing.  We are after all talking about an error calling 

You keep saying this, but that doesn't make it any less incorrect. A 
frame can be too large for any number of reasons: latency, denial of 
service limitations, etc. Requirements for a particular situation trump 
the theoretical limits of a standard every time. So long as the standard 
does not require that all frame sizes be handled in the same way, it is 
a legitimate implementation decision to choose to handle a particular 
frame size by closing the connection. That behaviour is not broken nor 
bogus, it is completely conforming to the specification. Point me to ANY 
portion of the specification that prohibits it.

And if it is conforming, there are going to be more people than me 
choosing to implement that way. And their implementations will be just 
as correct as yours is.

>
> Because frame processing is performed packet by packet, with the 
> single exception of this proposed frame signing extension, actually 
> the frame layer just deals with a packet at a time.

It can do. It doesn't have to. There is nothing in the specification 
that requires that behaviour. As I have said many times before, and say 
here for the last time.

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Tue, 16 Aug 2011 06:43:50 +0100

> frame can be too large for any number of reasons: latency, denial of

Evidently the "any number of reasons" you could come up with is two. 
However if we think about what they mean to a receiver on a duplex 
transport, they are not convincing as reasons for the receiving peer to 
object to with "frame too big" concept.

The problems with latency if the transmitting peer sends a big frame are 
all at the sender.  The receiver can asynchronously send frames any 
time; the error you like to send relies on that.  So the only latency 
that gets stretched is that of the sender being able to send a new frame 
while he's still going on with the old one.

Likewise the service being denied is the ability of the sender to send 
other frames, the receiver can still send frames, accept new connections 
etc.  If that's what you want to call a DoS then you can do the same 
with telling a 2-byte frame is coming and then send one byte of payload 
and go silent.  "DoS" has nothing to do with "frame too long".

Actually, you need to manage all three of these cases ("latency", "long 
frame DoS", "never complete DoS") with timeouts for the things that 
care, if anything cares.  Making an arbitrary frame length limit (which 
requires you to assess connection bandwidth to come up with a reasonable 
estimate) and killing the connection does not actually address latency 
that can be caused other ways too.  Whereas timeouts do solve all kinds 
of latency and do not require guessing a frame limit and having a "frame 
too long" error.  Further, the timeouts only operate if something 
actually cares, and let the sender babble on optimally if that's OK for 
that protocol / situation and the data is wanted.

> portion of the specification that prohibits it.

These arguments are about what should appear in and should be removed 
from the specification itself.  Many times on the list we detected the 
spec is broken or not self-consistent at the moment.  So it's better if 
we use logic, clear descriptions, honest argumentation without 
handwaving to figure out what's good and bad as well as what the spec 
seems to be intending.

> that requires that behaviour. As I have said many times before, and say

That's right but it brings us full circle, you are standing up for the 
right to make a crappy and fragile implementation when the good one is 
simple and there's no reason not to use it.

If that was my argument, I wouldn't wanna repeat it either ^^

-Andy

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 16 Aug 2011 16:09:58 +1000

On 16 August 2011 15:43, "Andy Green (林安廸)" <[email-masked]> wrote:
> That's right but it brings us full circle, you are standing up for the right
> to make a crappy and fragile implementation when the good one is simple and
> there's no reason not to use it.
>

C'mon Andy - keep the scorn out of the group.  It's not helpful.
I'm mostly convinced that we don't need frame-too-long, but I have to
say your attitude is not helping me to that conclusion.

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Tue, 16 Aug 2011 07:53:01 +0100

On 08/16/2011 07:09 AM, Somebody in the thread at some point said:
> On 16 August 2011 15:43, "Andy Green (林安廸)"<[email-masked]>  wrote:

> say your attitude is not helping me to that conclusion.

What you take as 'scorn' is actually a fair summary of the argument 
given (replaced from your snipping above); and I think you're 
misinformed if you believe I'm posting on this list to keep you happy.

-Andy

---

From: Bruce Atherton <[email-masked]>
Date: Tue, 16 Aug 2011 10:08:50 -0700

On 15/08/2011 10:43 PM, "Andy Green (林安廸)" wrote:
> transport, they are not convincing as reasons for the receiving peer 
> to object to with "frame too big" concept.

Your arguments seem to be descending into rhetoric. It is a shame that 
rhetoric and reality are orthogonal.

Since I have mentioned other examples before in this and other threads, 
your statement is demonstrably false. Let me give an example I have 
mentioned before.

It is usually a good implementation decision to model a protocol. The 
Websockets specification talks about connections and frames and messages 
and URLs. It does not talk about buffers or streams between layers.

So it is a reasonable expectation that many people will implement 
Websockets with a Connection class and a Frame class and a Message class 
and a URL class. Given those classes, what is the simplest thing to do 
when gathering data from the connection class? Buffer within the Frame.

So following the good design principles of modelling the spec and doing 
the simplest thing possible, it is highly likely that many people will 
opt for a design similar to mine with a limited maximum frame size.

> spec is broken or not self-consistent at the moment.  So it's better 
> if we use logic, clear descriptions, honest argumentation without 

It is hard to understand what idea you intended to convey with this 
obfuscation. But let me go back to the point.

The reason we have a specification is to allow compliant implementations 
to communicate. It is designed to facilitate interoperability. As I have 
described, there are going to be many people who choose to implement 
with a maximum frame size. These implementations will be in full 
conformance with the spec. The spec even goes so far as to explicitly 
allow closing the connection if a frame is too large, and even you do 
not disagree with that remaining. So these people will fully expect 
their conforming implementations to work with other implementations, 
only to find that there is a hidden gotcha not mentioned in the spec: 
that if another implementation sends a larger frame size there will be 
no way to interoperate with them.

> simple and there's no reason not to use it.

No. I am standing up for interoperability. Even if you get me to agree 
that it is crappy and fragile, I am just one person. This is not about 
your implementation or mine, it is about all the implementations that 
are going to be out there in the future. I want to make everyone able to 
communicate as simply as possible, and the cheapest, easiest way to 
accomplish that is with a maximum frame size.

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Tue, 16 Aug 2011 18:24:24 +0100

> your statement is demonstrably false. Let me give an example I have

Sorry you quoted two statements: that you only gave two reasons 
inbetween "any number of reasons" and "etc", and that sender can't block 
a receiver from sending... which one are you telling me is "false"?  I'm 
pretty sure they are both accurate.

> opt for a design similar to mine with a limited maximum frame size.

Sorry you're talking about "good design principles" "usually a good 
implementation decision" here but you promised to "demonstrate" the 
falsity of something I said?

> obfuscation. But let me go back to the point.

I see.  Then let me explain it again ^^

What we're discussing is whether to add or remove stuff from the 
specification.  So the current state of the specification, especially in 
the areas we are talking about changing is not a good argument without 
backing it up with some reasoning.  Otherwise we would go around in 
circles every time we changed the spec since some pieces sometimes 
conflict others.

In this case I mean "Point me to ANY portion of the specification that 
prohibits it" is not really convincing reason 'for' something when we're 
talking about changing the specification in this area maybe to outlaw it.

> with a maximum frame size. These implementations will be in full

Just curious, would anyone planning to have their implementation break 
at the frame layer based on frame size like to participate in a straw 
poll here?

> no way to interoperate with them.

It's as I wrote, it would be a "crappy and fragile" implementation and 
you want the spec bent out of shape to service it, when no good reason 
has been brought forward to do it.

> accomplish that is with a maximum frame size.

Earlier you were going on about rhetoric...

-Andy

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 16 Aug 2011 11:41:21 -0600

<hat type='AD'/>

> art of argumentation" I referred to.  Peter, did you come up with that

You seem to think that the suggested choice was malicious. You might
want to re-acquaint yourself with Hanlon's Razor.

In fact the chairs and I worked out that text and I posted it to the
list in an effort to further focus the discussion.

Folks, we're here to work together toward consensus. That means
listening to each other, understanding the issues that people raise,
trying to achieve a common view of the technology we're developing, and
formulating specification text that captures the consensus. Having just
reviewed the thread to date, I must say it seems that we're not quite
there yet.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 16 Aug 2011 10:43:41 -0700

> and a URL class. Given those classes, what is the simplest thing to do when
> gathering data from the connection class? Buffer within the Frame.
> 
> So following the good design principles of modelling the spec and doing the
> simplest thing possible, it is highly likely that many people will opt for a design
> similar to mine with a limited maximum frame size.

Since you are so bold, my answer is also:

No, it is not "reasonable" and not "highly likely".

No one would opt for a broken design that follows a naive schoolbook
OO approach, introducing arbitrary hard-coded limits.

Again: you can have a streaming implementation, that by default buffers
up to frames, and further to messages. Then, your users have the convenience
of a message-based API for general use, but can override the default buffering
to get at the frame or streaming level.

> your implementation or mine, it is about all the implementations that

It is about yours. As it stands, Andy's and mine can talk to each other
as well as Firefox and Chrome ... using any frame size up to 2^63.

> accomplish that is with a maximum frame size.

No, it's not. The cheapest way is: fix your broken implementation.

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Tue, 16 Aug 2011 18:51:54 +0100

> want to re-acquaint yourself with Hanlon's Razor.

Actually I asked if you came up with that either-or yourself, because 
actually the two options are not mutually exclusive and they were formed 
from Greg's, well, observation.

> list in an effort to further focus the discussion.

In that case you are right, we are not quite there yet.

> reviewed the thread to date, I must say it seems that we're not quite

I've been on the list a while and I most time "consensus" is really elusive.

Just look at this consensus call, it's split about 50:50.  Other times 
there has been a "raging consensus" about masking, were you around 
during the "how much crypto is enough" wars?  I guess some issues must 
be "is black black?" type questions but most of them are not.

When they are not, the issues have to be argued through.  If the people 
who don't agree with the people who are speaking loudest, or speaking 
from the centre of gravity of the group stay silent, wrong decisions 
will be made.

In short you can't arrive at an accurate consensus without the 
argumentation step.  You would be better off focused on keeping the 
argument honest rather than trying to pretend it is not necessary.

-Andy

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 16 Aug 2011 11:58:53 -0600

<hat type='AD'/>

> I've been on the list a while and I most time "consensus" is really

Sadly, yes. The issues are difficult and often the discussion has been
contentious, but we've managed to hold together this far. I'd hate to
see us break apart over something as minor as error code 1004. :)

> argument honest rather than trying to pretend it is not necessary.

Me specifically, or everyone?

I completely agree that each one of us has a responsibility to
understand the issues that other people raise here, and that people who
raise such issues need to clearly describe the issue and explain the
reasons for considering the issue. Ideally, people would propose text
that can be incorporated into the spec. Unfortunately, only Len Holgate
has proposed text of late. Replies to his message might help us move
forward.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Tue, 16 Aug 2011 19:47:04 +0100

> see us break apart over something as minor as error code 1004. :)

That error code's welcome to stay by me, although I have explained at 
length why it is useless.  It's the rest of the frame mtu stuff trying 
to ride into the spec on its coat-tails that is really objectionable.

> Me specifically, or everyone?

All of us are in the same boat.

> has proposed text of late. Replies to his message might help us move

However because of what has been proposed by others, I am trying to keep 
stuff out of the spec and like it as it is in this area.  It's not going 
to yield new text then but try to keep bloat out of the spec.  This is 
also a vital part of "moving forward".

-Andy

---

From: Bruce Atherton <[email-masked]>
Date: Tue, 16 Aug 2011 11:47:38 -0700

> No, it is not "reasonable" and not "highly likely".

You may be right. I could be completely wrong about people using this 
design. What would be the impact of a maximum frame size in that case?

Since using max frame size would be completely optional, that would mean 
no one would ever send out the header. And that would mean many 
implementations would never bother coding to react to it.

Some might. Those implementations would then have a tested mechanism for 
fragmenting messages, which would come in handy when adopting an 
extension that required reduced frame sizes like Heartbeat or Mux.

Now consider the consequences if you are wrong and there are a number of 
implementations with a maximum frame size but no way to indicate it. You 
would have situations where everything seemed to be compatible, messages 
flow smoothly from one end to another, when suddenly, unexpectedly 
things fail. The code may already have deployed into production. It may 
be extremely rare to fail, and depending on circumstances it may be 
difficult to figure out why things fail. Or you may have purchased a 
product with a WS implementation that completely fails to work with 
yours. You may know why, but it doesn't help you in getting things to 
work properly.

> No, it's not. The cheapest way is: fix your broken implementation.

Given the behaviour of Firefox, if there is no maximum frame size 
communication in the spec then I will change my implementation because 
requirements will have changed. I'll be one of the lucky ones, because I 
will be aware of the issue.

In the interests of interoperability, if there is no ability to 
communicate a maximum frame size then I suggest that the spec make it 
explicit that an implementation that refuses a frame for being too large 
is non-compliant. At least give people a fighting chance.

---

From: Bruce Atherton <[email-masked]>
Date: Tue, 16 Aug 2011 14:30:39 -0700

On 15/08/2011 10:43 PM, "Andy Green (林安廸)" wrote:
>
> The problems with latency if the transmitting peer sends a big frame 
> are all at the sender.  The receiver can asynchronously send frames 
> any time; the error you like to send relies on that.  So the only 
> latency that gets stretched is that of the sender being able to send a 
> new frame while he's still going on with the old one.

No. The issue for me concerning latency is implementing a heartbeat.

I know, I've heard it before on this list. "That's an extension issue. 
Make the extension define the maximum frame size." There is actually an 
argument that can be made for this. The heartbeat extension knows the 
timing of the heartbeat and can adjust the frame size accordingly.

There is a downside, though. It means more coupling between the 
extension and the core implementation. Right now each part of my 
implementation passes information up to the next through a chain of 
responsibility with a single method call in the interface. That API 
becomes more complicated if the extension has to be able to configure 
the framing layer (which it previously had no direct knowledge of) as well.

Ordinarily protocol specifications give people the right to decide on 
which tradeoffs make sense to them. In my opinion, a protocol which is 
specified in such a way as to severely limit the options for 
implementers is overly constrained. Ironically, in this case the 
over-constraint comes from NOT specifying something that would allow for 
a greater variety of implementations to work together.

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Wed, 17 Aug 2011 07:01:25 +0100

On 08/16/2011 10:30 PM, Somebody in the thread at some point said:
>> are all at the sender. The receiver can asynchronously send frames any
>> that gets stretched is that of the sender being able to send a new
>> frame while he's still going on with the old one.
>

Alright, knowing your server is dead is a reasonable thing to want.

There's an issue with this scheme you also have to face, what 
constitutes a reasonable max "hogging the channel" fragment size is 
dependent on the minimum bandwidth that channel can be expected to 
experience.  That can vary widely depending on the physical transport 
and even dynamically.

> a greater variety of implementations to work together.

It is possible to exclude too much.  But another way to break it is to 
be too lassaiz-faire and include anything until the simplest explanation 
of the protocol must include bandwidth estimation, negotiation of frame 
sizes and so on which actually can be eliminated.  Actually really good 
protocols just do the needful once it's understood what that minimum set is.

"Heartbeat" was discussed in some good threads some months ago under the 
name "keep alive".  The proposal there was to allow the client or server 
to shutdown a link if the peer had actually gone away, to be able to 
tell the difference between your conversation partner falling silent for 
a bit or having dropped dead.

During that discussion, I realized (I think proposed but can't remember) 
that incoming traffic of any kind is evidence of life, meaning you do 
not need to issue a PING to stimulate incoming traffic until you did not 
hear anything from the remote side for some threshold time.  If you take 
that approach, it's true you are open to taking endless babble as 
evidence of all being well (same as with actual heartbeat the heartbeat 
extension may respond while the actual server side is dead), but 
otherwise because you regard any incoming traffic as telling you about 
remote peer state you can detect if the server is dead just fine without 
having to get into frame fragmentation issues at all.

-Andy

---

From: "Len Holgate" <[email-masked]>
Date: Wed, 17 Aug 2011 12:51:01 +0100

> also a vital part of "moving forward".

Fair enough, except that what I was proposing was a change to make it
clearer that a message based api is not the only way. The existing wording
tends to lead one down a path towards a 'whole message' API which is
probably what leads people to think in terms of frame and message limits. 

If you want to "keep stuff out of the spec" then why not simply remove the
comments in section 1.2 that imply that a message based approach is
preferable.

For your convenience, in my previous message I suggested....

And 

Len Holgate

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Wed, 17 Aug 2011 13:12:54 +0100

> probably what leads people to think in terms of frame and message limits.

Sure.

> If you want to "keep stuff out of the spec" then why not simply remove the

Well, keep needless stuff out of the spec at least.

>     specifications to implement buffering and piecing together of

The first one seems fine but the second one I am not sure it is 
capturing the deal with frames.

AIUI frames are there to compartmentalize payload more or less 
arbitrarily in exactly the same way as TCP/IP packets do.  The frames 
being of finite duration allows sharing of the connection resource for 
other different frame content by interleaving.

These events you're talking about can equally well be generated by every 
packetload of message payload coming if you like, it does not require 
framing to do it.

How about this kind of thing instead:

"The websocket protocol uses framing only to allow interleaving of 
control or other extended frames to share the same connection with 
finite latency.  Similar to how multiple TCP/IP packets may contain 
streaming payload but the size of packets or how they are fragmented is 
not predictable and shouldn't be relied on as being received in the same 
units it was sent, payload frame sizes are subject to manipulation by 
intermediaries and should just be regarded as more message payload, not 
as containing payload with meaningful boundaries at the start and end of 
the frame."

> protocol in such a way that they have inherent hidden limits.

Right.  Maybe it's a good step in that direction to explain received 
frame extents have no inherent meaning other than how many bytes follow 
and may be unrelated to transmitted frame extents, particularly may be 
bigger.

-Andy

---

From: Bruce Atherton <[email-masked]>
Date: Wed, 17 Aug 2011 08:59:01 -0700

> experience.  That can vary widely depending on the physical transport 

Believe me, I know. The application, which currently uses two web 
servers making requests to one another, is already deployed to hundreds 
of customers networks.

> be too lassaiz-faire and include anything until the simplest 
> explanation of the protocol must include bandwidth estimation, 
> negotiation of frame sizes and so on which actually can be 
> eliminated.  Actually really good protocols just do the needful once 
> it's understood what that minimum set is.

I don't disagree at all with this sentiment that the protocol should be 
kept as simple as possible, but I would add the clause "but no simpler".

I get that adding a maximum frame size is unnecessary for you. I get 
that everything can be implemented using the tradeoffs you have chosen. 
I even get that those choices are good ones for generalized solutions, 
as proven by the fact that a number of people have opted for the same.

But try to understand that not everyone is writing a generalized 
solution. For those people other tradeoffs can be more desirable. And in 
those situations, a maximum frame size can greatly aid in 
interoperability. So even if you don't need it, try to allow for the 
fact that others may.

>
> During that discussion, I realized (I think proposed but can't 
> remember) that incoming traffic of any kind is evidence of life, 
> meaning you do not need to issue a PING to stimulate incoming traffic 
> until you did not hear anything from the remote side for some 
> threshold time.

That can be a good solution in some circumstances. It depends on 
requirements.

In my situation, a heartbeat from each instance is rebroadcast to a 
cluster of nodes, as well as stored in a database so that a UI can 
display the time of the last heartbeat. If I did that every time a byte 
was received, the traffic would be ridiculous. If I had a disconnect 
between the original data and the notifications to the nodes and 
database then I would have more complication, more code, more places for 
things to fail. Since fault tolerance and high availability are 
critical, these would be bad choices in my case.

But the point is there are more requirements out there than are dreamt 
of in your philosophy. A good protocol design would allow each to make 
their own decisions while still allowing interoperability.

---

From: Scott Ferguson <[email-masked]>
Date: Wed, 17 Aug 2011 09:08:18 -0700

On 08/17/2011 05:12 AM, "Andy Green (林安廸)" wrote:
> streaming payload but the size of packets or how they are fragmented 
> is not predictable and shouldn't be relied on as being received in the 
> same units it was sent, payload frame sizes are subject to 
> manipulation by intermediaries and should just be regarded as more 
> message payload, not as containing payload with meaningful boundaries 
> at the start and end of the frame."

Ulg.

The only original purpose of frames was to avoid buffering an entire 
message before sending it. Frames were just for the purposes of 
fragmentation to allow for fixed-buffer senders, particularly for 
dynamic messages like serializing JSON.

The interleaving capability is a side-benefit of the actual frame 
design, but it's not the purpose of the framing. (Or at least not the 
original purpose.) Mux is important, but it's not the reason for framing.

At its heart, WebSockets is still about sending messages.

-- Scott

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Wed, 17 Aug 2011 22:15:55 +0100

> dynamic messages like serializing JSON.

Fixed buffer senders continue to be fine.

> At its heart, WebSockets is still about sending messages.

Sure but you can't send a message without sending at least one frame.

Whatever the "original purpose" the issue Len and others have pointed 
out is we seem to have engendered implementations which are limited to 
fit one messages in one frame because of the potential for huge frame 
sizes, or fragile implementations which believe they should buffer a 
frame because it has some unitary meaning when they actually can't bank 
on that.

Then we get these threads with people being defensive about their 
implementation.

It's going to help if we sum up the reasoning and a good way for 
implementers to understand what the framing is doing and how to treat it 
I think.

If there's anything wrong or can be improved with what I suggested I'm 
glad to hear it.

-Andy

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 22 Aug 2011 11:01:28 +1000

> send frames larger than the maximum frame size signalled by its peer.

I think this ballet has got a little lost.   However, in case anybody
is still actually counting, I'd like to change my vote from #2 to an
unconditional #1 - remove the frame-too-large close code.

In addition to that, I think that we should also:
a) add a message-too-large close code
b) define some optional headers (or control packet) to give
advice/hints on a hop-by-hop basis of the maximal frame size that
should be sent and the preferred frame size to be received

But regardless for support for these proposals, I think we should
remove frame-too-large.

regards

---

From: Sylvain Hellegouarch <[email-masked]>
Date: Mon, 22 Aug 2011 09:49:36 +0200

> should be sent and the preferred frame size to be received
I am not in favour of #2 for the added complexity. Since the initial
motivation for framing was to allow bytes to be sent as soon as available,
we may end up in a situation where all the bytes have been sent in various
frames before the endpoint had a chance to receive and process them all. If
it fails because a frame is too large and indicates to the emitter the
max-size, it won't be that useful since this will mean the original sender
have to re-send the whole message which it may not have at hand any longer.
Since frames are not identified, there's no way to send from the frame that
failed either way. Therefore if a message must be sent again in its
entirety, we might as well not bother with frame size.

>From a transport protocol perspective, #2 makes sense but from an
application protocol perspective, I'd say it makes things more complex. As I
understand it, WS is still considered as an application protocol right?

-- 
- Sylvain
http://www.defuze.org
http://twitter.com/lawouach

---

From: Gabriel Montenegro <[email-masked]>
Date: Fri, 26 Aug 2011 18:57:06 +0000

Sal and I, in consultation with the editors and Peter have arrived at a Consensus call on #1:

	- Remove error code 1004 "Frame Too Large".

The editors will remove use of this code in draft 13. At the same time, they will add clarifying text. 

We note there is interest and the possibility of introducing related functionality, error codes, etc., in the future, perhaps in conjunction with multiplexing support.

Thanks,

Gabriel

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Peter Saint-Andre
> Sent: 11 August, 2011 09:49
> To: Greg Wilkins
> Cc: [email-masked]
> Subject: [hybi] CONSENSUS CALL (was:Re: WebSocket Frame Size and latency)
> 
> This is your Area Director speaking. I would very much like to issue an IESG ballot
> for our base document in the near future. The WG needs to decide this matter in
> order for us to proceed. See below for details.
> 
> >> The guys who like frame mtu plan to create implementations which
> >> break if you send them legal frames bigger than some size they liked
> >> when they wrote the code. So they will create an implementation which
> cannot cope with frames above a certain size.
> >
> Apparently there is a disconnect in the specification: as currently written, an
> implementation can reject a frame because it is too big (error code 1004), but it
> can't signal its maximum frame size.
> 
> Therefore, as your Area Director, I am requesting a consensus call on the
> following alternative.
> 
> 2. Retain error code 1004, add an OPTIONAL way for an entity to signal its
> maximum frame size, and specify that an implementation SHOULD NOT send
> frames larger than the maximum frame size signalled by its peer.
> 
> Please reply to this consensus call by the close of business next Wednesday,
> August 17, 2011. When you reply, please indicate your preference ("1" or "2")
> and provide a brief reason, if appropriate.
> 
> Thank you.
> 
> Peter
> 
> --
> Peter Saint-Andre
> https://stpeter.im/
> 
>

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 26 Aug 2011 21:20:26 +0200

> We note there is interest and the possibility of introducing related functionality, error codes, etc., in the future, perhaps in conjunction with multiplexing support.

Will the core draft include some way to negotiate max frame size (and
maybe also max message size)?

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 26 Aug 2011 14:58:41 -0600

<hat type='AD'/>

> maybe also max message size)?

Based on my discussions with the chairs and editors, I would say that
any such parameter negotiation feature is out of scope for the WS 1.0
specification we are all working hard to finish as soon as possible.
Perhaps after this community has some experience with WS 1.0 it will
find the energy to work on WS 1.1 a few years from now. Because I think
iterative development is a good thing, I would welcome such an effort
(although I won't be your Area Director at that point).

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Greg Wilkins <[email-masked]>
Date: Sat, 27 Aug 2011 11:58:51 +1000

> (although I won't be your Area Director at that point).

Can we at least have clarification on what close code should be used
if the implementation receives a message (NOT FRAME) that it cannot
handle due to some resource limitation (perhaps the message is too
large, disk full, OOM exception, server too busy, etc.).

1000 is wrong because it is not successful
1001 going away  is not really the right as the endpoint is still there
1002 is wrong because it is not a protocol error
1003 for can't handle data type is a possible.  Maybe change the
description to just that the message cannot be handled for unspecified
reasons.

I'm happy with some generic "can't handle the messages you're sending
me" close code rather than a specific too-large code.

cheers

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Sat, 27 Aug 2011 18:26:18 +0200

> (although I won't be your Area Director at that point).

Just an idea coming to my mind:

1) Client sends the HTTP GET:

  GET /lalala HTTP/1.1
  Upgrade: websocket
  Connection: Upgrade
  Host: ws.domain.org
  Sec-WebSocket-Origin: http://www.domain.org
  Sec-WebSocket-Protocol: foo
  Sec-WebSocket-Key: USS3+Z6mzQg327mqqUabJw==
  Sec-WebSocket-Version: 8
  Sec-WebSocket-Max-Frame-Out: 65536
  Sec-WebSocket-Max-Frame-In: 65536

- Sec-WebSocket-Max-Frame-Out means the max frame size the client
could send to the server.
- Sec-WebSocket-Max-Frame-In means the max frame size the client
accepts from the server.

2) An intermediary (WOOW, I'm speaking about intermediaries!!!) can
modify (or append if not present) those two headers. In case of
modifying them, it can only decrease values. So let's suppose an
intermediary receives the above GET and decides to decrease bot
values, and sends the GET to the WS server with these headers
modified:

  Sec-WebSocket-Max-Frame-Out: 32768
  Sec-WebSocket-Max-Frame-In: 32768

3) The WS server receives the GET, inspects such values and,
optionally, can decrease them (never increase), and MUST add both
headers to the 101 reply (even if they are not present in the GET
request), so it replies:

  HTTP/1.1 101 Switching Protocols
  Upgrade: websocket
  Connection: Upgrade
  Sec-WebSocket-Accept: af8bl6g+YNEpwsc34843OKbe9Fk=
  Sec-WebSocket-Protocol: foo
  Sec-WebSocket-Max-Frame-Out: 16384
  Sec-WebSocket-Max-Frame-In: 32768

4) The intermediary receives the 101 and ensures that those values are
not higher that the values it previously set in the GET request.

5) The client receives the 101 and already knows the max frame size it
can send to the server, and the max frame size it could receive from
the server.

So there is a frame size negotiation mechanism that also makes
"intermediaries" happy. Opinions?

NOTE: instead of two headers, there could be a single one as follows:

  Sec-WebSocket-Max-Frame: 16384 / 32768

so first value is the client->server max frame size and second value
is the server->client max frame size.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: John Tamplin <[email-masked]>
Date: Sat, 27 Aug 2011 12:33:42 -0400

> Just an idea coming to my mind:

How is this anything more than a potential way of implementing option #2,
which was rejected?

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Sat, 27 Aug 2011 18:42:38 +0200

> How is this anything more than a potential way of implementing option #2,

AFAIK the problem with option #2 is that it does not define a way for
including an intermediary in such frame size negotiation (this is what
I suppose after reading other related threads). The mechanism I've
suggested theorically solves that requirement and allows an
intermediary to take active part in the negotiation.

With my suggestion, the intermediary can impose its own restrictions
so there is no need at all for reassembling frames in the
intermediary.

NOTE: Of course, an intermediary should not be WS "message" aware. It
should work at "transport" level (so just WS frames). Just the
endpoint should be WS message/protocol aware.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: John Tamplin <[email-masked]>
Date: Sat, 27 Aug 2011 13:18:01 -0400

> intermediary to take active part in the negotiation.

Option #2 said peer, which would apply equally to an intermediary
terminating the TCP connection as well as the ultimate endpoint.

---

From: Greg Wilkins <[email-masked]>
Date: Sun, 28 Aug 2011 07:59:20 +1000

Iñaki,

As much as I support the declaration of any size limits, I don't think
we managed to convince enough people to establish any consensus around
that idea in any of the many ways that it could be implemented.

The AD has called consensus on #1, so I think we should move on....
for example to consider what standard action to take if the message
cannot be handled for reasons including being too large (as per my
previous email).

cheers

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 29 Aug 2011 11:08:28 -0600

<hat type='individual'/>

> cannot be handled for reasons including being too large (as per my

In side discussions among the working group chairs and document editors
during the recent consensus call thread, I suggested one possible
compromise: an error condition like "policy-violation" as a catch-all
that the server or client could return if it has received data that
violates some local service policy (message size, frame size, sending
rate, etc.). However, such an error condition would probably lead to a
rat hole about designing a general policy framework, and as far as I can
see the WG does not have a strong interest in working on such a beast at
this time (although it might do so in the future).

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 30 Aug 2011 09:22:42 +1000

> this time (although it might do so in the future).

So I still don't know what close code to use if I get a message that
is larger than the application has given me permission to accept?

I'm currently sending a 1003, but that is a mis-use according to the
description.

regards

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 29 Aug 2011 17:24:42 -0600

> I'm currently sending a 1003, but that is a mis-use according to the

If we were to pursue that approach, I think we'd need to define a new
code for policy violation.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 30 Aug 2011 10:53:50 +1000

> code for policy violation.

But we are going to have to pursue some approach.

I just did an experiment and send a stream of 32k continuation frames
to FF6 and FF7. Both of them just closed the connection after 487
frames.  The server just got a broken pipe (and gave a 1006 to the
app) and the client got an onclose callback with wasClean set to
false. So that looks to both client and server as just a network
failure.  The client application would be totally justified in just
reopening the connection and trying whatever operation it wanted to do
again.  The server would continue trying to send large messages
blissfully unaware that it was breaking clients.

I'm not saying that we need a specific close code, but we need
something at least to distinguish it from a general network failure.
If an endpoint is closing for a specific reason, then just closing the
TCP/IP connection making it look like a network failure and many
applications will just retry.

regards

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 29 Aug 2011 19:34:03 -0600

<hat type='individual'/>

On 8/29/11 6:53 PM, Greg Wilkins wrote:
> frames.  

Yes, that's an example of what I would call a policy violation.

> The server just got a broken pipe (and gave a 1006 to the

Just retrying forever seems sub-optimal to me...

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 30 Aug 2011 12:03:49 +1000

> Just retrying forever seems sub-optimal to me...

Totally agree.  Good implementations should have backoffs and retry limits.

However in the case of message too large, even 1 retry is sub-optimal.
 It's never going to work and every retry is a waste of bandwidth.

Currently we have 1 close code for NORMAL close and then a few
specific close code for specific cases.   But without a shit-happens
close code, the entire mechanism is less worthwhile because we lose
the ability to tell the difference between a deliberate close on error
and a network failure.

We have 1003, which is bad data type.  That's kind of a very specific
policy failure.  Can't we just broaden the description of that to say
bad message - ie the message could not be handled for unspecified
reasons - might be bad data type, might be too large, etc.    The main
semantic is like a HTTP 4xx - the failure is something to do with the
message and not something to do with the server.

cheers

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Tue, 30 Aug 2011 10:05:37 +0800

On 08/30/2011 10:03 AM, Somebody in the thread at some point said:
> On 30 August 2011 11:34, Peter Saint-Andre<[email-masked]>  wrote:

Sounds pretty reasonable (and cheap) to me.

-Andy

---

From: Joe Mason <[email-masked]>
Date: Wed, 31 Aug 2011 15:40:50 -0400

> Sounds pretty reasonable (and cheap) to me.

+1 on this.

---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.