Subject: Re: [hybi] a single CONTROL opcode for all control frames (was Re: Hello frames?)

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Mon, 18 Apr 2011 23:51:37 -0700

In making this change, I am stuck on the following, which leads me to prefer
Greg's proposal to John's:

1. In John's proposal, the type of control frame is determined by the first
byte of the payload. How does this work in the presence of extensions? The
payload is defined as extension data followed by concatenation data. If I
get an opcode of 0x1, followed by 12 bytes, which of those 12 bytes is the
actual byte that tells me if it's close/ping/pong? As an intermediary, I'm
now basically unable to determine anything. If one of the extensions is
"gzip all the application data" I am hopeless. Even if the extensions don't
touch the application data, then I still have to figure out where the
extension data ends and where the control frame data begins. (Is it simply
the last byte of the payload? What if we ever want to allow things like an
application-provided closure reason along with the close frame? Now where is
it?) This gets really complicated, unless we say that extension data is
never present with control frames, but that also seems undesirable.

2. The masking consensus was that all payload data is masked. The control
frame is now in the payload. Yuck. We essentially have to say there's no
application-provided data that will ever ride along with a control frame and
that control frames are not masked, or we have to unmask control frames, and
it starts getting overly complicated.

Basically, I'm worried that this is over-complicating things with little
actual benefit. Greg's proposal adds some benefit (clear distinction between
control frame and data frame) with little cost.

-Ian

2011/3/8 Ian Fette (イアンフェッティ) <[email-masked]>

> I am rather ambivalent, however if we do it we should do it now. I do agree
> with the following:
>
> 1. It would be nice to understand what is a "control" frame, especially if
> an extension wants to define its own definition of a control frame that
> should not be fragmented. If we had a single control frame opcode and the
> first byte defined which control it was (with some number reserved for
> extensions), this would be much clearer.
>
> The other points about scarcity of opcodes or making it lower cost hold
> some value to me, but frankly not nearly as much as the first.
>
> I will make the change unless anyone objects strongly.
>
>
> On Tue, Mar 8, 2011 at 8:13 PM, Greg Wilkins <[email-masked]> wrote:
>
>> Ian,
>>
>> What are your thoughts about this as editor?  there does not appear to
>> be any opposed and there is support for Johns original proposal.
>> It would be good to know soon if you think this will be in -07, as it
>> affects the server APIs somewhat.
>>
>> cheers
>>
>>
>>
>>
>> On 4 March 2011 04:25, Bruce Atherton <[email-masked]> wrote:
>> > On 02/03/2011 11:13 AM, John Tamplin wrote:
>> >>
>> >> My suggestion would be to define a CONTROL opcode, and then for that
>> >> opcode the first byte of the payload defines which control frame is
>> >> present.
>> >
>> > +1. Simply from a design perspective it makes more sense. Text frame,
>> binary
>> > frame, continuation frame, and control frame all feel like the same
>> level of
>> > abstraction. Breaking control down into ping, pong, and close seems like
>> an
>> > abstraction imbalance. And what do you gain by taking this deep-dive on
>> the
>> > control frames? Saving a single byte from a frame that is already tiny?
>> >
>> > Plus there are the other benefits you mention. I just think this makes
>> > sense.
>> >
>> >
>> > _______________________________________________
>> > hybi mailing list
>> > [email-masked]
>> > https://www.ietf.org/mailman/listinfo/hybi
>> >
>> _______________________________________________
>> hybi mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/hybi
>>
>
>

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Tue, 19 Apr 2011 00:07:35 -0700

Forgot that -06 added close reasons already. "<t>The Close message MAY
contain a body (the &quot;application
          data&quot; portion of the frame) that indicates a reason
          for closing, such as an endpoint shutting down, an endpoint
          having received a message too large, or an endpoint having
          received a message that does not conform to the format expected
          by the other endpoint. If there is a body, the first two bytes of
the
          body MUST be a 2-byte integer (in network byte order) representing
          a status code defined in <xref target="status_codes"/>. Following
the
          2-byte integer the body MAY contain UTF-8 encoded data, the
          interpretation of which is not defined by this specification.</t>"

So again, in this case, figuring out where the opcode is in the case of a
close frame is, without understanding the extensions, would be a nightmare.
Imagine an extension were in use that added 4 bytes to every frame. Ping
would now appear as a 5-byte payload with the opcode in byte 5 (end), but
close would be a 7+-byte payload with the close opcode not at the end (still
at 5, but an intermediary not knowing that the extension length was 4 would
now be hopeless to find this.)

-Ian

2011/4/18 Ian Fette (イアンフェッティ) <[email-masked]>

> In making this change, I am stuck on the following, which leads me to
> prefer Greg's proposal to John's:
>
>>> > abstraction. Breaking control down into ping, pong, and close seems
>>> like an

---

From: Salvatore Loreto <[email-masked]>
Date: Tue, 19 Apr 2011 14:36:16 +0200

for the sake of clarification,
the Greg proposal is related to partitioning the opcodes;

ref:  http://www.ietf.org/mail-archive/web/hybi/current/msg06696.html

/Sal

On 4/19/11 9:07 AM, Ian Fette (イアンフェッティ) wrote:
>           by the other endpoint. If there is a body, the first two 
> bytes of the
>           a status code defined in <xref target="status_codes"/>. 
> Following the
> So again, in this case, figuring out where the opcode is in the case 
> of a close frame is, without understanding the extensions, would be a 
> nightmare. Imagine an extension were in use that added 4 bytes to 
> every frame. Ping would now appear as a 5-byte payload with the opcode 
> in byte 5 (end), but close would be a 7+-byte payload with the close 
> opcode not at the end (still at 5, but an intermediary not knowing 
> that the extension length was 4 would now be hopeless to find this.)
>
> -Ian
>
> 2011/4/18 Ian Fette (イアンフェッティ) <[email-masked] 
> <mailto:[email-masked]>>
>
>     In making this change, I am stuck on the following, which leads me
>     to prefer Greg's proposal to John's:
>
>     1. In John's proposal, the type of control frame is determined by
>     the first byte of the payload. How does this work in the presence
>     of extensions? The payload is defined as extension data followed
>     by concatenation data. If I get an opcode of 0x1, followed by 12
>     bytes, which of those 12 bytes is the actual byte that tells me if
>     it's close/ping/pong? As an intermediary, I'm now basically unable
>     to determine anything. If one of the extensions is "gzip all the
>     application data" I am hopeless. Even if the extensions don't
>     touch the application data, then I still have to figure out where
>     the extension data ends and where the control frame data begins.
>     (Is it simply the last byte of the payload? What if we ever want
>     to allow things like an application-provided closure reason along
>     with the close frame? Now where is it?) This gets really
>     complicated, unless we say that extension data is never present
>     with control frames, but that also seems undesirable.
>
>     2. The masking consensus was that all payload data is masked. The
>     control frame is now in the payload. Yuck. We essentially have to
>     say there's no application-provided data that will ever ride along
>     with a control frame and that control frames are not masked, or we
>     have to unmask control frames, and it starts getting overly
>     complicated.
>
>     Basically, I'm worried that this is over-complicating things with
>     little actual benefit. Greg's proposal adds some benefit (clear
>     distinction between control frame and data frame) with little cost.
>
>     -Ian
>
>     2011/3/8 Ian Fette (イアンフェッティ) <[email-masked]
>     <mailto:[email-masked]>>
>
>         I am rather ambivalent, however if we do it we should do it
>         now. I do agree with the following:
>
>         1. It would be nice to understand what is a "control" frame,
>         especially if an extension wants to define its own definition
>         of a control frame that should not be fragmented. If we had a
>         single control frame opcode and the first byte defined which
>         control it was (with some number reserved for extensions),
>         this would be much clearer.
>
>         The other points about scarcity of opcodes or making it lower
>         cost hold some value to me, but frankly not nearly as much as
>         the first.
>
>         On Tue, Mar 8, 2011 at 8:13 PM, Greg Wilkins
>         <[email-masked] <mailto:[email-masked]>> wrote:
>
>             Ian,
>
>             On 4 March 2011 04:25, Bruce Atherton <[email-masked]
>             <mailto:[email-masked]>> wrote:
>             > +1. Simply from a design perspective it makes more
>             sense. Text frame, binary
>             > frame, continuation frame, and control frame all feel
>             like the same level of
>             > abstraction. Breaking control down into ping, pong, and
>             close seems like an
>             > abstraction imbalance. And what do you gain by taking
>             this deep-dive on the
>             > [email-masked] <mailto:[email-masked]>
>             [email-masked] <mailto:[email-masked]>

---

From: Patrick McManus <[email-masked]>
Date: Tue, 19 Apr 2011 09:03:30 -0400

> ref:  http://www.ietf.org/mail-archive/web/hybi/current/msg06696.html

If we are to rearrange the opcodes, doing it this way (i.e. just
rearranging the assignments so you can bit-test the is-opcode()
attribute) is my preferred path.

I am also fine, prefer even, leaving them as is.

-Patrick

---

From: Andy Green <[email-masked]>
Date: Tue, 19 Apr 2011 14:25:57 +0100

On 04/19/2011 08:07 AM, Somebody in the thread at some point said:

Hi -

> So again, in this case, figuring out where the opcode is in the case of
> a close frame is, without understanding the extensions, would be a
> nightmare. Imagine an extension were in use that added 4 bytes to every
> frame. Ping would now appear as a 5-byte payload with the opcode in byte
> 5 (end), but close would be a 7+-byte payload with the close opcode not
> at the end (still at 5, but an intermediary not knowing that the
> extension length was 4 would now be hopeless to find this.)

You can't really solve this without knowing the length of the extension 
data either implicitly because you know the extension or explicitly so 
you can skip it.  (Partitioning the opcode space isn't related to this 
at all).

If the close frame payload was only the 2 bytes it'd be easy you would 
just look at the last two bytes of the payload.  But with arbitrary 
additional bytes there's no way to know what the middle is.

Nothing's using this per-frame extended data at the moment.  One way to 
solve it would be to get rid of the per-frame extended data concept, and 
evict what would have been associated with the payload to a prior frame 
using extended opcode.  Then a dumb intermediary knows how to skip that 
frame and the control frame is parseable again no matter what extensions 
exist.

-Andy

---

From: John Tamplin <[email-masked]>
Date: Tue, 19 Apr 2011 10:13:07 -0400

> frame and the control frame is parseable again no matter what extensions

Then you have to maintain state across frames to know that extension
data goes with the next frame.  For example, if the extension data is
some form of compression dictionary, you can't interpret the
compressed payload contents without it, so it feels totally wrong to
move that to another frame.

When we came up with this framing, having the extension data area was
important to show that all of the ways that people wanted to extend
the protocol could be accommodated.  If we drop that now, then we are
reopening the framing discussion and have to find new ways to handle
extension mechanisms that were enabled by it, or convince everyone
involved that they really aren't necessary.  The inability to convince
everyone that their pet extension ideas weren't important was why the
three vectors for extension (reserved bits, reserved opcodes,
extension data area) were put in the framing in the first place.

I'm not sure it is necessary that intermediaries need to be able to
understand frames containing extensions they don't understand.  If it
is, then instead of leaving the extension area open to definition by
extensions, we would need to define some tag/value format that allows
intermediaries to skip unknown extensions (and probably tag some
extensions as critical, meaning the frame can't be processed if you
don't understand the extension).  However, note that defining exactly
what this format would be was exactly one of the circular discussions
that led to months of debate about the framing.  Note that making the
mask key be extension data becomes either inconsistent (no tag) or
more expensive (tag bytes) in this scenario.

Personally, I think the way we have things defined is the best
compromise for the base protocol -- the endpoints must know all the
negotiated extensions, and having that information allows them to know
exactly where and in what format various fields are in the extension
data.  Intermediaries cannot participate in the extension negotiation
(if they do, then they aren't an intermediary but both a server and a
client in the connection between ultimate endpoints), so they have to
be prepared for the endpoints negotiating extensions they don't
understand.  We already have language in the spec that deals with
this, in that intermediaries are not allowed to fragment/coalesce
frames if an extension is in use they don't understand.

Regarding the question at hand about control opcodes or close frame
information, is it necessary that an intermediary be able to interpret
them in the face of unknown extensions?  If it is, then what if we
wanted to define a new compression scheme as an extension -- an
intermediary that doesn't understand that extension has no chance of
understanding any content, so I believe that restriction imposes too
much cost.  So, my feeling is that it is fine if only the endpoints,
which necessarily know how to determine the size and contents of the
extension data area, are able to interpret the extended control opcode
and close frame flags.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Andy Green <[email-masked]>
Date: Tue, 19 Apr 2011 15:48:45 +0100

On 04/19/2011 03:13 PM, Somebody in the thread at some point said:
> On Tue, Apr 19, 2011 at 9:25 AM, Andy Green<[email-masked]>  wrote:

Well, everything on the wire is depending on what went before in a real 
way anyway, you can't re-sync up in this protocol, come in after the 
handshake and expect to know what was negotiated and so on.  So it seems 
OK to loosely bind frames like that to me, it's not fragile since only 
control frames can come inbetween.  But anyway I agree with the rest of 
what you wrote making it moot -->

> extension data area, are able to interpret the extended control opcode

You're right, everybody needs to know that there has been a close event. 
  But only the endpoints, who must know the extended part length, will 
care about any detail though and then the opcode is enough for the other 
guys.

-Andy

---

From: Bruce Atherton <[email-masked]>
Date: Tue, 19 Apr 2011 09:21:40 -0700

On 19/04/2011 12:07 AM, Ian Fette (イアンフェッティ) wrote:
> nightmare.

I agree. When I voted for this, I didn't fully appreciate the details. I 
was thinking that the opcode section of the header would be extensible, 
just as the length field is. So the opcode would be 1 byte unless it was 
the control opcode, in which case the next byte would be the sub-opcode 
for the specific control message being sent.

Putting it into the payload after the extension data seems to me like a 
bad idea for the same reasons that masking the framing seemed like a bad 
idea.

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 20 Apr 2011 08:33:11 +1000

On 19 April 2011 23:25, Andy Green <[email-masked]> wrote:
> Nothing's using this per-frame extended data at the moment.

I think because it is close to unusable.   One of the oft cited uses
for it was for MUX, but interleaving of fragments is not possible with
the current design, which has forced x-google-mux to go to great
lengths to associated channel ID's with fragments.

There is effectively no extension data anyway - since with out any
length or structure any extension that used it is essentially just
mutating the payload and then there is nothing to say the extension
data is at the front, end or munged in the middle of the payload.  I
think at this stage, we need to just embrace this - and rather than
saying we have extension data area, just say extension can mutate
payloads, including pre-pending and/or appending data.

The issue then is that intermediaries cannot fragment or aggregate
frames unless they understand the extension.   Again  I think we
should embrace this and allow arbitrary fragment orderings so that
MUXing extensions can interleave fragments.  The aggregating of these
fragments into a message for a channel can only be done by something
that understands the MUX extension, but then we already have that
restriction, so why not allow the interleaving?

cheers

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Tue, 19 Apr 2011 15:48:02 -0700

Going back to the matter at hand re: the opcodes. I'm fairly convinced I
don't want to have a generic control frame opcode and then define which
control it is in the body. That said, I think something like Greg's earlier
proposal would help and give us a bit of flexibility.

e.g. 0x0-0x7 == data opcodes
0x8-0xF == control opcodes

we would still have something like 4 unused opcodes in each bucket, and if
we wanted to, we could always define something like "this particular opcode
means you should look at this other place for more information" and still
have intermediaries understand that this is a control frame, and shouldn't
be e.g. fragmented or buffered excessively or merged with the next frame
etc. It would be a relatively cheap change to make.

-Ian

---

From: John Tamplin <[email-masked]>
Date: Tue, 19 Apr 2011 19:17:36 -0400

> etc. It would be a relatively cheap change to make.

I don't understand how you can say it would be cheap to add later, yet
infeasible now.  Either we are saying the entire opcode needs to be visible
to intermediaries that don't understand the negotiated extensions, in which
case adding it later is just as problematic as adding it now, or we are
saying we don't care if they don't understand it in which case it is no
problem to add it now.

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Tue, 19 Apr 2011 16:24:58 -0700

> saying we don't care if they don't understand it in which case it is no
I'm saying that I think the opcodes we have now are basic and probably are
useful to be understood by intermediaries (e.g. the fact that something is a
control frame is useful to understand. The fact that something is a close
frame is useful to understand). If we have additional control frames that we
designate in the future, they will by definition not be understood by v1
intermediaries. If they are useful to understand standalone, we can allocate
one of the remaining opcodes. If they are not useful outside some additional
context (e.g. an extension wants to introduce its own control frames) we
could set aside a single opcode to mean "control frame with additional data
at location X (payload)". Either way, by rearranging v1 intermediaries can
at least understand that it's a control frame.

-Ian

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Tue, 19 Apr 2011 19:08:56 -0700

To be clear:

Right now, we have a situation where intermediaries can't determine what is
a control frame and what is not a control frame. We can argue about just how
bad this is, but it does seem suboptimal. Greg's proposal of 0x0-0x7 and
0x8-0xF cleans this up with minimal cost. The cost is that it forces an
allocation of the remaining opcodes to two buckets, either DATA or CONTROL.
The alternative, a single opcode, means that we get into all the nasty
issues I described above. I would like to avoid that. If we do continue to
allocate opcodes in future versions, we still have some headroom for
judiciously allocated opcodes. Should we get close to running out, we can
always decide that "Well, we actually did need more opcodes, so let's say
0xF is a control frame opcode that means more data about the control frame
is in the payload." We get into the nasty issues I pointed out, but frankly
if we define any new opcodes at a later point, we will already have the
issues of deployed intermediaries not understanding the opcode, so the
marginal cost of that decision is much lower at that point in time, and we
still do have the option open to us.

As such, I am planning on re-ordering the opcodes in 07 to have 0x0-0x7 be
data and 0x8-0xF be control.

-Ian

2011/4/19 Ian Fette (イアンフェッティ) <[email-masked]>

>>> we would still have something like 4 unused opcodes in each bucket, and
>>> if we wanted to, we could always define something like "this particular
>>> opcode means you should look at this other place for more information" and
>>> still have intermediaries understand that this is a control frame, and
>>> shouldn't be e.g. fragmented or buffered excessively or merged with the next
>>> frame etc. It would be a relatively cheap change to make.
>>>
>>

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 19 Apr 2011 20:27:36 -0600

(no hats)

On 4/19/11 8:08 PM, Ian Fette (イアンフェッティ) wrote:
> To be clear:
> 
> Right now, we have a situation where intermediaries can't determine what
> is a control frame and what is not a control frame. We can argue about
> just how bad this is, but it does seem suboptimal. Greg's proposal of
> 0x0-0x7 and 0x8-0xF cleans this up with minimal cost. The cost is that
> it forces an allocation of the remaining opcodes to two buckets, either
> DATA or CONTROL. The alternative, a single opcode, means that we get
> into all the nasty issues I described above. I would like to avoid that.
> If we do continue to allocate opcodes in future versions, we still have
> some headroom for judiciously allocated opcodes. Should we get close to
> running out, we can always decide that "Well, we actually did need more
> opcodes, so let's say 0xF is a control frame opcode that means more data
> about the control frame is in the payload." We get into the nasty issues
> I pointed out, but frankly if we define any new opcodes at a later
> point, we will already have the issues of deployed intermediaries not
> understanding the opcode, so the marginal cost of that decision is much
> lower at that point in time, and we still do have the option open to us.
> 
> As such, I am planning on re-ordering the opcodes in 07 to have 0x0-0x7
> be data and 0x8-0xF be control.

Based on the discussion so far, that sounds like a reasonable approach.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: John Tamplin <[email-masked]>
Date: Tue, 19 Apr 2011 22:30:00 -0400

> data and 0x8-0xF be control.

Ok.

---

From: Andy Green <[email-masked]>
Date: Wed, 20 Apr 2011 08:27:36 +0100

On 04/19/2011 11:33 PM, Somebody in the thread at some point said:
> On 19 April 2011 23:25, Andy Green<[email-masked]>  wrote:

Right.  But if nobody does use it, actually it has zero footprint in the 
packet since it is defined as notionally payload in length terms.  So if 
people like to keep it, despite it's a little extra to think about to 
keep it able to work, it doesn't sound worth complaining against.

> payloads, including pre-pending and/or appending data.

Yeah I guess that is in fact the case, at least nobody not knowing the 
extension could tell if something had been appended as it has no idea 
where the payload started or its length even.

> restriction, so why not allow the interleaving?

I agree when you think that frames are already broken up into a bunch of 
arbitrary packets, there's nothing so special about atomic frames in mux 
case, and if monster frame lengths are still allowed mux granularity and 
latency can be held to ransom by a bad client or server insisting to put 
2^63 frames down it with extensions the mux doesn't understand so it 
can't fragment and breaks, just sitting there spamming one channel.

The cost of doing that though is so far x-google-mux takes the approach 
that its "mux block" payload length is a function of the history of 
handshakes in the heirarchy of channels and sub-muxes and the length of 
the websocket frame that is the leaf payload at each level - there's no 
explicit mux block length.  If mux blocks can fragment frames how the 
mux layer likes - which is good I think, certainly a more normal muxing 
action - then it requires introduction of explicit mux block length 
field.  I don't think that's a big problem but we add, eg, a couple of 
bytes per mux block then.

-Andy

---

From: Andy Green <[email-masked]>
Date: Wed, 20 Apr 2011 08:36:35 +0100

On 04/20/2011 08:27 AM, Somebody in the thread at some point said:

> explicit mux block length. If mux blocks can fragment frames how the mux
> layer likes - which is good I think, certainly a more normal muxing

... actually it's enough if the muxed peers tells a mux block MTU to 
each other at the handshake.  Then both sides might understand they 
expected a frame length of 0x1234 on seeing such a websocket frame 
header, but understand it will come in mux block chunks each of <= MTU 
length without having to explicitly say each time.  So it's no more 
expensive than trying to keep atomic frames in the mux.

-Andy