Subject: [hybi] first draft of WS mux extension

From: John Tamplin <[email-masked]>
Date: Fri, 21 Oct 2011 13:20:10 -0700

I have modified the original mux proposal (apologies to those of you who
didn't see it) based on feedback from early reviewers and implementation
experience (and also a late change to the WS spec which allowed
simplification of the framing), and now I think it is ready for wider
review.

I have uploaded it, and it is currently available at
http://www.ietf.org/staging/draft-ietf-hybi-google-mux-00.txt

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 21 Oct 2011 23:31:55 +0200

2011/10/21 John Tamplin <[email-masked]>:
> I have uploaded it, and it is currently available
> at http://www.ietf.org/staging/draft-ietf-hybi-google-mux-00.txt

Hi, I get I 404:

404: Page Not Found

The page you have requested could not be found.

You requested: http://www.ietf.org/staging/draft-ietf-hybi-google-mux-00.txt

That URL does not currently exist on the IETF website. We have
recently rearranged and redesigned our website, so things have
changed.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: John Tamplin <[email-masked]>
Date: Fri, 21 Oct 2011 14:41:13 -0700

> Hi, I get I 404:

I used the wrong name and had to cancel it and resubmit.  It is now
available at

http://www.ietf.org/id/draft-tamplin-hybi-google-mux-00.txt

---

From: Gabriel Montenegro <[email-masked]>
Date: Fri, 21 Oct 2011 21:50:36 +0000

Thanks John and Takeshi san!

For those that prefer html, the draft is also available at:
http://tools.ietf.org/html/draft-tamplin-hybi-google-mux-00

Gabriel

From: [email-masked] [mailto:[email-masked]] On Behalf Of John Tamplin
Sent: 21 October, 2011 14:41
To: Iñaki Baz Castillo
Cc: Hybi
Subject: Re: [hybi] first draft of WS mux extension

On Fri, Oct 21, 2011 at 2:31 PM, Iñaki Baz Castillo <[email-masked]<mailto:[email-masked]>> wrote:
Hi, I get I 404:

I used the wrong name and had to cancel it and resubmit.  It is now available at

---

From: Greg Wilkins <[email-masked]>
Date: Sat, 22 Oct 2011 09:26:01 +1100

FYI,

I just noticed that this -00 draft is significantly different to the
other -00 draft for MUX that I have seen (I guess the earlier one had
not been submitted, so no increment in the draft number).

So for anybody else like me that had not looked at this for a while
thinking it had not changed, have a look at the new draft.   It has
been updated to support interleaved WS frames and looks really clean
and simple - Good work guys!

cheers

On 22 October 2011 08:50, Gabriel Montenegro
<[email-masked]> wrote:
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>
>

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Sat, 22 Oct 2011 00:29:46 +0200

> http://www.ietf.org/id/draft-tamplin-hybi-google-mux-00.txt

The Overview says:

--------------
Initially, it will be identified by the private-use token "x-google-mux".
--------------

Please, don't use "x-" for a private extension, or we will end with
lot of clients and servers implementing the "x-" private name and the
standarized name forever. Check
http://tools.ietf.org/html/draft-saintandre-xdash-considered-harmful-01

I strongly propose to rename it to "mux".

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Sat, 22 Oct 2011 00:33:50 +0200

> I strongly propose to rename it to "mux".

I mean right now, without waiting for it to be an approved RFC.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: John Tamplin <[email-masked]>
Date: Fri, 21 Oct 2011 18:45:59 -0400

> not been submitted, so no increment in the draft number).

The previous version only existed in a Google Doc, and had no version suffix
at all.

> and simple - Good work guys!

Any feedback, such as suggestions for the TBDs?

---

From: John Tamplin <[email-masked]>
Date: Fri, 21 Oct 2011 18:51:59 -0400

> I mean right now, without waiting for it to be an approved RFC.

Until there is feedback that others besides Google are interested in
implementing it, it seems presumptuous to take the name "mux".

I am aware of the article you reference, but since non-private use names
have to be registered before use, it isn't clear the alternative is any
better.  WS has already shown that you can have early deployments of a work
in progress, make breaking changes, and have implementations adapt to the
new version fairly quickly. So, I would much rather have some
implementations use x-google-mux first and have to change them later after
it is standardized, than to use the name "mux" now and then find out we
really want a different mux protocol for that name.  There are only a few
browsers, so if they all update the servers will have no choice but to
update to match.

---

From: "Len Holgate" <[email-masked]>
Date: Mon, 24 Oct 2011 08:37:05 +0100

John,

This seems nice and clean and pretty easy to implement. 

Have you considered removing the "relaxation" of the fragmented frame rules
from the base protocol (ie the changes that permit the example shown in 7).

I would have thought that if all fragments on mux channels were sent as
complete messages you could include the actual message's fin bit in the
extension data of the mux frame and so not change the base protocol at all.
This would mean that mux could be implemented on existing websocket
implementations without changing the underlying implementation at all. The
layer "above" could deal with all aspects of the muxing.

Len 

> -----Original Message-----
> Sent: 21 October 2011 23:52
> To: Iñaki Baz Castillo
> Cc: Hybi
> Until there is feedback that others besides Google are 
> interested in implementing it, it seems presumptuous to take 
> the name "mux".
> 
> 
> I am aware of the article you reference, but since 
> non-private use names have to be registered before use, it 
> isn't clear the alternative is any better.  WS has already 
> shown that you can have early deployments of a work in 
> progress, make breaking changes, and have implementations 
> adapt to the new version fairly quickly. So, I would much 
> rather have some implementations use x-google-mux first and 
> have to change them later after it is standardized, than to 
> use the name "mux" now and then find out we really want a 
> different mux protocol for that name.  There are only a few 
> browsers, so if they all update the servers will have no 
> choice but to update to match.
> 
> -- 
> John A. Tamplin

---

From: John Tamplin <[email-masked]>
Date: Mon, 24 Oct 2011 03:52:30 -0400

> layer "above" could deal with all aspects of the muxing.

That would require an additional framing layer to allow that, and objections
to that is what led to the support in the base spec allowing extensions to
define interleaving behavior.

---

From: "Len Holgate" <[email-masked]>
Date: Mon, 24 Oct 2011 09:30:12 +0100

No, you'd just use the existing framing in the base protocol and all
fragments would have the FIN bit set. Thus any channel can interleave
without changing any rules from the base protocol. You'd then have a single
FIN bit in the extension data, before the channel id, which acts as the fin
bit for this channel's data.

Your example in 7 becomes

81 06 [FIN BIT HERE = 0]01 "Hello" 81 04 [FIN BIT HERE = 1]02 "bye" 81 07
[FIN BIT HERE = 1]01 " world"

Len 

> From: John Tamplin [mailto:[email-masked]] 
> Sent: 24 October 2011 08:53
> To: Len Holgate
> Cc: Hybi
> That would require an additional framing layer to allow that, 
> and objections to that is what led to the support in the base 
> spec allowing extensions to define interleaving behavior. 
> 
> -- 
> John A. Tamplin

---

From: John Tamplin <[email-masked]>
Date: Mon, 24 Oct 2011 05:00:00 -0400

> [FIN BIT HERE = 1]01 " world"

Then those same intermediaries would have to understand the MUX extension in
order to do anything with frames anyway -- I don't see what this gains.

---

From: "Len Holgate" <[email-masked]>
Date: Mon, 24 Oct 2011 10:09:12 +0100

It means that any existing websocket implementation can support the mux
extension without needing to change the implementation at all, it could all
be implemented above the base protocol layer. This means that if an
implementation has a pluggable extension mechanism anyone, mux can be
implemnted now rather than when the implementation has been changed to
support it. 

I think it would be good practice to try and avoid changing the base
protocol for every extension. Avoiding changes to the base protocol will
likely see more support for various extensions as they'll be easier to
implement.

Your intermediary example doesn't work IMHO. Right now, if the intermediary
doesn't understand the mux extension it MUST fail the connection. If the
base protocol is unchanged then the intermediary can allow the mux extension
and ignore the frame contents, perhaps it's simply applying compression, or
SSL, or stripping masking, or routing via a non websocket transport, or
whatever, it can do all of that without understanding the mux extension.
Only an intermediary that wants to inspect data need worry about the fact
that it doesn't understand the mux extension in this case and if the
intermediary DOES want to inspect payload data and DOES understand the mux
extension then the changes are similar (it needs to understand that the
channel id and per channel FIN bit are in the extension data, rather than
just the channel id).

Len 

> Sent: 24 October 2011 10:00
> To: Len Holgate
> Cc: Hybi
> Then those same intermediaries would have to understand the 
> MUX extension in order to do anything with frames anyway -- I 
> don't see what this gains.
> 
> -- 
> John A. Tamplin

---

From: Tobias Oberstein <[email-masked]>
Date: Mon, 24 Oct 2011 03:04:50 -0700

> Your intermediary example doesn't work IMHO. Right now, if the
> intermediary doesn't understand the mux extension it MUST fail the
> connection. If the base protocol is unchanged then the intermediary can

Why must it fail?

"""
An intermediary might coalesce and/or split frames, if no
extensions were negotiated by the client and the server, or if some
extensions were negotiated, but the intermediary understood all the
extensions negotiated and knows how to coalesce and/or split frames
in presence of these extensions.
"""

"""
An intermediary MUST NOT change the fragmentation of any message
in the context of a connection where extensions have been
negotiated and the intermediary is not aware of the semantics of
the negotiated extensions.
"""

So when the intermediary does not understand the mux extension,
it MUST NOT change fragmentation.

Where does the spec say it MUST fail?

It may of course, due to local policy .. like "we don't allow any WS extension".

---

From: "Len Holgate" <[email-masked]>
Date: Mon, 24 Oct 2011 11:28:35 +0100

Ok, sorry, it must fail to enable the extension. 

If we don't change the base protocol and the intermediary doesn't care about
payload data then it could allow the extension to be enabled and simply deal
with the normal websocket framing rules and ignore the content of the frames
and any mux specific stuff that's going on.

Len 

> From: Tobias Oberstein [mailto:[email-masked]] 
> Sent: 24 October 2011 11:05
> To: Len Holgate; 'John Tamplin'
> Cc: 'Hybi'
> Subject: AW: [hybi] first draft of WS mux extension
>

---

From: "Len Holgate" <[email-masked]>
Date: Mon, 24 Oct 2011 11:43:15 +0100

Ok, it cant ;) as my suggestion breaks if the intermediary changes any of
the framing; however, since intermediaries are always the "get out of jail
free card", an intermediary could be configured to allow some extensions to
pass through even though it doesn't fully understand them as long as it left
the framing alone...

I still think it would be good to avoid changing the base protocol for every
extension. So far we have two proposed extensions and both require changes
to the base protocol handling code to support them. I think it's likely that
more extensions will be more widely supported if they can be plugged into a
base protocol handler without needing to change it, rather than each one
requiring updates to existing, deployed code. Though, perhaps, that's what
sub protocols should be for...

Len

---

From: Tobias Oberstein <[email-masked]>
Date: Mon, 24 Oct 2011 03:47:31 -0700

> If we don't change the base protocol and the intermediary doesn't care
> about payload data then it could allow the extension to be enabled and
> simply deal with the normal websocket framing rules and ignore the content
> of the frames and any mux specific stuff that's going on.

I think that using _any_ extension lowers the chance of successful connection:

either the intermediary needs to implement/understand the extension

or

the intermediary / admin needs to allow and ignore the extension, and then be disallowed
to change fragmentation and possibly be unable to do its thing, like i.e. content inspection

==

there might be extensions which don't change the base WS framing rules, where an
intermediary not understanding the extension might nevertheless safely change
the fragmentation

i.e. "per-frame compression" when intermediary does not understand

==

a MUX extension that does not change the base WS framing rules, doesn't introduce
RSV bits or opcodes would have higher chance for successful connection

by definition, that would not any longer be an "WS extension", but a "WS subprotocol".

thus, maybe what you are striving for is a "mux subprotocol" ?

---

From: "Len Holgate" <[email-masked]>
Date: Mon, 24 Oct 2011 11:50:35 +0100

Agreed.

BUT if the extension can be passed through untouched then that will increase
the chance of a connection being allowed with that extension. 

I'm not that bothered to be honest.

Len

> Sent: 24 October 2011 11:48

---

From: John Tamplin <[email-masked]>
Date: Mon, 24 Oct 2011 11:05:12 -0400

> Ok, sorry, it must fail to enable the extension.

While the intermediary could conceivably alter the handshake, in most cases
they are simply observers of the traffic.  If they actually terminate one
connection and create another, they are more properly tw WS endpoints rather
than an intermediary.

> about

We aren't changing the base protocol -- the base protocol already allows
extensions to define the meaning of such things.  If existing intermediaries
violate this, then perhaps it is good to get an extension out early that
will break them, so they properly implement the spec.

The whole point of having MUX in the first place is that it can be done
without changing the endpoints.  In a typical scenario, the user agent
automatically reuses existing connections and sends to a mux-aware server,
with application code on both ends neither knowing or caring that this is
taking place.

---

From: John Tamplin <[email-masked]>
Date: Mon, 24 Oct 2011 11:07:54 -0400

> i.e. "per-frame compression" when intermediary does not understand

No, if you move one byte from one fragment to another, you will have to
alter both that byte (and possibly substitute a different ending byte for
the first frame) and the succeeding bytes in a hypothetical
per-frame-compression extension.  Since an intermediary that doesn't
understand an extension doesn't even know if there are any extension data
bytes at the beginning of the payload or not, or whether these are
per-message or per-frame, it cannot change fragmentation at all if it does
not understand any negotiated extension.

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 25 Oct 2011 08:59:27 +1100

On 24 October 2011 21:43, Len Holgate <[email-masked]> wrote:
> extension.

Len,

While I have not yet implemented MUX yet, I do not expect that I will
have to change the base jetty implementation to support it.

The enforcement of non-interleaved fragments is done in a layer above
the frame parsing and below the message delivery API.  It is into this
layer that extensions will be plugged, thus the MUX extension will
intercept frames and redirect them to multiple message aggregation and
delivery APIs.  Thus the same code that checks for no interleaved
messages will still execute, but after the mux layer and it will check
that the demuxed streams contain legal frame sequences.

regards

---

From: "Len Holgate" <[email-masked]>
Date: Tue, 25 Oct 2011 12:03:17 +0100

Greg, 

I doubt I have to change my code either. 

I just think that in general it would be better to avoid flexing the base
protocol in extension designs unless absolutely necessary. 

Len

> From: Greg Wilkins [mailto:[email-masked]] 
> Sent: 24 October 2011 22:59
> To: Len Holgate
> Cc: Tobias Oberstein; John Tamplin; Hybi

---

From: John Tamplin <[email-masked]>
Date: Tue, 25 Oct 2011 10:40:27 -0400

> protocol in extension designs unless absolutely necessary.

It sounds like what you want is a subprotocol, which does not meet the needs
this extension was designed for. Doing some crude hack like making the FIN
bit set in every frame so you can interleave them arbitrarily, then putting
the real FIN bit in extension data seems far worse than using the
extensibility hooks that were designed into the base protocol specifically
to support extensions like this.

---

From: "Len Holgate" <[email-masked]>
Date: Tue, 25 Oct 2011 17:05:04 +0100

John,

Yes, I think you're right, I think I'm just confusing where a subprotocol
might be used and where an extension might be used. 

Len

> Sent: 25 October 2011 15:40
> To: Len Holgate
> Cc: Greg Wilkins; Tobias Oberstein; Hybi
> It sounds like what you want is a subprotocol, which does not 
> meet the needs this extension was designed for. Doing some 
> crude hack like making the FIN bit set in every frame so you 
> can interleave them arbitrarily, then putting the real FIN 
> bit in extension data seems far worse than using the 
> extensibility hooks that were designed into the base protocol 
> specifically to support extensions like this.
> 
> -- 
> John A. Tamplin

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 26 Oct 2011 16:27:35 -0600

On 10/21/11 4:51 PM, John Tamplin wrote:
> On Fri, Oct 21, 2011 at 6:33 PM, Iñaki Baz Castillo <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
>     2011/10/22 Iñaki Baz Castillo <[email-masked] <mailto:[email-masked]>>:

There is nothing presumptuous about it, IMHO.

> better.

Says who?

> WS has already shown that you can have early deployments of a
> work in progress, make breaking changes, and have implementations adapt
> to the new version fairly quickly. 

In the case of this community, that's true.

> So, I would much rather have some
> implementations use x-google-mux first and have to change them later
> after it is standardized, than to use the name "mux" now and then find
> out we really want a different mux protocol for that name.  There are
> only a few browsers, so if they all update the servers will have no

http://tools.ietf.org/html/draft-ietf-appsawg-xdash has suggestions
about naming things without the "x-" prefix. Please have a look, because
it's been changed since it became a WG draft.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: John Tamplin <[email-masked]>
Date: Wed, 26 Oct 2011 19:19:34 -0400

> There is nothing presumptuous about it, IMHO.

Ok, I have no problem changing the name to mux.

Even though there hasn't been much feedback, what little there has been
seems to have been positive, so I want to go ahead and finish it up.

What is the recommended way to write up the IANA considerations section,
about registering a value for the WS Sec-WebSockets-Extension header?
http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-17#section-11.4
establishes a registry, but I don't see details of what should be included
in the MUX document.  The closest I could find was
http://tools.ietf.org/html/rfc4169#section-5.1 -- can you recommend how this
should be written up?

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 26 Oct 2011 17:28:11 -0600

On 10/26/11 5:19 PM, John Tamplin wrote:
> On Wed, Oct 26, 2011 at 6:27 PM, Peter Saint-Andre <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
> establishes a registry, but I don't see details of what should be
> included in the MUX document.  The closest I could find
> was http://tools.ietf.org/html/rfc4169#section-5.1 -- can you recommend
> how this should be written up?

I think you would include an IANA Considerations section that says it is
registering a value for the Sec-WebSocket-Extension header field in
accordance with Section 11.4 of draft-ietf-hybi-thewebsocketprotocol,
and specify the information mentioned there:

   Extension Identifier

   Extension Common Name

   Extension Definition

   Known Incompatible Extensions

Feel free to post what you come up with to the list here and we'll help
you with wordsmithing.

In fact the registration policy is "First Come First Served" so you
don't need to wait on publication of your RFC to complete the
registration, but given that you're working on an Internet-Draft there's
no harm in including the IANA registration in that document.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Brian <[email-masked]>
Date: Wed, 26 Oct 2011 16:57:08 -0700

>> seems to have been positive, so I want to go ahead and finish it up.
Awesome!  I look forward to following its progress.  I might have an
opportunity to take a stab at an implementation some time in the coming
weeks.  Do you have a prototype implementation against which I could test?

Cheers,
Brian

---

From: John Tamplin <[email-masked]>
Date: Wed, 26 Oct 2011 21:17:33 -0400

> weeks.  Do you have a prototype implementation against which I could test?

Not currently -- Andy and Greg both had implementations of the earlier
version, so presumably it wouldn't be too hard to adapt those
implementations.

---

From: John Tamplin <[email-masked]>
Date: Wed, 26 Oct 2011 21:16:20 -0700

Updated version, mostly cleaning up I-D nits and adding an IANA section to
register the extension name.

http://tools.ietf.org/html/draft-tamplin-hybi-google-mux-01

---

From: "Len Holgate" <[email-masked]>
Date: Thu, 27 Oct 2011 06:39:25 +0100

John, Brian,

I was just going to ask the same thing, something that I haven't written to
test against would be very useful.

Len

> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Brian
> Sent: 27 October 2011 00:57
> To: John Tamplin
> Cc: Hybi
> 	On Wed, Oct 26, 2011 at 6:27 PM, Peter Saint-Andre 
> <[email-masked]> wrote:
> 	
> 
> 		Even though there hasn't been much feedback, 
> what little there has been seems to have been positive, so I 
> want to go ahead and finish it up.
> 
> 
> 
> Awesome!  I look forward to following its progress.  I might 
> have an opportunity to take a stab at an implementation some 
> time in the coming weeks.  Do you have a prototype 
> implementation against which I could test?
> 
> Cheers,
> Brian 
> 
>

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 28 Oct 2011 11:26:57 +1100

> implementing it, it seems presumptuous to take the name "mux".

If you could somehow convince gmail that all postings from
[email-masked] to hybi are not spam, then that would greatly assist in
giving you feedback!
Do any other readers using gmail have this problem with Johns postings?

We in the Jetty project are keen to implement it.  We'll be even
keener once we know there will be a browser implementation available.

perhaps the name x-mux   might be a compromise while it is in draft status.

cheers

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 28 Oct 2011 11:29:17 +1100

> Ok, I have no problem changing the name to mux.

Ah - another spam'ed post from John.

As you are happy with mux, ignore my previous x-mux suggestion.

cheers

---

From: Brian McKelvey <[email-masked]>
Date: Thu, 27 Oct 2011 17:30:59 -0700

I use gmail and johns postings never end up in my spam box to my knowledge.

Sent from my iPhone