Subject: [hybi] Multiplexing, Binary/Text; Endpoint Identification

From: Bob Gezelter <[email-masked]>
Date: Mon, 11 Apr 2011 09:56:59 -0500

The recent discussion concerning some of the issues raised in my blog
(http://rlgsc.com/r/20110323) has been interesting. However, my point
is and was not to restart discussion from the beginning, but to ensure
as flexible a future as possible. The WebSocket protocol seems poised
for mass adoption; significant changes following official
standardization will be disruptive and painful. Past history shows that
such changes are often all but unimplementable, a reality with no
shortage of examples (SMTP was temporary until issues were resolved,
hence the "Simple"; also see "Mythical Man Month" by Fred Brooks
concerning OS/360 architectural mis-steps).

The multiplexing discussion is of great interest, and I am certainly
not trying to steal John's thunder, but perhaps the implications of my
main points have been obscured by the history of the discussion.

Many of the points raised in the ensuing discussion of multiplexing
are significant, yet the concerns have little effect on the
specification of multiplexing at the level of the wire protocol.

My concerns do not require a complete definition of multiplexing
behavior, nor do I intend to imply that all WebSocket protocol implementations 
support large-scale multiplexing. Rather, I see it
as critical to define the structural framework within which
multiplexing is implemented. The issues of implementation can then
be appropriately resolved at a future time. These concerns only
require a modest change in the wire protocol and related parts of the
the specification (these are not suggested text, merely checklist
items):

   - Addition of two frame types:
     o Open WebSocket subChannel (endpoint ID, quota, UTF-8/binary)
     o Close WebSocket subChannel

   - Small modification to Close frame ("close all subChannels")

   - Addition to the opening WebSocket protocol handshake
     ("SubChannels: n"; baseline, non-multiplexed implementations use
     "SubChannels: 1"). From both sides, count is minimum of exchanged
     values.

   - Coalesce frame types for UTF-8/binary data into a single frame type;
     "Data" (mode is associated with the subChannel and specified in the
     Open WebSocket subChannel control frame).

   - Addition of subChannel identification to a new "Switch SubChannel"
     control frame.

I know that many prefer to implement communication schemes based upon
printable representations (e.g., UTF-8). While this is not my preferred
approach for inter-machine communications (I personally prefer binary
specifications), I have no desire to have a "Coke vs. Pepsi" dispute.
I would prefer to enable all constituencies to have their favorite flavor.

That said, I do not see many applications routinely switching back and forth
on a frame-by-frame basis between UTF-8 and binary. If that capability is
desired, it can parsimoniously be provided by implementing a "Switch Mode"
control frame (it could also be included as part of the "Switch subChannel"
control frame).

This approach also removes the need for the restriction (at the end of 4.4
in the -06 draft) that all fragments within a message must be either text or
binary. The current specification does not permit intra-message mode
switching and this proposal does not change that rule. Under my proposal
the mode of a subchannel is established at open, and can be changed.
Intra-message mode switching is not permitted by definition.

The Open SubChannel command frame type now has three minimal requirements:

   - definition of endpoint (suggestion: TAG URI of desired partner; this
     allows a wide degree of flexibility in design and implementation
     without imposing anything more than an IETF-sanctioned way to
     hierarchically distinguish names for endpoints; including explicit
     ways to define "well-known" services, should they evolve; for
     example: WebSocket.ietf.org:ftp for an ftp portal over an
     existing WebSocket protocol connection).

   - Transfer quota specification (default value permissible).

   - specifying UTF-8 or binary mode for the subChannel.

Note that the above does not discuss the "Hows" of multiplexing, it
merely establishes a framework. Different multiplexing schemes can be
used as extensions, and it is certainly possible to enhance the Open
subchannel control frame with supplemental commands specifying options.

Needless to say, the existence of multiplexing at the WebSocket protocol
level is all but invisible to the JavaScript user/developer. The modest
changes are limited to the WebSockets library-class layer.

My reasoning is straightforward. Supersetting has a very poor track
record. Unintended mutually incompatible supersetting has been the bane of
interoperability since the long before the Internet protocol suite. Defining
an architectural space with multiple viable subset implementations enable
interoperability in the long term with few side effects at the outset.

Needless to say, I will be happy to clarify any of my thoughts on the above.

- Bob
+--------------------------------------------------------------------------+
| Robert "Bob" Gezelter                       E-Mail:  [email-masked]  |
| Robert Gezelter Software Consultant                                      |
| 35-20 167th Street, Suite 215               Fax:       (on Request)      |
| Flushing, New York  11358-1731                                           |
| United States of America                    http://www.rlgsc.com         |
+--------------------------------------------------------------------------+

---

From: John Tamplin <[email-masked]>
Date: Mon, 11 Apr 2011 11:20:06 -0400

On Mon, Apr 11, 2011 at 10:56 AM, Bob Gezelter <[email-masked]> wrote:

> behavior, nor do I intend to imply that all WebSocket protocol
> implementations support large-scale multiplexing. Rather, I see it

I believe the current extension model supplies the necessary infrastructure.
 If an endpoint does not support multiplexing, then having these new frame
types defined isn't going to help any more anyway.

>  - Addition to the opening WebSocket protocol handshake

In the non-MUXed case, we already have this:  the open subchannel is the
handshake (and there is no quota for non-MUXed channels, as that would
complicate simple uses), close subchannel is just a close frame, and since
there is only one subchannel it automatically means to close all of them.

>    Open WebSocket subChannel control frame).

This seems totally independent of MUX discussions.  As mentioned, the JSAPI
receives a callback when a message is complete, and that callback receives
an object of the proper type -- until binary frames are supported, that is
only a string.  Given the API choices there, which if you disagree with them
should be discussed in WHATWG, a text frame has to be differentiated from a
binary frame.

>  - Addition of subChannel identification to a new "Switch SubChannel"

I strongly disagree with relying on context to know which channel a
particular frame should be delivered to - I think it is much better to keep
the logical channel associated with a frame.

> I would prefer to enable all constituencies to have their favorite flavor.

At an application level, many of them want to exchange text information,
such as JSON objects, text in a chat program, etc.

> Intra-message mode switching is not permitted by definition.

Not just that all fragments must be text or binary, but that a message has a
single opcode type, and the fragments are all fragments of one message.  If
the first fragment is of some extension opcode, then all remaining fragments
are of that same opcode, with the exception that control frames can be
injected into the middle of a fragmented message.  The point of mixing
message/frame information like this is to keep things simple and allow small
frames to remain very small (with a 2-byte overhead, plus masking for
client->server).

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Andy Green <[email-masked]>
Date: Tue, 12 Apr 2011 09:04:03 +0100

On 04/11/2011 03:56 PM, Somebody in the thread at some point said:

Hi -

> interoperability in the long term with few side effects at the outset.

The danger here is that we instead introduce a sclerosis about the mux 
approach too early in the base spec.  And we cannot put other things in 
the base spec to support extensions that we did not think of yet, so 
it's questionable to do it for mux.  Leaving the reserved opcode 
wormhole for extension basis seems the better bet.

Actually, partway through implementing x-google-mux, the mux stuff is 
pretty scary and not lacking in implications all over the place.  Here 
are some examples.  They're not killers just hidden implications.

1) Existing libwebsockets is heavily using states and POLLOUT from 
poll() to rate-limit sending, I think it is best practice.

Trying to keep that good structure in mux world means nothing works 
properly until intra-mux quotas are implemented, since then you don't 
know when to make the synthetic POLLOUT event on the subchannel.  I 
assume it's still possible to do everything driven by a real poll() 
POLLOUT loop with the real socket, but that is pretty subtle and not 
mentioned anywhere in the mux spec since it's implementation.

2) A pre-connection speculative child list is required due to racing. 
Say the first ws:// connection is requested from the script, we compose 
a handshake or contact the proxy first, mark that state and return the 
as-yet unconnected websocket pointer.

So the client asks for the second ws:// right away.  We can see it is to 
the same address, port, url as the first guy.  But actually, we don't 
know then if x-google-mux extension was allowed by the server since the 
handshake is still pending for the first guy we would like to be our parent.

By default, we would deny muxing then to all the other connections until 
the first guy is established, which might be 500ms or whatever, usually 
all the other websocket connections from the script are requested long 
before.  So no muxing actually happens!

The answer is the speculative child list, if the parent connection fails 
they are freed to make their own connections, likewise if it succeeds 
without x-google-mux accepted by the server.  If the parent succeeds 
with x-google-mux only then do the children move to the state of having 
the MUX addchannel being sent for them and so on.

3) I already hated masking but x-google-mux correctly defines mux 
content is unmasked and you can mask the mux stream.  Well it's 
recursive, mux streams can be muxed but somehow this masking is magic at 
upper level?  Is that possible since my upper level may be someone 
else's mux subchannel and due to masking not being subordinate to 
framing, you don't know if you are looking at something which is masked 
or not just by looking at it.

4) Similar to 3), mux stream is a bunch of 
recursively-parse-it-to-determine-length random junk.  I have no idea 
how people will manage to debug and interoperate these streams. 
Traditionally here that concern has everyone sniggering behind their 
handkerchiefs but really - try it.  The sins so far in the lowest level 
protocol come right back to roost.

-Andy