Subject: Re: [hybi] Binary/Text Distinction

From: "Bob Gezelter" <[email-masked]>
Date: Mon, 22 Aug 2011 06:41:54 -0700

John,

With all due respect, I must disagree.

You are correct that what is in the record needs to be a matter of
agreement between the sender and the recipient.

However, the WebSocket protocol is not, IMHO, the correct place for that
distinction. It is a higher-level distinction, that should be a
"protocol" between the respective WebSockets APIs. There is no need for
this distinction within the WebSocket protocol itself.

Admittedly, the coincidence of the WebSockets API and the WebSocket
protocol using similar names does contribute to a degree of "misspeak"
hazard.

- Bob Gezelter, http://www.rlgsc.com

> -------- Original Message --------
> Subject: Re: [hybi] Binary/Text Distinction
> From: John Tamplin <[email-masked]>
> Date: Mon, August 22, 2011 6:35 am
> To: Iñaki_Baz_Castillo <[email-masked]>
> Cc: Bob Gezelter <[email-masked]>, [email-masked],        Bjoern
> Hoehrmann <[email-masked]>
> 
> 
> On Mon, Aug 22, 2011 at 9:32 AM, Iñaki Baz Castillo <[email-masked]> wrote:
> 
> > 2011/8/22 Bob Gezelter <[email-masked]>:
> > > The transformations of "text" mode, while often useful, have been the
> > > source of innumerable accidents and problems since their advent.
> >
> > So ***great*** real case.
> >
> > WS authors, please, learn from the History (or you will repeat it).
> 
> 
> I don't think this proves what you think it does -- in fact, to me it argues
> for exactly what we have now, which is defining exactly how text messages
> are transported and represented, rather than leaving it up to each
> applcation.
> 
> -- 
> John A. Tamplin
> Software Engineer (GWT), Google

---

From: Bruce Atherton <[email-masked]>
Date: Mon, 22 Aug 2011 11:22:00 -0700

> this distinction within the WebSocket protocol itself.

But the Websocket protocol does specify the application-level protocol 
through the Sec-Websocket-Protocol field. The question is what the 
application protocol should look like when the field is not provided.

The Javascript layer in a browser is itself an application layer above 
Websockets, just one that is scriptable. It needs an application 
protocol to work with. Since working within a browser is the most common 
usecase, it makes sense that this application protocol should be the 
default one that is used when no Sec-Websocket-Protocol field is sent. 
Imagine if the spec made it explicit that any browser that wanted to use 
a mix of binary and text messages as its application protocol had to 
send a "Sec-Websocket-Protocol" header with the value "mixed-message". 
The default when no header is present just accomplishes the same thing.

This is exactly like the way that HTTP works. HTTP can be used to 
transfer any type of content, but you can specify the type within the 
HTTP protocol by using a Content-Type header. For historical reasons, 
when the Content-Type is not specified the default behaviour is 
typically to use heuristics to determine the actual type, but the 
details of this behaviour are not specified in the HTTP protocol. 
Websockets, on the other hand, is specifying the default behaviour up-front.

I think, though, that there may be value in clearly separating the 
default subprotocol from the rest of Websockets in the specification to 
make this clearer. That way another subprotocol could reuse the opcodes 
differentiating binary from text with no fear of incompatibility issues. 
The control opcodes, of course, are specific to the Websocket layer and 
remain the same across all subprotocols, and so cannot be reused.

So once we understand the mixed-message application protocol as one 
imposed by the Javascript appplication layer, the only question is 
whether that is a good design for the application. I think it is 
desirable for all the reasons that have been listed in these threads.

---

From: Stephen Williams <[email-masked]>
Date: Mon, 22 Aug 2011 13:36:11 -0700

On 8/22/11 11:22 AM, Bruce Atherton wrote:
> But the Websocket protocol does specify the application-level protocol through the Sec-Websocket-Protocol field. The question is 
> what the application protocol should look like when the field is not provided.
>
> The Javascript layer in a browser is itself an application layer above Websockets, just one that is scriptable. It needs an 
> application protocol to work with. Since working within a browser is the most common usecase, it makes sense that this application 
> protocol should be the default one that is used when no Sec-Websocket-Protocol field is sent. Imagine if the spec made it explicit 
> that any browser that wanted to use a mix of binary and text messages as its application protocol had to send a 
> "Sec-Websocket-Protocol" header with the value "mixed-message". The default when no header is present just accomplishes the same 
> thing.
>
> This is exactly like the way that HTTP works. HTTP can be used to transfer any type of content, but you can specify the type 
> within the HTTP protocol by using a Content-Type header. For historical reasons, when the Content-Type is not specified the 
> default behaviour is typically to use heuristics to determine the actual type, but the details of this behaviour are not specified 
> in the HTTP protocol. Websockets, on the other hand, is specifying the default behaviour up-front.
>
> I think, though, that there may be value in clearly separating the default subprotocol from the rest of Websockets in the 
> specification to make this clearer. That way another subprotocol could reuse the opcodes differentiating binary from text with no 
> fear of incompatibility issues. The control opcodes, of course, are specific to the Websocket layer and remain the same across all 
> subprotocols, and so cannot be reused.
>
> So once we understand the mixed-message application protocol as one imposed by the Javascript appplication layer, the only 
> question is whether that is a good design for the application. I think it is desirable for all the reasons that have been listed 
> in these threads.

I hope that we don't slip into a situation where that Javascript default subprotocol is the only one easily usable.  Based on recent 
comments, it seems that current thinking is that Javascript+Browser automatically forces you into the "default subprotocol".  There 
should be an option to layer application semantics on top of that, and also for a Javascript+browser application to select a 
different subprotocol and manage things directly.  I don't see rationale for why the latter seems to not be expected to be allowed.  
If there are specific security concerns, I'd like to see them reiterated.

sdw

---

From: Bruce Atherton <[email-masked]>
Date: Mon, 22 Aug 2011 14:01:29 -0700

On 22/08/2011 1:36 PM, Stephen Williams wrote:
>
> I hope that we don't slip into a situation where that Javascript 
> default subprotocol is the only one easily usable.  Based on recent 
> comments, it seems that current thinking is that Javascript+Browser 
> automatically forces you into the "default subprotocol".  There should 
> be an option to layer application semantics on top of that, and also 
> for a Javascript+browser application to select a different subprotocol 
> and manage things directly.  I don't see rationale for why the latter 
> seems to not be expected to be allowed.  If there are specific 
> security concerns, I'd like to see them reiterated.

Note that the Websocket API defined for browsers[1] allows specifying a 
list of subprotocols. It is just that there are no other subprotocols 
defined within the Javascript layer, so these subprotocols would be ones 
layered on top of the Javascript-specific subprotocol.

Perhaps someday browsers will have standard plugin APIs for defining 
Websocket extensions and subprotocols, or ones will become so common 
that the vendors will incorporate them directly. Time will tell.

[1] http://dev.w3.org/html5/websockets/

---

From: Stephen Williams <[email-masked]>
Date: Mon, 22 Aug 2011 14:30:31 -0700

On 8/22/11 2:01 PM, Bruce Atherton wrote:
>> I hope that we don't slip into a situation where that Javascript default subprotocol is the only one easily usable.  Based on 
>> recent comments, it seems that current thinking is that Javascript+Browser automatically forces you into the "default 
>> subprotocol".  There should be an option to layer application semantics on top of that, and also for a Javascript+browser 
>> application to select a different subprotocol and manage things directly.  I don't see rationale for why the latter seems to not 
>> be expected to be allowed.  If there are specific security concerns, I'd like to see them reiterated.
>
> Note that the Websocket API defined for browsers[1] allows specifying a list of subprotocols. It is just that there are no other 
> subprotocols defined within the Javascript layer, so these subprotocols would be ones layered on top of the Javascript-specific 
> subprotocol.

Why on top of the "Javascript-specific subprotocol"?  While it is easy to see why it should be friendly to Javascript, it doesn't 
seem right to aim for "Javascript-specific".

I can see that "WS default subprotocol" being equivalent to HTTP/HTTPS in a browser.  (BTW, I'd argue for TLS-only operation at this 
point.  You can always get plain text for debugging via a man in the middle debugging proxy.)  However, outside of the lowest level 
of the protocol, and maintain security basics (only connecting back to origin server / path), it seems possible to support different 
base subprotocols.  At the API level, simply have Javascript callbacks or similar as needed.  The setup/teardown and basic control 
messages can be fixed while allowing freedom in the rest of the details.

While this can be layered, the current stance is likely to result in everything useful being layered over a single default 
subprotocol simply to allow servers to have the same view of all clients, Javascript web browsers or not.

>
> Perhaps someday browsers will have standard plugin APIs for defining Websocket extensions and subprotocols, or ones will become so 
> common that the vendors will incorporate them directly. Time will tell.
>

sdw

---

From: Bruce Atherton <[email-masked]>
Date: Mon, 22 Aug 2011 15:07:25 -0700

On 22/08/2011 2:30 PM, Stephen Williams wrote:
>> Note that the Websocket API defined for browsers[1] allows specifying 
>> a list of subprotocols. It is just that there are no other 
>> subprotocols defined within the Javascript layer, so these 
>> subprotocols would be ones layered on top of the Javascript-specific 
>> subprotocol.
>
> Why on top of the "Javascript-specific subprotocol"?  While it is easy 
> to see why it should be friendly to Javascript, it doesn't seem right 
> to aim for "Javascript-specific".

Because we are talking about clients in the browser that have to be 
written in Javascript. That means that the protocol must first pass 
through the forms required by the Javascript layer. That means 
DOMString, Blob, or ArrayBuffer. So the UTF8/binary distinction in the 
Javascript layer of the subprotocol makes a lot of sense, at least until 
there is a Plugin subprotocol in the browser that allows the Javascript 
layer to be bypassed for something more specific.

For clients not in the browser, then there is no need to have the 
subprotocol deal with a divide between binary and text, it can be 
completely controlled by the application.

>
> ... it seems possible to support different base subprotocols.  At the 
> API level, simply have Javascript callbacks or similar as needed.  The 
> setup/teardown and basic control messages can be fixed while allowing 
> freedom in the rest of the details.

I'm not sure how this would work, but it seems like something to bring 
up on the W3 HTML5 list, not here.

---

From: Brian <[email-masked]>
Date: Mon, 22 Aug 2011 21:57:22 -0700

On Mon, Aug 22, 2011 at 11:22 AM, Bruce Atherton <[email-masked]>wrote:

> I think, though, that there may be value in clearly separating the default
> subprotocol from the rest of Websockets in the specification to make this
> clearer. That way another subprotocol could reuse the opcodes
> differentiating binary from text with no fear of incompatibility issues. The
> control opcodes, of course, are specific to the Websocket layer and remain
> the same across all subprotocols, and so cannot be reused.
>

To be clear.. a "subprotocol" as defined in the spec has absolutely nothing
whatsoever to do with frame types, opcodes, message types, binary vs text,
or any of that.  It represents an agreement between the client and server
regarding what *content* to send in the binary or text messages (or
combination) that is provided by the base protocol, and how to interpret
that content.  Examples of "subprotocols" are things like "SuperChat",
"MyRealTimeGameProtocol", "SharedWhiteboard" etc.  It's layered on top of
the messaging semantics.  Lets stop using the word "subprotocol" to mean
something else -- it's confusing.  There is no "default subprotocol," and a
subprotocol cannot define new types of messages.  It can use the existing
text and binary message types only.

Brian