Subject: [hybi] WebSocket subprotocol parameters

From: Tobias Oberstein <[email-masked]>
Date: Fri, 17 Jan 2014 01:24:45 -0800

The "Sec-WebSocket-Extensions" allows for parameters for each element, eg (from "permessage-deflate"):

Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits; server_max_window_bits=10, permessage-deflate; client_max_window_bits

Similar seems to be disallowed for _subprotocols_.

RFC6455 says:

"""
The elements that comprise this value
MUST be non-empty strings with characters in the range U+0021 to
U+007E not including separator characters as defined in
[RFC2616] and MUST all be unique strings.  The ABNF for the
value of this header field is 1#token, where the definitions of
constructs and rules are as given in [RFC2616].
"""

and RFC2616 says:

"""
       token          = 1*<any CHAR except CTLs or separators>
       separators     = "(" | ")" | "<" | ">" | "@"
                      | "," | ";" | ":" | "\" | <">
                      | "/" | "[" | "]" | "?" | "="
                      | "{" | "}" | SP | HAT
"""

So

Sec-WebSocket-Protocol: wamp; version=2; serialization=msgpack; batched, wamp; version=1

is disallowed, right?

This is unfortunate, since "simulating parameters" like

Sec-WebSocket-Protocol: wamp.2.msgpack.batched, wamp

is a hack, and more so, RFC6455 requires that all subprotocol name be registered via the " WebSocket Subprotocol Name Registry".

This would mean registering all subprotocol variants:

wamp
wamp.2.json
wamp.2.msgpack
wamp.2.json.batched
wamp.2.msgpack.batched
..

Was there any good reason disallowing parameters for "Sec-WebSocket-Protocol"?

If not, could this be fixed via errata for RFC6455?

/Tobias

---

From: Julian Reschke <[email-masked]>
Date: Fri, 17 Jan 2014 12:10:17 +0100

>         separators     = "(" | ")" | "<" | ">" | "@"

I think that was discussed and the decision was made on purpose. For 
details you'll have to visit the mailing list archives though.

> If not, could this be fixed via errata for RFC6455?

Um, no. This is not a bug but a change request (with the potential of 
breaking existing code).

Best regards, Julian

---

From: Tobias Oberstein <[email-masked]>
Date: Fri, 17 Jan 2014 05:01:05 -0800

> I think that was discussed and the decision was made on purpose. For details
> you'll have to visit the mailing list archives though.

I can't find it .. searching the Hybi archives is a pain ..

> Um, no. This is not a bug but a change request (with the potential of breaking
> existing code).

Alright. I see.

It seems, WebKit is absolutely strict about allowed characters

https://trac.webkit.org/browser/trunk/Source/WebCore/Modules/websockets/WebSocket.cpp#L71

will only match "exactly"

https://trac.webkit.org/browser/trunk/Source/WebCore/Modules/websockets/WebSocketHandshake.cpp#L552

and, interestingly, will nevertheless escape any characters normally disallowed using Unicode escape sequences

https://trac.webkit.org/browser/trunk/Source/WebCore/Modules/websockets/WebSocket.cpp#L95

Thanks!
/Tobias

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 17 Jan 2014 14:31:35 +0100

>I can't find it .. searching the Hybi archives is a pain ..

It is not as easy as taking the first result on 

  https://www.google.com/search?q=site:http://www.ietf.org/mail-archive/web/hybi/+%22Sec-WebSocket-Protocol%22+parameters

But e.g. http://www.ietf.org/mail-archive/web/hybi/current/msg07700.html
uses `Sec-WebSocket-Protocol` parameters in an example, so that might be
nearby, if anybody feels archaeologically inclined.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Takeshi Yoshino <[email-masked]>
Date: Mon, 20 Jan 2014 17:35:34 +0900

On Fri, Jan 17, 2014 at 10:01 PM, Tobias Oberstein <
[email-masked]> wrote:

> details

I can't recall such discussion.

In this thread, Brodie used an example similar to that, but the main topic
of the post was to update the spec to include multiple subprotocol offer
mechanism.
http://www.ietf.org/mail-archive/web/hybi/current/msg04467.html

In this long thread, what character should be used for subprotocol is
discussed. Some people preferred more liberal rule, but no one discussed
qvalue-like grammar.
http://www.ietf.org/mail-archive/web/hybi/current/msg00913.html

Subprotocol History FYI:

We added the constraint "U+0021 ..  U+007E" from
http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol-31 -> 32

Single value -> multiple value (SP-separated)
http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-02 -> 03

We made it follow 1#(token | quoted-string) ABNF from
http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-03 -> 04

Changed from (token | quoted-string) to token
http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-07

---

From: Tobias Oberstein <[email-masked]>
Date: Mon, 20 Jan 2014 03:26:03 -0800

Hi Takeshi,

thanks alot for digging out these posts!

In retrospect, I think it's a pity that we did not specify "qvalue-like" grammer for Sec-WebSocket-Protocol as we did for Sec-WebSocket-Extension.

The latter provides a lot of useful flexibility … eg as used in "permessage-deflate".

Hacks like having subprotocols like this

wamp.2.json
wamp.2.msgpack
wamp.2.json.batched
wamp.2.msgpack.batched
..

negotiated only works for "discrete value parameters" (json/msgpack, batched/unbatched), but breaks down quite quickly.

As Julian pointed out, making Sec-WebSocket-Protocol using the same "qvalue-grammer" as Sec-WebSocket-Extension
would be a significant change to the RFC6455, and break existing code.

It would also possibly require changes to the WS API in JavaScript to be useful.

Mmh. Too bad;(

/Tobias

Von: Takeshi Yoshino [mailto:[email-masked]]
Gesendet: Montag, 20. Januar 2014 09:36
An: Tobias Oberstein
Cc: Julian Reschke; [email-masked]
Betreff: Re: [hybi] WebSocket subprotocol parameters

In this thread, Brodie used an example similar to that, but the main topic of the post was to update the spec to include multiple subprotocol offer mechanism.
http://www.ietf.org/mail-archive/web/hybi/current/msg04467.html

In this long thread, what character should be used for subprotocol is discussed. Some people preferred more liberal rule, but no one discussed qvalue-like grammar.
http://www.ietf.org/mail-archive/web/hybi/current/msg00913.html

Subprotocol History FYI:

---

From: Peter Thorson <[email-masked]>
Date: Mon, 20 Jan 2014 09:58:30 -0600

My impression was that subprotocol primarily changed the interpretation of messages payloads by the end applications themselves. WebSocket messages allow essentially arbitrary data so you can define whatever parameter setting mechanism you like with whatever grammar makes sense for your subprotocol and send the parameters in the opening “hello” message. Defining a grammar in the socket spec to allow application protocols to configure themselves seems limiting and out of scope.

Extensions on the other hand needed a way to communicate with themselves as defined and thus needed an explicitly provided grammar.

On Jan 20, 2014, at 5:26, Tobias Oberstein <[email-masked]> wrote:

> On Fri, Jan 17, 2014 at 10:01 PM, Tobias Oberstein <[email-masked]> wrote:
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi

---

From: Tobias Oberstein <[email-masked]>
Date: Mon, 20 Jan 2014 08:13:02 -0800

Take an example:

Client sends:
Sec-WebSocket-Protocol: foo; max_param1=10

Server answers:
Sec-WebSocket-Protocol: foo; max_param1=8

There are 2 problems:

1) ";" is not allowed as a character in subprotocols
2) the client would reject the WS connection anyway since " foo; max_param1=8" does not match _exactly_ (one of the) subprotocols announced: "foo; max_param1=10"

"""
   6.  If the response includes a |Sec-WebSocket-Protocol| header field
       and this header field indicates the use of a subprotocol that was
       not present in the client's handshake (the server has indicated a
       subprotocol not requested by the client), the client MUST _Fail
       the WebSocket Connection_.
"""

Von: Peter Thorson [mailto:[email-masked]]
Gesendet: Montag, 20. Januar 2014 16:59
An: Tobias Oberstein
Cc: Takeshi Yoshino; Julian Reschke; [email-masked]
Betreff: Re: [hybi] WebSocket subprotocol parameters

My impression was that subprotocol primarily changed the interpretation of messages payloads by the end applications themselves. WebSocket messages allow essentially arbitrary data so you can define whatever parameter setting mechanism you like with whatever grammar makes sense for your subprotocol and send the parameters in the opening "hello" message. Defining a grammar in the socket spec to allow application protocols to configure themselves seems limiting and out of scope.

On Jan 20, 2014, at 5:26, Tobias Oberstein <[email-masked]<mailto:[email-masked]>> wrote:

Hi Takeshi,

The latter provides a lot of useful flexibility ... eg as used in "permessage-deflate".

wamp.2.json
wamp.2.msgpack
wamp.2.json.batched
wamp.2.msgpack.batched
..

Mmh. Too bad;(

/Tobias

Von: Takeshi Yoshino [mailto:[email-masked]]
Gesendet: Montag, 20. Januar 2014 09:36
An: Tobias Oberstein
Cc: Julian Reschke; [email-masked]<mailto:[email-masked]>
Betreff: Re: [hybi] WebSocket subprotocol parameters

Subprotocol History FYI:

[email-masked]<mailto:[email-masked]>

---

From: Peter Thorson <[email-masked]>
Date: Mon, 20 Jan 2014 10:28:36 -0600

On Jan 20, 2014, at 10:13, Tobias Oberstein <[email-masked]> wrote:

>        the WebSocket Connection_.“
> ""

What do you gain by negotiating max_param1 at the subprotocol level vs negotiating “foo” as your subprotocol and having the foo protocol define how to negotiate its own settings via regular websocket messages?

---

From: Tobias Oberstein <[email-masked]>
Date: Mon, 20 Jan 2014 09:12:48 -0800

>What do you gain by negotiating max_param1 at the subprotocol level vs negotiating "foo" as your subprotocol and having the foo protocol define how to negotiate its own settings via regular websocket messages?

If "param1" would influence how to parse raw WS payload into "foo" protocol, that would not be possible.

But I agree in general: as long as "param1" does not influence the application protocol parsing, the app protocol could do (any further) parametrization/negotiation itself and no WS subprotocol level negotiation is necessary.

/Tobias

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 21 Jan 2014 12:53:09 +0900

On Tue, Jan 21, 2014 at 2:12 AM, Tobias Oberstein <
[email-masked]> wrote:

> >What do you gain by negotiating max_param1 at the subprotocol level vs
> negotiating "foo" as your subprotocol and having the foo protocol define
> how to negotiate its own settings via regular websocket messages?
>
> But I agree in general: as long as "param1" does not influence the
> application protocol parsing, the app protocol could do (any further)
> parametrization/negotiation itself and no WS subprotocol level negotiation
> is necessary.
>

Yes, subprotocol negotiation can be continued while extension negotiation
needs to be completed in opening handshake before handing off the channel
for the app layer.

The only difference between them is latency. It takes 1 more RTT if param1
is not encoded into subprotocol (or url) but sent as a frame.

http://tools.ietf.org/html/rfc6455#section-4.1
>   Once the client's opening handshake has been sent, the client MUST
>   wait for a response from the server before sending any further data.
>   The client MUST validate the server's response as follows:

In practice, I'd recommend embedding offered subprotocol parameters into
subprotocol (which fits in token ABNF) or url if setup latency is
important. If you want to reply to coolproto.param1.10 with param1=8, you
could just acknowedge coolproto.param1.10 but send param1=8 in the first
WebSocket message.

---

From: "Martin J. Dürst" <[email-masked]>
Date: Tue, 21 Jan 2014 18:17:15 +0900

> My impression was that subprotocol primarily changed the interpretation of messages payloads by the end applications themselves. WebSocket messages allow essentially arbitrary data so you can defin…

Yes indeed. It has always been my impression that even subprotocols 
themselves are rather unnecessary. On the server side, just use a 
different URI/IRI as endpoint. On the client side, it's all JS anyway.

Regards,   Martin.

---

From: Tobias Oberstein <[email-masked]>
Date: Tue, 21 Jan 2014 02:12:14 -0800

> > My impression was that subprotocol primarily changed the interpretation
> of messages payloads by the end applications themselves. WebSocket
> messages allow essentially arbitrary data so you can define whatever
> parameter setting mechanism you like with whatever grammar makes sense
> for your subprotocol and send the parameters in the opening “hello”
> message. Defining a grammar in the socket spec to allow application
> protocols to configure themselves seems limiting and out of scope.
> 
> themselves are rather unnecessary. On the server side, just use a different
> URI/IRI as endpoint. On the client side, it's all JS anyway.

No quite, since using a different URL does not allow the server to communicate back
it's subprotocol selection, e.g.

Sec-WebSocket-Protocol: foo.msgpack, foo.json

allows the client to announce the "foo" protocol in 2 variants (MsgPack and JSON serialization)
and the server to _select_ one of those, and communicate back it's selection to the client.

An URL

ws://server.com?serializers=msgpack;json

does not allow the server to communicate back it's selection.

Further, as an example, imagine a subprotocol that allows message batching with length prefixes, that wants to negotiate the number of bytes used for the length prefix.

Sec-WebSocket-Protocol: foo; max_len_prefix=4

A server that wants to use "len_prefix=3" cannot communicate that - neither with URL (since there is no "URL response"), and not with Sec-WebSocket-Protocol,
since the server _must_ choose one subprotocol announced by the client _exactly_.

It cannot answer

Sec-WebSocket-Protocol: foo; len_prefix=3

Since the number of bytes per length prefix in batched messages is needed to _parse_ already the first WebSocket message at application level,
negotiating at the WS application message level does not work either (or gets complicated).

So I still think having Sec-WebSocket-Protocol work like Sec-WebSocket-Extension would have been useful.

/Tobias

---

From: Takeshi Yoshino <[email-masked]>
Date: Thu, 23 Jan 2014 13:49:51 +0900

On Tue, Jan 21, 2014 at 7:12 PM, Tobias Oberstein <
[email-masked]> wrote:

> different
The server can immediately send back a frame following the opening
handshake response. So, there's not much difference between subprotocol
selection reply in form of opening handshake and in form of the first
message. No additional RTT.

> does not allow the server to communicate back it's selection.

So, you'll define a thin extension layer to your application level protocol
which uses the URI and the first message from the server to negotiate
subprotocol parameter.

>
> Further, as an example, imagine a subprotocol that allows message batching
> with length prefixes, that wants to negotiate the number of bytes used for
> the length prefix.
>
> A server that wants to use "len_prefix=3" cannot communicate that -
> neither with URL (since there is no "URL response"), and not with
> Sec-WebSocket-Protocol,
> Since the number of bytes per length prefix in batched messages is needed
> to _parse_ already the first WebSocket message at application level,

I understand it'll be simpler if we could complete that with in subprotocol
negotiation framework. But does it really get so complicated if we do that
using URI and bootstrap message? The client just need to pass one phase
where it parses len_prefix from the server and then transit to real app
mode where len_prefix will be used.

I don't object to raising a request to relax subprotocol constraint from
the next version. But I think it doesn't make much difference technically.