Subject: 6455 Websockets and the relationship to HTTP

From: Patrick McManus <[email-masked]>
Date: Thu, 1 Dec 2016 10:48:20 -0500

Hi All.

The continued websockets discussion is interesting to me - its clear the
number of interested parties is small but significant (e.g. almost nobody
in the room in Seoul had read the presented draft, but a diverse number of
participants are making related technical contributions on the list.). The
general advice remains in effect - 6455 wasn't done in HTTPbis so its best
to seek another venue (a new wg, or perhaps dispatch help) for an update -
but when it comes to its interaction with HTTP that's on topic for this WG
and there probably is some expertise. So I don't mind the conversation
continuing as long as a parallel effort to find a home for actually
adopting the work is made.

I would like to use this thread to better understand, for my own
edification, what the problem being addressed here is and how big of a
problem it is in practice. Its not controversial to say that 6455 has
warts, and given the toolbox we have now it would be done differently - but
what in terms of operations would be improved by an update to a h2 world?
Would that be a different set of answers for a quic world?

Here's what I think I'm hearing, but there are so many messages that are
done in the weeds of the solution space I don't want to lose track of the
problems being solved - I think this list might help in any chartering
discussion:

* in a practical sense there is no mux and when you have mux you need
priority and flow control. h2 solves this.
* operational overhead of maintaining/admin h1 just to boostrap to
websockets.
* latency of a new h1 connection just to bootstrap to websockets
* operational overhead of separate conns for http and ws

is there more? some data on this stuff would be good. Is this really mostly
about mux?

I can see where the introduction of quic makes this gap even bigger.. with
its lower latency and potential to deploy more modern transport features.
It would be great to be more crisp on this - I have read
https://github.com/bidiweb/bidiweb-semantics/blob/master/SurveyOfProtocolGaps.md
and its still not clear to me what the driving concern is.

-Patrick

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 2 Dec 2016 11:09:41 +1100

On 2 Dec. 2016, at 2:48 am, Patrick McManus <[email-masked]> wrote:
> 
> Hi All.
> 
> The continued websockets discussion is interesting to me - its clear the number of interested parties is small but significant (e.g. almost nobody in the room in Seoul had read the presented draft, but a diverse number of participants are making related technical contributions on the list.). The general advice remains in effect - 6455 wasn't done in HTTPbis so its best to seek another venue (a new wg, or perhaps dispatch help) for an update - but when it comes to its interaction with HTTP that's on topic for this WG and there probably is some expertise. So I don't mind the conversation continuing as long as a parallel effort to find a home for actually adopting the work is made.
> 
> I would like to use this thread to better understand, for my own edification, what the problem being addressed here is and how big of a problem it is in practice. Its not controversial to say that 6455 has warts, and given the toolbox we have now it would be done differently - but what in terms of operations would be improved by an update to a h2 world? Would that be a different set of answers for a quic world?
> 
> Here's what I think I'm hearing, but there are so many messages that are done in the weeds of the solution space I don't want to lose track of the problems being solved - I think this list might help in any chartering discussion:
> 
> * in a practical sense there is no mux and when you have mux you need priority and flow control. h2 solves this.
> * operational overhead of maintaining/admin h1 just to boostrap to websockets.
> is there more? some data on this stuff would be good. Is this really mostly about mux?
> 
> I can see where the introduction of quic makes this gap even bigger.. with its lower latency and potential to deploy more modern transport features. It would be great to be more crisp on this - I have read https://github.com/bidiweb/bidiweb-semantics/blob/master/SurveyOfProtocolGaps.md and its still not clear to me what the driving concern is.

In particular, my recollection of the outcome of the discussion of WS in H2 was that a new SETTING or a new ALPN token could be used to indicate that a connection supports both H2 and WS. If there's a problem with doing so, that would be good to talk about as well. Especially considering QUIC.

Cheers,

--
Mark Nottingham   https://www.mnot.net/

---

From: Martin Thomson <[email-masked]>
Date: Fri, 2 Dec 2016 11:19:44 +1100

> In particular, my recollection of the outcome of the discussion of WS in H2 was that a new SETTING or a new ALPN token could be used to indicate that a connection supports both H2 and WS. If there'…

There seems to be some reluctance to exercise that option.  I don't
understand why; I've a bunch of candidate theories, but none of them
make a lot of sense.

---

From: Andy Green <[email-masked]>
Date: Fri, 02 Dec 2016 08:49:22 +0800

>* operational overhead of separate conns for http and ws

Yes these are correct, but they are more optimization opportunities.

>is there more? some data on this stuff would be good. Is this really

For people using ws on http/1, it seems quite strange it's more or less alone in terms of http/1 features widely implemented in browsers NOT provided a way to work on h2.

>From the point of view of this group it's "obviously" a different standard and the focus is, and has been, on moving discussion to someone else's lawn.  But from browser user perspective it's all mixed in and wants to be handled by the same server and protocol, from that perspective there is an impedance mismatch with the feeling here, and it looks to them like either ws is being effectively deprecated, or h2 guys dropped the ball and 'missed it out'.

I understand "that's not how it is" from the perspective of the process of the people here.  But for people not involved that's not an unreasonable take on the situation they will find trying to use ws on h2 right now.

So this should be solved, I think...

-Andy

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 2 Dec 2016 09:56:32 +0900

2016-12-02 9:19 GMT+09:00 Martin Thomson <[email-masked]>:

> > In particular, my recollection of the outcome of the discussion of WS in
> H2 was that a new SETTING or a new ALPN token could be used to indicate
> that a connection supports both H2 and WS. If there's a problem with doing
> so, that would be good to talk about as well. Especially considering QUIC.
>
My understanding is that the cons of using SETTINGS only is that it
requires an additional roundtrip on connection establishment. I've heard
people oppose to the use of ALPN since they want to use both H2 and WS (and
possibly DNS?) on the same connection.

Personally, I think using both SETTINGS (or introducing a new frame) and
ALPN solves the shortcomings (and the reluctance). We could consider ALPN
as a method to specify the application protocol (e.g. HTTP or WS or DNS),
and use SETTINGS for permitting additional protocols to be coalesced.

-- 
Kazuho Oku

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 2 Dec 2016 09:59:30 +0900

2016-12-02 9:56 GMT+09:00 Kazuho Oku <[email-masked]>:

>> > In particular, my recollection of the outcome of the discussion of WS
>> in H2 was that a new SETTING or a new ALPN token could be used to indicate
Please let me correct the last sentence to the following:

We could consider ALPN as a method to specify the application protocol
(e.g. HTTP or WS or DNS) on the first flight, and use SETTINGS for
permitting additional protocols to be coalesced.

"on the first flight" was missing.

-- 

-- 
Kazuho Oku

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 2 Dec 2016 12:00:34 +1100

> On 2 Dec. 2016, at 11:56 am, Kazuho Oku <[email-masked]> wrote:
> 
> 
> 
> My understanding is that the cons of using SETTINGS only is that it requires an additional roundtrip on connection establishment. I've heard people oppose to the use of ALPN since they want to use both H2 and WS (and possibly DNS?) on the same connection.

The semantics of the ALPN token can be "this connection supports H2 *and* WS."

It's true that taken to an extreme, this could lead to a combinatorial explosion. If I remember discussions at the time correctly, it was felt that having some positive pressure on implementations to identify and implement common profiles of protocols to support was a good thing.

> Personally, I think using both SETTINGS (or introducing a new frame) and ALPN solves the shortcomings (and the reluctance). We could consider ALPN as a method to specify the application protocol (e.g. HTTP or WS or DNS), and use SETTINGS for permitting additional protocols to be coalesced.

That's another way to do it too, provided the latency hit isn't critical. Since you've already got the H2 connection open in the typical case for WS, I think that'd work well, but I could be unaware of some use case that requires WS on the first RT.

Cheers,

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 2 Dec 2016 10:13:12 +0900

2016-12-02 10:00 GMT+09:00 Mark Nottingham <[email-masked]>:

> It's true that taken to an extreme, this could lead to a combinatorial
> explosion.

Agreed. While I believe it would be a good idea to have some kind of
mechanism to restrict the use of an H2 connection, a client need to be
forbidden for making an anticipation that the connection can also handle
HTTP requests. We have 421 that can be used as a safe-guard.
(Yes, this is the discussion we had with ORIGIN frame).

If I remember discussions at the time correctly, it was felt that having
> some positive pressure on implementations to identify and implement common
> profiles of protocols to support was a good thing.
>
>
> That's another way to do it too, provided the latency hit isn't critical.
> Since you've already got the H2 connection open in the typical case for WS,
> I think that'd work well, but I could be unaware of some use case that
> requires WS on the first RT.
>
> Cheers,
>
> --

-- 
Kazuho Oku

---

From: Patrick McManus <[email-masked]>
Date: Thu, 1 Dec 2016 20:34:26 -0500

> requires WS on the first RT.

Doing it with settings on alpn=h2 is a little tricky.. if you need to make
a new connection (e.g. you have none open) for wss do you do it with
alpn=h2 and hope for the setting and then fall back to a new connection
with h1 and upgrade if it doesn't work? That's a new pile of round trips
for legacy servers. otoh doing it via alpn makes it easy by offering
wsoverh2 and h1 in the handshake.. it also makes it easy for the h1 server
to use alt-svc to advertise a better version elsewhere as we see quic do.

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 2 Dec 2016 14:53:08 +1300

On 2/12/2016 2:13 p.m., Kazuho Oku wrote:
> 2016-12-02 10:00 GMT+09:00 Mark Nottingham:
> 
>>
>>> On 2 Dec. 2016, at 11:56 am, Kazuho Oku wrote:
>>>
>>>
>>>
>>> 2016-12-02 9:19 GMT+09:00 Martin Thomson:
>>> On 2 December 2016 at 11:09, Mark Nottingham wrote:
> HTTP requests.

Please don't go there. The 'h' in h2 means HTTP by definition.

WS/1 has its own TCP based sockets etc for use by connections without HTTP.

An ALPN for that or for a separate WS/2 protocol spec that references
HTTP/2 mechanisms and framing as a re-use (like ICAP with HTTP/1) should
be sufficient.

But not re-defining h2 to mean non-HTTP-only traffic.

Amos

---

From: Andy Green <[email-masked]>
Date: Fri, 02 Dec 2016 10:33:32 +0800

On Fri, 2016-12-02 at 14:53 +1300, Amos Jeffries wrote:
> > > The semantics of the ALPN token can be "this connection supports
> > > H2 *and*
> > > WS."
> > > 

ws does not exist standalone without http first, since the ws protocol
is negotiated in http before the upgrade and there's no way to do it
after the upgrade.

The actual game here is "provide a transport for JS WS API on h2".  It
does not need to retain any relationship at all to "WS/1" transport
implementation.  Eg as was discussed here a long time ago there may be
a way for the whole overkill 64-bit ws fragment size business to go and
just use h2 framing.

> HTTP/2 mechanisms and framing as a re-use (like ICAP with HTTP/1)

Since there will be servers that support ws/2 and those that don't,
something is needed.  Whatever it is preferably shouldn't force new tcp
connections or roundtrips on ws/2 needlessly.

-Andy

---

From: Patrick McManus <[email-masked]>
Date: Thu, 1 Dec 2016 22:14:34 -0500

> The actual game here is "provide a transport for JS WS API on h2".  It

indeed - but what I'm trying to get to the root of in this thread is what
motivates that. If the game is to get it into somebody's charter. that is
going to need to be crisp. So far I've really only heard 2 strong
motivators (along with a few complementary smaller ones)

1] it is inherently a problem that 6455 can only be done on h1 because it
is left behind the h2 curve. To be honest, this is generally stated as a
given but it isn't obviously true to me. I haven't heard this fleshed out
very convincingly - I tried to give some seeds to help build that argument
in the first message of this thread.. but to my mind there hasn't been a
convincing case made yet that this is a problem (which isn't to say I think
the case can't be made).

2] ws needs mux (and priority and flow control that go with it)  and h2 has
already solved that thorny problem. I buy this if ws needs mux. I supported
mux with Roberto in the hybi days and the wg decided against doing it in
the base version as part of 6455. This seems to be a stronger argument  But
has the lack of mux been a problem in practice for ws? anecdotes or data to
support that?

-P

---

From: Andy Green <[email-masked]>
Date: Fri, 02 Dec 2016 11:43:54 +0800

On Thu, 2016-12-01 at 22:14 -0500, Patrick McManus wrote:
> indeed - but what I'm trying to get to the root of in this thread is
> what motivates that. If the game is to get it into somebody's
> charter. that is going to need to be crisp. So far I've really only
> heard 2 strong motivators (along with a few complementary smaller
> ones)

Charters and such are about the politics of implementing it.  If it
doesn't get implemented, or maybe not implemented until more h2 rollout
amongst the Great Unwashed makes them turn up here asking the same
thing over and over, life will go on.

> 1] it is inherently a problem that 6455 can only be done on h1
> because it is left behind the h2 curve. To be honest, this is
> generally stated as a given but it isn't obviously true to me. I 

It's consistently been the case trying to discuss it here, but it's
still surprising to me.  One major browser feature is orphaned as it is
from h2.

> haven't heard this fleshed out very convincingly - I tried to give
> some seeds to help build that argument in the first message of this
> thread.. but to my mind there hasn't been a convincing case made yet
> that this is a problem (which isn't to say I think the case can't be
> made).
> 
> 2] ws needs mux (and priority and flow control that go with it)  and
> h2 has already solved that thorny problem. I buy this if ws needs
> mux. I supported mux with Roberto in the hybi days and the wg decided
> against doing it in the base version as part of 6455. This seems to
> be a stronger argument  But has the lack of mux been a problem in
> practice for ws? anecdotes or data to support that?

Maybe I'm crazy but although these are advantageous, they are just
optimizations to me.

The basic problem is you can't deploy an h2 server that also does ws,
even in a not very efficient way.  This seems like something failed
somewhere, and one way or another should be enabled.

^--- that's all the "convincing" I plan to do.

-Andy

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 2 Dec 2016 07:35:13 +0100

> to use alt-svc to advertise a better version elsewhere as we see quic do.

But I think that it's important to focus on the future, ie assume support by
default, and lose a round trip in case the assumption was bad. When properly
done, it adds the round trips only for old servers, giving incentive for
them to upgrade, without breaking compatibility.

Ideally we would need to use new frame types to create streams of type WS,
which would be rejected by an RST_STREAM by implementations not supporting
it.

Another possibility would be to define H2.1 which would support extra
protocols by default unless the server explicitly rejects them. That would
be immediately advertised by ALPN and servers would progressively move
towards this one.

Willy

---

From: Loïc Hoguin <[email-masked]>
Date: Fri, 2 Dec 2016 10:34:50 +0100

On 12/01/2016 04:48 PM, Patrick McManus wrote:
> done in the weeds of the solution space I don't want to lose track of
> the problems being solved - I think this list might help in any
> chartering discussion:
>

Multiplexing is interesting but probably a special case that few people 
require, although I can imagine people using it "because they can".

What is more interesting is the flow control (regardless of mux). While 
you can implement a backpressure mechanism on the server by not reading 
from the socket, the flow control allows you to prevent the client or 
the server from sending anything at all, saving resources for other 
connections. This would be a pretty good improvement for RabbitMQ's 
STOMP/MQTT over Websocket, for example.

Ultimately it's just optimizations I think.

WiSH is very interesting because it provides this at minimal costs: same 
framing as Websocket (therefore, same code), existing connection 
handling from H2, HTTP semantics for request routing, methods, content 
negotiation. It could even be used over HTTP/1 should one wish to do so.

I don't think WiSH requires any updates to HTTP to work.

I would be wary of solutions that add new H2 frames, as they require yet 
another implementation. WiSH would work as long as the HTTP semantics 
don't change, and these are shared between HTTP/1 and H2. It's a more 
future proof solution IMO.

-- 
Loïc Hoguin
https://ninenines.eu

---

From: Cory Benfield <[email-masked]>
Date: Fri, 2 Dec 2016 11:33:45 +0000

> On 2 Dec 2016, at 03:43, Andy Green <[email-masked]> wrote:
> 

I assume that what you mean here is you can’t deploy a H2 *only* server that also does WS: that is, a server with no HTTP/1.1 stack.

To which I reply: so what? Last I looked no-one was deploying servers that can *only* do HTTP/2 except in very specific cases where they are deliberately seeing the HTTP/2 use-cases (the only two instances I know of are Apple’s new Push Notification Service and Amazon’s Alexa API, both of which are HTTP/2 only: presumably they considered and rejected the use of WS, and it didn’t stop them shipping their product).

I don’t think anyone is planning to move to a HTTP/2-only server stack anytime soon, and we have a whole bunch of servers that have mature and battle-tested HTTP/1.1 stacks that aren’t going anywhere. So I’m not really convinced that there’s any demand for H2-only + WS. Of course, I might be wrong (I’m wrong a lot).

Cory

---

From: Jacob Champion <[email-masked]>
Date: Fri, 2 Dec 2016 09:36:38 -0800

On 12/02/2016 03:33 AM, Cory Benfield wrote:
> I assume that what you mean here is you can’t deploy a H2 *only*
> server that also does WS: that is, a server with no HTTP/1.1 stack.
>
> To which I reply: so what? Last I looked no-one was deploying servers
> that can *only* do HTTP/2 except in very specific cases where they
> are deliberately seeing the HTTP/2 use-cases (the only two instances
> I know of are Apple’s new Push Notification Service and Amazon’s
> Alexa API, both of which are HTTP/2 only: presumably they considered
> and rejected the use of WS, and it didn’t stop them shipping their
> product).
>
> I don’t think anyone is planning to move to a HTTP/2-only server
> stack anytime soon, and we have a whole bunch of servers that have
> mature and battle-tested HTTP/1.1 stacks that aren’t going anywhere.
> So I’m not really convinced that there’s any demand for H2-only + WS.
> Of course, I might be wrong (I’m wrong a lot).

I think this ends up being a circular argument. If people rely on both 
HTTP/2 and WS, but WS requires HTTP/1.1, then they will not demand an 
HTTP/2-only stack. Likewise with the Amazon and Apple examples: they're 
going to ship what works; it doesn't necessarily mean the engineers 
didn't want something different. (Perhaps there are employees watching 
the list who can chime in?)

This is not a server-only thing, either. A conforming WebSocket client 
has to implement enough of HTTP/1.1 to handle, or at least correctly 
bail out from, any pre-upgrade shenanigans such as 3xx redirects, 401 
authentications, Set-Cookies, etc.

(For the record, my background is primarily in the embedded security 
space, where opportunities to consolidate and reduce software footprint 
-- and attack surface -- are pretty much always welcomed.)

--Jacob

---

From: Jacob Champion <[email-masked]>
Date: Fri, 2 Dec 2016 09:52:18 -0800

On 12/01/2016 07:14 PM, Patrick McManus wrote:
>
> On Thu, Dec 1, 2016 at 9:33 PM, Andy Green <[email-masked]
> <mailto:[email-masked]>> wrote:
>
> indeed

I, for one, would be pretty disappointed if that were the case. Last I 
checked, the Javascript WS API still only provided a subset of 6455, and 
browsers are not the only clients I care about.

I want the "actual game" to be "provide a transport over h2 that 
provides the same semantics as the WebSocket protocol", so that existing 
clients of WS subprotocols can be switched over to the new transport 
without modifications to their business logic.

--Jacob

---

From: Patrick McManus <[email-masked]>
Date: Fri, 2 Dec 2016 13:02:29 -0500

On Fri, Dec 2, 2016 at 12:52 PM, Jacob Champion <[email-masked]>
wrote:

> so that existing clients of WS subprotocol

you had me at subprotocol.

which one(s)? Pointers to definitions? It would be interesting to see how
well that mechanism took off - mostly I've seen it used for versioning.

but secondarily, sub-protocol negotiation is part of the js api so I
probably need a little clarification about what you have in mind as the gap
you objected to wrt subprotocol. Thanks.

---

From: Jacob Champion <[email-masked]>
Date: Fri, 2 Dec 2016 10:36:56 -0800

On 12/02/2016 10:02 AM, Patrick McManus wrote:
>
> On Fri, Dec 2, 2016 at 12:52 PM, Jacob Champion <[email-masked]
> <mailto:[email-masked]>> wrote:
>
> which one(s)? Pointers to definitions? It would be interesting to see
> how well that mechanism took off - mostly I've seen it used for versioning.

If you mean the registered list:

     https://www.iana.org/assignments/websocket/websocket.xml

It's grown a bit since last I looked. Twenty-five or so?

> probably need a little clarification about what you have in mind as the
> gap you objected to wrt subprotocol. Thanks.

Right, the negotiation itself wasn't so much what my fear was about. 
I'll need to dig the specs out again, but the two (admittedly minor) 
gaps that I remember right now are application-data pings and 
frame-by-frame message streaming. There might be others.

(Since frame boundaries don't have *semantic* meaning in WS, the major 
thing to keep for WS/2 is the ability to send control frames in the 
middle of a very large message, which IIUC the WS API does not currently 
allow. As for pings, the data reflected by the server could allow 
clients to perform rudimentary latency monitoring or something else 
clever at the application level, perhaps for clock synchronization...?)

For the record, I'm not currently aware of a *specific* subprotocol that 
would break using only the WS API subset. That would be better answered 
by the HyBi experts, I think.

--Jacob

---

From: Andy Green <[email-masked]>
Date: Sat, 03 Dec 2016 03:54:47 +0800

>frame-by-frame message streaming. There might be others.

h2 has a PING concept at connection (not stream) level... differences are fixed 8-byte payload instead of up to 126 bytes, and no requirement in h2 that the PONG payload is identical.

I've only ever seen ws PING used as a 'communication still alive' probe, but actually h2 connectivity PING and a PING in the muxed ws2 stream would be semantically different aside from the actual differences.  A PING in the stream can also tell you the ws2 server is still alive, not just the connection to the h2 endpoint.  Ws PING does tell you the ws peer is also up.

So you're probably right ws2 explicit PING is also needed.

>middle of a very large message, which IIUC the WS API does not

Nah... if it's sharing an h2 connection, he's just one muxed stream.  And ws specifies any intermediary may refragment ws frames anyway.  Individual ws1 frames have no guarantee of making it to the peer atomically.

Of course ws (see RFC6455 5.4) explicitly "allows" "send[ing] control frames in the middle of a very large message"... a ws message comprises any number of frames of any legal length.  Inbetween the message fragment frames you may send control frames.

So that one is just wrong I think.

>clever at the application level, perhaps for clock synchronization...?)

I never saw ws PING data like that; they used their protocol if they had such things.  It's used purely a probe in my experience.

But I think you don't need to make that kind of reach... the ability to confirm the peer who understands ws2 is still up for that stream, like ws PING does, instead of just the h2 endpoint is enough to also require explicit ws2 PINGs as you suggested.

But that's all, over the JS API...

-Andy

---

From: Jacob Champion <[email-masked]>
Date: Fri, 2 Dec 2016 12:18:06 -0800

> Nah... if it's sharing an h2 connection, he's just one muxed stream.  And ws specifies any intermediary may refragment ws frames anyway.  Individual ws1 frames have no guarantee of making it to the…

Correct. That's what I meant by "frame boundaries don't have semantic 
meaning"; sorry if that was unclear.

> Of course ws (see RFC6455 5.4) explicitly "allows" "send[ing] control frames in the middle of a very large message"... a ws message comprises any number of frames of any legal length.  Inbetween th…

My point is just that this ability to multiplex control frames inside of 
in-flight messages is not something that is explicitly exposed by the JS 
API, but it may be something that a (non-browser) client requires for 
proper operation. I think WS/2 should still support it, regardless of 
whether or not a JS client can make use of it.

Let me step back: when you say that your goal is to "provide a transport 
for JS WS API on h2", my fear is that this could lead to a situation 
where semantics that are part of WS/1 are removed from WS/2 for no 
reason other than "Javascript clients don't need it, so no one else does 
either." I would like to avoid that.

> I never saw ws PING data like that

I have. So... there's that. :D In my particular case, it didn't make it 
into production (as far as I know), but I've *seen* it.

--Jacob

---

From: Andy Green <[email-masked]>
Date: Sat, 03 Dec 2016 04:58:16 +0800

On December 3, 2016 4:18:06 AM GMT+08:00, Jacob Champion <[email-masked]> wrote:
>> Nah... if it's sharing an h2 connection, he's just one muxed stream. 
>And ws specifies any intermediary may refragment ws frames anyway. 
>Individual ws1 frames have no guarantee of making it to the peer
>atomically.
>
>> Of course ws (see RFC6455 5.4) explicitly "allows" "send[ing] control
>frames in the middle of a very large message"... a ws message comprises
>any number of frames of any legal length.  Inbetween the message
>fragment frames you may send control frames.
>

OK.  Although the only relevant control frames are PING / PONG.  And if a client wants to send control frames inside a huge message, that client must have explicitly fragmented the message already.

The general idea is just map ws2 payload frames direct to h2 framing... refragmenting them to fit.  In that way, 'supporting' ws1 63-bit frame lengths compatibly doesn't require 63-bit frame lengths in ws2 because ws always allows refragmentation of frames.

So it only creates more fragmentation / opportunities to insert control frames, so no problem.

>either." I would like to avoid that.

What I meant by that is ws1 wire protocol can go out the window completely.  The job is not wrap ws1 verbatim in h2 frames, keep ws1 negotiation headers, masking, etc.  It can be radically recast to align with h2 while following the JS API, and fully exploit new possibilities like roundtrip elimination.

I agree it should make some effort to not break non JS / browser uses.  But it's no coincidence there are only a tiny number of corner cases about that -- ws1 was itself designed to implement a transport for the ws JS API.

-Andy

---

From: Jacob Champion <[email-masked]>
Date: Fri, 2 Dec 2016 13:41:46 -0800

On 12/02/2016 12:58 PM, Andy Green wrote:
> OK.  Although the only relevant control frames are PING / PONG.

CLOSE too, plus any control frames added to the protocol between now and 
the release of WS/2.

> And if a client wants to send control frames inside a huge message, that client must have explicitly fragmented the message already.
>

Agreed.

> I agree it should make some effort to not break non JS / browser uses.  But it's no coincidence there are only a tiny number of corner cases about that -- ws1 was itself designed to implement a tra…

It sounds like we're on the same page, as long as the eventual 
solution's authors understand those corner cases, and that the 
functionality provided by WebSocket is (to a minor extent) a superset of 
what's provided by the JS API. In particular, I agree that we don't 
necessarily need to be bound by the current wire format, or the same 
HTTP-buster security features, as WS/1.

--Jacob

---

From: Van Catha <[email-masked]>
Date: Sun, 4 Dec 2016 19:59:11 -0500

I do not see the need for Ping, Pong or Close ws frames. The h2/quic
transport layer handles this. If you want to measure latency then send your
own pings/pongs, do not bake it into the protocol.  There is no need for
this, and if there is, please present a compelling reason.

The clientside API should not change, but it should definitely be extended
OR more functionality should be added to browsers, like the ability to
compress data from inside JS land.

On Fri, Dec 2, 2016 at 4:41 PM, Jacob Champion <[email-masked]> wrote:

> And if a client wants to send control frames inside a huge message, that
>> client must have explicitly fragmented the message already.
>>
>> The general idea is just map ws2 payload frames direct to h2 framing...
>> refragmenting them to fit.  In that way, 'supporting' ws1 63-bit frame
>> lengths compatibly doesn't require 63-bit frame lengths in ws2 because ws
>> always allows refragmentation of frames.
>>
>
> Agreed.
>
>> What I meant by that is ws1 wire protocol can go out the window
>> completely.  The job is not wrap ws1 verbatim in h2 frames, keep ws1
>> negotiation headers, masking, etc.  It can be radically recast to align
>> with h2 while following the JS API, and fully exploit new possibilities
>> like roundtrip elimination.
>>
>> I agree it should make some effort to not break non JS / browser uses.
>> But it's no coincidence there are only a tiny number of corner cases about
>> that -- ws1 was itself designed to implement a transport for the ws JS API.
>>
>
> It sounds like we're on the same page, as long as the eventual solution's
> authors understand those corner cases, and that the functionality provided
> by WebSocket is (to a minor extent) a superset of what's provided by the JS
> API. In particular, I agree that we don't necessarily need to be bound by
> the current wire format, or the same HTTP-buster security features, as WS/1.
>
> --Jacob
>
>

---

From: Andy Green <[email-masked]>
Date: Mon, 05 Dec 2016 11:02:59 +0800

On Sun, 2016-12-04 at 19:59 -0500, Van Catha wrote:

> transport layer handles this. If you want to measure latency then
> send your own pings/pongs, do not bake it into the protocol.  There
> is no need for this, and if there is, please present a compelling
> reason.

He's making a slightly different point... there are some corner cases
about ws1 RFC6455 protocol that are not grounded in the JS API,
although almost everything else is.

1) ws1 PING / PONG for example aren't exposed in the JS API at all.

But they are in wide use by non-browser clients, who at intervals want
to confirm their connection to the ws server is still live if it's
idle.

The meaning of ws1 PING is different than the h2 link PING, which is
defined to not be able to have a specific stream context but only
connectionwide.  For ws1 it means the websockets server (which may be a
different process or on a different machine than the h2 endpoint) still
acknowledges the ws1 connection as alive and communication in both ways
on the stream is working.  For h2 it means the h2 server is still
connected at the stream bundle level.  They're not actually giving you
the same assurance.

2) Jacob points out there is implicit serialization in ws1, a fragment
is atomic, blocking PING, PONG and CLOSE until it is all sent.  Again
it's not explicit in the JS API.  But again he's suggesting that
behaviour should be maintained in a ws2.

These are at least reasonable-to-argue points.

> The clientside API should not change, but it should definitely be
> extended OR more functionality should be added to browsers, like the
> ability to compress data from inside JS land.

I dunno.  I implemented permessage-deflate nicely, with good control
over its memory and streaming fixed-size buffers in and out, it can
pass Autobahn, but personally I have never had (or even seen) a good
use for it.  I know some people love it... better to discuss with them
rather than me...

-Andy

> > > > My point is just that this ability to multiplex control frames
> > > > inside
> > > > of
> > > > in-flight messages is not something that is explicitly exposed
> > > > by the
> > > > JS
> > CLOSE too, plus any control frames added to the protocol between
> > now and the release of WS/2.
> > 
> > > And if a client wants to send control frames inside a huge
> > > message, that client must have explicitly fragmented the message
> > > already.
> > > 
> > > The general idea is just map ws2 payload frames direct to h2
> > > framing... refragmenting them to fit.  In that way, 'supporting'
> > > ws1 63-bit frame lengths compatibly doesn't require 63-bit frame
> > > lengths in ws2 because ws always allows refragmentation of
> > > frames.
> > > 
> >  
> > Agreed.
> > 
> > > > reason other than "Javascript clients don't need it, so no one
> > > > else
> > > > does
> > > completely.  The job is not wrap ws1 verbatim in h2 frames, keep
> > > ws1 negotiation headers, masking, etc.  It can be radically
> > > recast to align with h2 while following the JS API, and fully
> > > exploit new possibilities like roundtrip elimination.
> > > 
> > > I agree it should make some effort to not break non JS / browser
> > > uses.  But it's no coincidence there are only a tiny number of
> > > corner cases about that -- ws1 was itself designed to implement a
> > > transport for the ws JS API.
> > > 
> >  
> > functionality provided by WebSocket is (to a minor extent) a
> > superset of what's provided by the JS API. In particular, I agree
> > that we don't necessarily need to be bound by the current wire
> > format, or the same HTTP-buster security features, as WS/1.
> > 
> > --Jacob
> > 
> > 
> 
>

---

From: Van Catha <[email-masked]>
Date: Sun, 4 Dec 2016 22:23:40 -0500

> connected at the stream bundle level.  They're not actually giving you

That is the responsibility of the h2 endpoint (if h2 is proxing/forwarding
the request to another server).

I do not see why this should be the responsibility of ws.  I see your
point, but I see it as out of scope.

---

From: Andy Green <[email-masked]>
Date: Mon, 05 Dec 2016 11:58:16 +0800

On Sun, 2016-12-04 at 22:23 -0500, Van Catha wrote:
> That is the responsibility of the h2 endpoint (if h2 is
> proxing/forwarding the request to another server).
> 

Ws1 already does that and it's in wide use.

Telling people to use the mux ping on h2 will be a departure in
functionality from ws1.  And depending how its put together, the guy
that binds streams in h2 may not have a way for the child streams to
request / see / manage delivery of h2-connection-level h2 PING/PONGs
that are already defined to not have any stream index other than zero.

It's a valid topic for discussion as to whether enough people care or
not to define a way to get the same functionality in ws2...

> point, but I see it as out of scope.

...ws1 found it "in scope".

-Andy

> > > On Fri, Dec 2, 2016 at 4:41 PM, Jacob Champion <champion.p@gmail.
> > com>
> > > wrote:
> > > > > On December 3, 2016 4:18:06 AM GMT+08:00, Jacob Champion
> > <champio
> > > > > [email-masked]> wrote:
> > > > > So it only creates more fragmentation / opportunities to
> > insert
> > > > > control frames, so no problem.
> > > > >
> > > > > > Let me step back: when you say that your goal is to
> > "provide a
> > > > > > transport
> > > > > > where semantics that are part of WS/1 are removed from WS/2
> > for
> > > > > > no

---

From: Van Catha <[email-masked]>
Date: Mon, 5 Dec 2016 10:56:52 -0500

> not to define a way to get the same functionality in ws2...

Agreed here is my take on it.

WS1 was batteries included.  PING/PONG also was critical as you could have
a zombie socket
(send does not fail, but data not delivered to endpoint). If you send ping
from serverside you can
destroy the zombie clients if no pong comes.  BUT what WS1 did not consider
is you could have a zombie socket from
client->server, WS1 did not expose any API to check ping/pong this way.
There is no onping/onpong
callback, nor is there a ws.send_ping function in browser side js.

So essentially the Ping/Pong baked into WS1 was half useless and you had to
roll your own pings/pongs ontop of
the WS1 transport layer.

Ping / Pong to ensure an endpoint is alive is crucial but it needs to be
sent from the client, WS1 browser
client API never exposed this.

So to repeat making the mistakes of the past we have two options. #1 Take
ping out altogether as its broken, no browser
client implemented the ability to send pings to server. #2 Extend WS
browser API to allow sending ping/pong.

Now enter H2 and QUIC, which has their own transport layer that guarantee
an endpoint will be alive. If not you will get
notified a stream is dead.  If the h2 is running in a distribution, it is
up to the internals of h2 to determine if a distributed node
in the system died, and to notify the client the connection was closed on
those streams.

The case of ping is important to not have to wait a certain timeout for h2
to determine a distributed node died and notify the client.
The case of ping is important to be re-enforced you are connected.

The simplest way I see is to make websocket protocol agnostic, its just a
binary stream, and let userland handle the protocol (pings, pongs etc).
What if one user wants to trammit a ticktime in the ping, etc.  Its simply
not flexible enough to be baked in.

If one is to consider baking in pinging just to determine if the h2 stream
going to your Websocket APP is not flooded due to your apps slow
consumption, they should not burden the protocol with personal problems.

---

From: Jacob Champion <[email-masked]>
Date: Mon, 5 Dec 2016 10:10:36 -0800

On 12/05/2016 07:56 AM, Van Catha wrote:
> BUT what WS1 did not consider is you could have a zombie socket from

This is not correct. WebSocket, as a protocol, is distinct from the JS 
API. WebSocket, as a protocol, has client->server pings. And there are 
WebSocket client implementations that expose them. Autobahn|Python, for 
example.

> So to repeat making the mistakes of the past we have two options. #1
> Take ping out altogether as its broken, no browser
> client implemented the ability to send pings to server.

The fact that browsers did not see fit to use a part of the protocol 
does not mean that the protocol is "broken", IMHO. The JS API is not 
WebSocket; please don't conflate the two.

Now: perhaps your goal is that WS/2 should be an incompatible upgrade 
that streamlines and removes redundancies between HTTP/2 and WebSocket? 
If so, I think that's a reasonable thing to propose. I just disagree 
with it; I think the protocol is more likely to succeed if existing 
WebSocket implementations can transparently switch between WS/1 and WS/2 
transports as needed.

--Jacob

---

From: Jacob Champion <[email-masked]>
Date: Mon, 5 Dec 2016 10:34:28 -0800

> is there more? some data on this stuff would be good. Is this really

I've contributed to a major derailing of the thread elsewhere. My 
apologies... To try to bring this back to your original point:

Andy mentioned that an HTTP/2 transport for WebSocket might mean that we 
could get rid of client-to-server masking. I don't have any data to 
support my spitballing, but that *could* be a pretty decent optimization 
for implementations, since they no longer have to mask or de-mask the 
data in place. For example, it might open up the possibility of 
scatter/gather I/O for frame handling?

--Jacob

---

From: Van Catha <[email-masked]>
Date: Mon, 5 Dec 2016 15:43:17 -0500

> This is not correct. WebSocket, as a protocol, is distinct from the JS
API. WebSocket, as a protocol, has client->server pings. And there
> are WebSocket client implementations that expose them. Autobahn|Python,
for example.

Yea this is true, but when running over H2/QUIC it kind of loses its
luster.  WebSocket works well when you need to connect to a backend HTTP
server say for a mobile app or non-browser app.  Then you can use some
quality client libraries.

> I think the protocol is more likely to succeed if existing WebSocket
implementations can transparently switch between WS/1 and
> WS/2 transports as needed.

The WiSH RFC was proposed that covered exactly this, there has been a lot
of discussion archived in the mailing related to it.  A lot of the
questions being brought up now have already been answered.  Specifically
related how to keep interop between WS1 while layering on top of H2.

> Andy mentioned that an HTTP/2 transport for WebSocket might mean that we
could get rid of client-to-server masking. I don't have any data
> to support my spitballing, but that *could* be a pretty decent
optimization for implementations, since they no longer have to mask or
> de-mask the data in place. For example, it might open up the possibility
of scatter/gather I/O for frame handling?

See answer above and check out WiSH. This has already been discussed to
great detail.

These discussions would be more productive if we can all get on the same
page with what was already covered in great detail.

https://tools.ietf.org/html/draft-yoshino-wish-00

Masking is not needed anymore, if anyone as knowledgeable in the matter as
Yoshino can suggest why it is, this mailing list is all ears.

WiSH drops ping/pong which is debatable, sure.

> perhaps your goal is that WS/2 should be

WS/2 or simply 2 way binary streaming to drop the WS association.  A basic
method to transmit binary data between two opted in endpoints.

On Mon, Dec 5, 2016 at 1:34 PM, Jacob Champion <[email-masked]> wrote:

> could get rid of client-to-server masking. I don't have any data to support
> my spitballing, but that *could* be a pretty decent optimization for
> implementations, since they no longer have to mask or de-mask the data in
> place. For example, it might open up the possibility of scatter/gather I/O
> for frame handling?
>
> --Jacob
>
>

---

From: Jacob Champion <[email-masked]>
Date: Mon, 5 Dec 2016 13:35:49 -0800

On 12/05/2016 12:43 PM, Van Catha wrote:
> The WiSH RFC was proposed that covered exactly this, there has been a
> lot of discussion archived in the mailing related to it.  A lot of the

My limited understanding of WiSH is that it explicitly drops some of the 
WebSocket semantics in favor of a more general set of bidirectional 
communication primitives. That's cool. It's also not what I'm advocating 
for.

That said, I haven't been following the WiSH effort as much as I should, 
and the -01 draft is too sparse for me to draw any real conclusions 
from. If you have a pointer to the conversation thread you're referring 
to, so that I can read up, that'd be much appreciated!

>> Andy mentioned that an HTTP/2 transport for WebSocket might mean that
> we could get rid of client-to-server masking. I don't have any data
>> de-mask the data in place. For example, it might open up the
> possibility of scatter/gather I/O for frame handling?
>

I thought the purpose of Patrick's original thread was to consolidate a 
list of concrete reasons for WebSocket to be ported to a new transport. 
If I've misunderstood that, I apologize for the noise.

> WS/2 or simply 2 way binary streaming to drop the WS association.  A
> basic method to transmit binary data between two opted in endpoints.

In that case, your goal and my goal are different. That's fine -- it 
doesn't mean your goal is wrong -- but having you and I debate here, as 
if we had the *same* endgame in mind, is going to be confusing and not 
very helpful for many people, I think.

--Jacob

---

From: Martin J. Dürst <[email-masked]>
Date: Tue, 13 Dec 2016 17:47:13 +0900

Sorry to revive an old thread.

> convincing case made yet that this is a problem (which isn't to say I think

My understanding is that H1 is, despite keep-alive and pipelining and 
all that, essentially request-response-done, whereas H2 is designed for 
long(er) term relationships. WS also is intended for longer-term 
relationships. So WS and H2 fit together much better than WS and H1. Put 
in a different way, if we had the choice between basing WS on H1 and 
basing it on H2, it would be a slam dunk for H2.

> has the lack of mux been a problem in practice for ws? anecdotes or data to

I sure also would like to see some data. But then, has the lack of mux 
in H1 been a problem? If no, why is it in H2? If yes, why is it a 
problem for HTTP, but not for WS?

Regards,   Martin.

---

From: Patrick McManus <[email-masked]>
Date: Tue, 13 Dec 2016 11:22:53 -0500

On Tue, Dec 13, 2016 at 3:47 AM, Martin J. Dürst <[email-masked]>
wrote:

> My understanding is that H1 is, despite keep-alive and pipelining and all
> that, essentially request-response-done,

H1 is request response, but as used in 6455 that request response is simply
used to bootstrap into a totally different protocol that has control of the
TCP channel at which point its request/response properties are not germane.

>
>>
> I sure also would like to see some data. But then, has the lack of mux in
> H1 been a problem?

yes it is the most serious, imo, problem with h1. Trying to attack it with
an increase in H1 parallelism leads to congestion control and priority
sub-optimalities, and trying to work around it with pipelines creates
unworkable head of line problems. mux and priority were the answers to that
and they are really the key features of h2.

> If no, why is it in H2? If yes, why is it a problem for HTTP, but not for
> WS?
>
>
It could be a problem for ws - but the advocates for the work have not
embraced that argument. Differences in workloads might be the
differentiator. dunno. that's why I started the thread :)

---

From: Jacob Champion <[email-masked]>
Date: Tue, 13 Dec 2016 09:44:04 -0800

On 12/13/2016 08:22 AM, Patrick McManus wrote:
> On Tue, Dec 13, 2016 at 3:47 AM, Martin J. Dürst <[email-masked]
> <mailto:[email-masked]>> wrote:
>     If no, why is it in H2? If yes, why is it a problem for HTTP, but
>     not for WS?
>

It might be hard(er) to find people who feel like they *need* mux... my 
guess is that since the one-connection-per-subprotocol rule is absolute 
for WS/1, anyone who wants to scale up to large numbers of clients will 
have already consolidated their application concerns into a single 
subprotocol to avoid connection overhead.

Mux would open up new architectures -- now, if you need to use four 
separate subprotocols, you can just use them; you don't need to merge 
them and reinvent the wheel out of fear of exhausting TCP slots. 
(Obviously mux isn't a magic bullet for performance, as evidenced by the 
priority/weighting stuff, but I think it would change things up 
significantly.)

--Jacob

---

From: Wenbo Zhu <[email-masked]>
Date: Thu, 15 Dec 2016 08:39:30 -0800

On Tue, Dec 13, 2016 at 8:22 AM, Patrick McManus <[email-masked]>
wrote:

> H1 is request response, but as used in 6455 that request response is
> simply used to bootstrap into a totally different protocol that has control
> of the TCP channel at which point its request/response properties are not
> germane.
>
>
>>
>>>

The lack of mux is really the only major problem, but mostly for browsers
(multiple tabs of the same page) as I understand. I guess service workers
could help but it would then require app-level multiplexing.

On non-browsers, I wonder how often a single process needs open more than
one long-lived session to the same host. And for clients with only a single
session, TCP mux adds overhead too but that's a separate topic.

Our biggest motivation behind WiSH is to simplify the Web (infrastructure,
frameworks ...). As mentioned in the bidi-web survey doc, as far as apps
are concerned, the only thing that is missing from http/* to support WS
APIs is really just message framing. And if you need mux and/or on browsers
(same-origin resources too), sure, do http/2 or quic.

And thanks for starting the thread.

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 23 Dec 2016 09:40:58 -0500

Sorry to perpetuate this thread, but...

On 15 Dec. 2016, at 11:39 am, Wenbo Zhu <[email-masked]> wrote:
> On non-browsers,

*confused*

AIUI the *only* reasonable use case for WebSockets is browsers; who is using it without a browser?

(keeping in mind that WS is effectively TCP constrained by the Web security / same-origin model)

Cheers,

---

From: Loïc Hoguin <[email-masked]>
Date: Fri, 23 Dec 2016 16:28:24 +0100

> (keeping in mind that WS is effectively TCP constrained by the Web security / same-origin model)

The three bigger Websocket users I have seen are:

* browsers;
* phone applications;
* servers

Excluding Websocket sub protocols, generally the pattern is the same: a 
server has an HTTP API and some of that API is implemented using 
Websocket to allow the server to push data to the client.

This usage can arguably be replaced by HTTP/2 with push. It is closer in 
spirit to text/event-stream, while also allowing binary data without 
using base64 on that data beforehand. It's not TCP but rather a more 
generalized event push mechanism than text/event-stream provides.

Most of the cases I have seen the usage is much lighter than HTTP/2 push 
provides; there's no cache involved, and the intent is to send the event 
to the client as soon as possible (ie without having to go through the 
push promise and response headers first).

A Websocket over HTTP/2 would allow those users to use the HTTP and 
Websocket API over a single connection.

Servers that use Websocket to speak to each other simply use the same 
interface other clients do. For servers it's more about code reuse than 
any other considerations. Developer implements HTTP/Websocket server for 
phone apps; writes a client module for testing this server; uses the 
client module to speak to other servers.

The same-origin only seems useful for browsers in practice, I have not 
seen it applied elsewhere.

-- 
Loïc Hoguin
https://ninenines.eu

---

From: Jacob Champion <[email-masked]>
Date: Fri, 23 Dec 2016 09:11:31 -0800

> Websocket to allow the server to push data to the client.

+1. This can be extended past phone applications, to any 
embedded/headless target. If you already have an HTTP+WS API in place 
for browsers and you want e.g. your embedded sensor to speak to it too, 
just use one of the many standalone WebSocket client libraries (Python, 
C++, whatever) and have the server point your target to the WS endpoint.

I think Crossbar (from the same people who made the Autobahn|TestSuite, 
which many WS implementations validate themselves with) is a concrete 
example of an architecture built on top of WebSocket but partially 
targeted at IoT embedded clients.

> seen it applied elsewhere.

I haven't either. IIRC, RFC 6455 calls out that Origin isn't necessary 
for non-user-agents, since those clients are acting on their own behalf.

--Jacob