Subject: Re: HTTP/2 and Websockets

From: Yutaka Hirano <[email-masked]>
Date: Wed, 3 Dec 2014 22:37:48 +0900

>
> So failing that, our best solution is to pick whatever hack we have to
> sooner rather than later so that we can at least hammer in special case
> handling for websockets into intermediaries and hope that no new semantics
> emerge.

+1

On Thu, Nov 27, 2014 at 8:53 AM, Greg Wilkins <[email-masked]> wrote:

>
> On 26 November 2014 at 08:25, Roberto Peon <[email-masked]> wrote:
>
>> Sadly, since it was something that a few of us in the WG were pushing
>> for, as it would have allowed intermediaries without knowledge of protocols
>> to do what was right for them.
>> That is why I truly hope that the WS extension happens (and quickly)--
>> I'd love to see loadbalancers relieved of having to introspect into the
>> data further than the HTTP2 frame layer in order to figure out how to deal
>> with it reasonably.
>>
>
> +1
>
> Non-constructive comments from me are that this is simply way too late in
> the process for this to be considered and I suspect that with the framing
> layer we have given ourselves, all possible solutions will now be pretty
> ugly.
>
> It comes down to a choice between:
>
> a) defining new frame types for WS.   Which is going to require very
> explicit support from intermediaries (imagine if we had to do that in TCP
> for every new protocol!).
>
> b) reusing the existing HEADER/DATA frames, but they miss some useful
> semantics and have been so conflated with HTTP semantics that we are back
> to websockets pretending to be HTTP - which is exactly the sort of protocol
> mis-use that we were chartered to resolve!
>
> Currently I can't pick between these two ugly ducklings.       We long ago
> missed the opportunity of coming up with a truly multiple protocol web
> framing layer that would support diverse semantics such as HTTP, websocket
> and whatever the future may bring.    So failing that, our best solution is
> to pick whatever hack we have to sooner rather than later so that we can at
> least hammer in special case handling for websockets into intermediaries
> and hope that no new semantics emerge.
>
> Will try to find the time in the next week to experiment with some of the
> options, so we can be a bit more constructive about which we prefer.
>
> regards
>
>
> --
> Greg Wilkins <[email-masked]>  @  Webtide - *an Intalio subsidiary*
> http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that
> scales
> http://www.webtide.com  advice and support for jetty and cometd.
>

---

From: Yutaka Hirano <[email-masked]>
Date: Thu, 4 Dec 2014 00:49:44 +0900

Roberto,

a mechanism (HEADERS would do if it can be flow controlled on the one
> stream) to modify metadata

Sorry I'm not sure what it means: Can you explain?

On Sat, Nov 29, 2014 at 1:51 PM, Roberto Peon <[email-masked]> wrote:

> All it needs is a new DATA frame which allows the signaling of
> end-of-message (with a single bit), and a mechanism (HEADERS would do if it
> can be flow controlled on the one stream) to modify metadata
>
> Everything else is there in the protocol already.
>
> -=R
>
> On Fri, Nov 28, 2014 at 8:26 PM, Amos Jeffries <[email-masked]>
> wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 27/11/2014 1:05 p.m., Andy Green wrote:
>> >
>> >
>> > On 26 November 2014 17:03:55 GMT+08:00, Amos Jeffries wrote: On
>> > 26/11/2014 2:43 p.m., Andy Green wrote:
>> >>>>
>> >>>>
>> >>>> On 26 November 2014 09:39:37 GMT+08:00, Yutaka Hirano wrote:
>> >>>>> Sorry that the proposal was updated in another place:
>> >>>>>
>> >
>> https://github.com/yutakahirano/ws-over-http2/blob/master/ws-over-http2-message-mapping.md
>> >>>>>
>> >>>>>
>> >
>> >
>> I forgot to link the document.
>> >>>>>
>> >>>>> In the github document, a WS message is encoded to HEADERS
>> >>>>> + DATAs.
>> >>>>
>> >>>> OK... then I guess we could agree, calling it super
>> >>>> inefficient was being kind ^^
>> >>>>
>> >>>> What we're talking about now, HEADERS to make the upgrade one
>> >>>> time and then pure DATA representing one ws frame, would be
>> >>>> perfect. But I don't know how to get rid of the WS_FLAGS byte
>> >>>> in the payload right now to get there.
>> >>>>
>> >
>> > andy, why are you still talking about "upgrade" ?
>> >
>> >> Because we will indeed be upgrading the default http2 protocol
>> >> for the stream to ws.
>>
>> But the stream does not exist as a HTTP/2 stream until its started as
>> a WS stream. So I think there is a paradigm shift needed in the thinking.
>>
>>
>> >
>> >> So we need to signal to intermediaries that we're changing the
>> >> rules for that stream.
>>
>> Opening a WS stream does that.
>>
>> >
>> >> If the intermediary thinks... "oho... I don't think so" then he
>> >> can NAK the upgrade right there and avoid a nasty situation the
>> >> endpoints wrote a cheque that the intermediaries cannot cash.
>> >
>>
>> If the intermediary advertised WS support in its SETTINGS previously
>> then it has already guaranteed that it will understand and properly
>> handle the WS streams.
>>
>> So...
>>
>> >
>> > moving the entire underlying protocol connection from HTTP/1 to WS
>> > was a HTTP/1 artifact that should no longer be necessary at all in
>> > HTTP/2. Just open a new WS-type stream to a peer advertising WS
>> > support and presto it contains WS protocol.
>> >
>>
>>
>> Overall I am thinking that WS just needs 1) a WSHEADERS frame type to
>> initiate WS streams, and 2) a SETTINGS parameter to advertise support
>> for WSHEADERS streams.
>>  DATA contains payload for the stream either WS or h2. Both handled in
>> the same way by intermediaries.
>>
>> Amos
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v2.0.22 (MingW32)
>>
>> iQEcBAEBAgAGBQJUeUr6AAoJELJo5wb/XPRjvxoH/2VG9f7uN+VZM2LJAghbEDK3
>> nEUoptPw4TIG/nt2LgBWU8qCnL+kNfVzfSWSEdw2FFI9XdS3T99eAAWXBEJcHsE6
>> Vf2AxpoassRYXr13p2Tmjo6Lz0/uLhoFgNYePGdSEyKS75tULaoQP0rM05H3w3Ym
>> EWPvn4vtFkmWgRm/39lMvGBZjvnTlaKBEIl8rrSRE2BUDAbfSBzcr/2V8ZOpPHm+
>> DnetSTFP2UpLRu1Ufsir4bnTh3vhf6810HBTz5ylO8rUYIuqkGJwLuGmjSIE2V3z
>> uB/tVxr7903nw6o4/FtwGwvJKXt8QJujEswdEan2SoQhBkJaO+tfBaq4hyO3Wc0=
>> =WW/v
>> -----END PGP SIGNATURE-----
>>
>>
>

---

From: Martin Thomson <[email-masked]>
Date: Wed, 3 Dec 2014 08:17:52 -0800

> Sorry I'm not sure what it means: Can you explain?

HEADERS
Subsequent-Frames-Will-Be: binary
Other-WebSockets-Meta-Information: here
DATA...
binary frame data
HEADERS
Subsequent-Frames-Will-Be: text
DATA...
text frames

Roberto wants HEADERS (other than the first) to be flow controlled,
but that isn't entirely relevant here.

---

From: Roberto Peon <[email-masked]>
Date: Wed, 3 Dec 2014 09:00:47 -0800

Basically one either needs flow controlled headers, or needs a headers
frame (or equivalent) which doesn't use a compression context, plus a
dataframe which can signal end-of-message, at which point we have
everything one could conceivably need for a message based protocol.

-=R

---

From: Brad Fitzpatrick <[email-masked]>
Date: Wed, 3 Dec 2014 09:18:25 -0800

Roberto, what's the motivation for flow-controlled headers? If every WS
message's DATA frame(s) is preceded by a HEADERS to say whether it's JSON
or binary, and DATA are flow controlled, doesn't that mean there's at most
one HEADERS that sneaks by without flow controlled when the peer wants the
other to stop sending? Is it that one small-ish header (the one saying
"next is text" or "next is binary") that concerns you?

---

From: Roberto Peon <[email-masked]>
Date: Wed, 3 Dec 2014 19:05:36 -0800

If headers/metadata isn't flow controlled, then it becomes possible to
create an infinite queue on a proxy, e.g. infinite zero-length messages.
If one wishes for an ability for a receiver to stop receiving messages,
then the headers themselves need to be flow controlled.

One way of accomplishing it, which I tried to allude to ( =] ) was that a
headers-like-frame which was guaranteed not to use stateful opcodes could
be serialized as data (i.e. subject to flow control).
Another way of accomplishing it is to have every message require at least a
byte of data.

-=R

---

From: Glen Knowles <[email-masked]>
Date: Wed, 3 Dec 2014 19:31:23 -0800

I've been getting slowly less and less interested in any message
framing/metadata etc that isn't already in http2. All of my server to
server or rich client to server use cases would be satisfied at least as
well by a bidirectional http2 mode as by any websocket proposal I've seen
so far.

On Wed, Dec 3, 2014 at 7:05 PM, Roberto Peon <[email-masked]> wrote:

> Another way of accomplishing it is to have every message require at least
> a byte of data.
>
> -=R
>

---

From: Roberto Peon <[email-masked]>
Date: Wed, 3 Dec 2014 19:38:39 -0800

The purpose of the proposal to indicate messages was to ensure that
intermediaries could do balancing and reliability things, and not have to
leave it to endpoints alone, which leads to more latency and heartache.

I liked the idea that one could buy a box from whatever loadbalancer vendor
and it would just work with your traffic with a simple configuration
regardless of the kind of protocol (websocket, rpc-foo, whatever) being
sent over HTTP2 since the message and stream boundaries were all delimited.
-=R