Subject: HTTP/2 and Websockets

From: Robert Collins <[email-masked]>
Date: Mon, 29 Sep 2014 16:10:58 +1300

Has anyone done any specification on integrating websockets into
HTTP/2? I couldn't find a spec, so I poked around a bit...

It looks to me like there are three issues.

Firstly we should define the handshake in terms of a stream opening on
an existing HTTP/2 connection, and since Upgrade: is not permitted in
HTTP/2 ("
Note:HTTP/2 purposefully does not support upgrade to another protocol.
The handshake methods described in Section 3 are believed sufficient
to negotiate the use of alternative protocols.
")

its not clear to me if that means we need to define a new frame type,
or whether we can still use the Upgrade header sanely.

Secondly, we need to define websocket frame mappings. The least work,
and I suspect the easiest for implementors, would be to put all the
websocket frames into HTTP/2's data frames, without worrying about
frame alignment: just treat the fully open stream as a series of bytes
in the same way TCP is treated by the websocket spec.
I suspect however that a better result would be achieved by defining
custom HTTP/2 frames, since websockets already have the basic support
for multiplexing (large application writes can be fragmented into
smaller frames as needed), we shouldn't run into HOL blocking issues.

Lastly, it seems (to me) that it would be desirable to allow
PUSH_PROMISE setup of websockets connections, but the PUSH_PROMISE
state machine goes straight into half-closed, rather than into 'open',
and that prohibits doing websockets with a pushed connection. There
isn't any prose in HTTP/2 about /why/ the state machine does that :(.

-Rob

-- 
Robert Collins <[email-masked]>
Distinguished Technologist
HP Converged Cloud

---

From: Yutaka Hirano <[email-masked]>
Date: Mon, 29 Sep 2014 14:26:34 +0900

Hi,

I am proposing a spec draft:
http://tools.ietf.org/html/draft-hirano-httpbis-websocket-over-http2-01 .
Since many modifications were made on on the HTTP/2 spec, some description
may be obsolete. Please let me know if you find any flaw.

> or whether we can still use the Upgrade header sanely.

The above draft describes the negotiation scheme. Please take a look.

> smaller frames as needed), we shouldn't run into HOL blocking issues.

Yeah, we can't simply use DATA frames because intermediaries may buffer
data. The HTTP/2 spec had "MSG_DONE" once and I wanted to use it, but it
was removed from the spec. Currently I think introducing a new frame type
is the right way.

> PUSH_PROMISE setup of websockets connections

Can you tell me why it is desirable?

Thanks,

---

From: Ilari Liusvaara <[email-masked]>
Date: Mon, 29 Sep 2014 12:51:29 +0300

> may be obsolete. Please let me know if you find any flaw.

The capability negotiation looks weird.

I would just signal SETTINGS_WEBSOCKET_CAPABLE with 0x1-bit set to 1 (and
the rest reserved for extension) from server (or from proxy towards
client).

After receiving SETTINGS_WEBSOCKET_CAPABLE with 0x1 flag, the client/
proxy knows the proxy/server can handle websockets (and is used via
:scheme 'ws'/'wss'). No need to ACK anything.

In the future, some more elaborate extensions might need to be enabled
from both sides, but be base functionality doesn't. Only features that
change existing semantics need to be acknowledged.

One example of something to signal from client to proxy/server would
be reverse connections (but this needs some way for proxies and browsers
to route the connection, maybe associated stream ID, and PUSH_PROMISE
can't be used due to its unidirectionality[1]).

Reverse connections can't be mapped to RFC 6455, so those would need
HTTP/2 path all the way between client and server.

> was removed from the spec. Currently I think introducing a new frame type

One way would be to take the base Websockets spec for data communications
and rip out everything unneeded and adapt some things:

- Server's/proxy(toward client) SETTINGS has notification about Websockets
  support.
- Client's opening handshake has no upgrade headers (upgrade or connection:
  upgrade). Instead it has :scheme set to either 'ws' or 'wss'.
- Client's opening handshake has :authority instead of host.
- Client's opening handshake has no Sec-websocket-key
- Server's opening handshake has 200 status instead of 101.
- Server's opening handshake has no Sec-websocket-accept
- Each DATA frame carries one websocket frame.
- FIN, RSVx and opcode are signaled as the first byte of DATA frame.
- There is no masking, and MASK is implcitly 0.
- Frame length is implicitly data frame length - 1.
- Zero-byte data frames are not valid.
- Proxies fragment or coalesce if needed (TODO: How to handle 16kB
  and up control frames?)
- Proxies can perform adaptation between RFC 6455 and Websockets over
  HTTP/2.
- No TLS handshakes are done (TLS is below HTTP/2).
- TCP connection is replaced by HTTP/2 stream.

This requires some support on proxies (e.g. priority boosting and
fragmenting), but proxies should know what they are dealing with,
and Websockets needs proxy support anyway (at least unless one does not
want to burn through sequence numbers really fast and to cause excessive
load on proxies).

Also, extensions that modify base header semantics (as opposed to just
using RSVx bits or adding their own header included in frame size)
don't work, but I think that is not a big problem. PMCE is OK.

[1] I earlier sent list of ideas for decoupling framing and HTTP layers,
PUSH_PROMISE is unidirectional even in that (that's the difference
between PUSH_PROMISE and HEADERS!).

-Ilari

---

From: Yutaka Hirano <[email-masked]>
Date: Mon, 29 Sep 2014 20:36:51 +0900

Thank you for the comments!

We have been not working actively for months because of two reasons:
1) The http/2 spec was changing rapidly and we couldn't write a spec on top
of it.
2) We weren't confident if wide range of people were interested in this
protocol.

1) will be resolved soon, I hope. For 2), If you are interested in
WebSocket over HTTP/2, it is very helpful to express it!
About one year ago, we were talking if we wanted perfect RFC6455 compatible
WebSocket or not (i.e. full-extension support). IIRC most of us didn't want
the full RFC6455 compatibility, but if you have any opinion, it is welcome.
In short, we would like to hear your voice.

> :scheme 'ws'/'wss'). No need to ACK anything.

My proposal doesn't need ACK.
The client sends a SETTINGS frame before sending the first handshake in the
stream. After that, the client starts the usual WebSocket opening handshake.

> - TCP connection is replaced by HTTP/2 stream.

Given that we have SETTINGS_WEBSOCKET_CAPABLE, I would use a new frame type
not to confuse intermediaries / endpoints. The actual data format will be
discussed, but IMHO it is less important and can be discussed later.

Thanks,

---

From: Greg Wilkins <[email-masked]>
Date: Tue, 30 Sep 2014 05:17:13 +1000

I think this is a timely subject as I believe it is very important to take
Yutaka's ideas forward to a working extension sooner rather than later.

My fear is that unless we test websocket over HTTP/2 before we go final, we
may discover some fatal flaw that prevents efficient handling of websocket
as a h2 stream.

Even if there are no nasties discovered, not having an extension defined at
the start will mean that a lot of h2 infrastructure will get deployed
without websocket support.  Once this occurs, it may be very difficult to
get good websocket coverage for some time with an extension approach.   I
fear that the response will be to come up with a websocket mapping that
abuses the h2 protocol so that it can tunnel through websocket unaware
intermediaries... and that is unlikely to be efficient because we have
dropped end_segment semantics from h2 streams.

Cheers

PS. My real preference is that this group get's recharted to support both
http and websocket semantics over h2 - and we go back to the drawing board
with the framing layer using >1 examples to generalise from.

On 29 September 2014 21:36, Yutaka Hirano <[email-masked]> wrote:

> 1) The http/2 spec was changing rapidly and we couldn't write a spec on
> top of it.
> About one year ago, we were talking if we wanted perfect RFC6455
> compatible WebSocket or not (i.e. full-extension support). IIRC most of us
> didn't want the full RFC6455 compatibility, but if you have any opinion, it
> is welcome.
> The client sends a SETTINGS frame before sending the first handshake in
> the stream. After that, the client starts the usual WebSocket opening
> handshake.
>
> Given that we have SETTINGS_WEBSOCKET_CAPABLE, I would use a new frame
> type not to confuse intermediaries / endpoints. The actual data format will
> be discussed, but IMHO it is less important and can be discussed later.
>
> Thanks,
>

-- 
Greg Wilkins <[email-masked]>
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.

---

From: Ilari Liusvaara <[email-masked]>
Date: Mon, 29 Sep 2014 22:36:24 +0300

> In short, we would like to hear your voice.

Couple points:
- It was just quick sketch using the standard "first idea that isn't known
  broken" way of problem solving.
- That sketch does not have full RFC6455 compability. Some kinds of extensions
  don't work (PMCE just happens to work).
- Some obvious-looking mappings don't work. E.g. one can't map Websockets
  PING to HTTP/2 PING (different scope).
- There should not be lots of HTTP2 header frames. Those are rather heavy,
  and thus increase resource usage. Control frames could be a possiblity,
  but I'd rather not have data frames use those (which would exclude using
  headers for delimiting frames).
- Mapping control frames to headers would be a possiblity. But even so, the
  semantics would likely be close enough for automatic translation at proxy
  to be possible.
- Automatic translation between WS1 and WS2 could be useful together with
  proxies capable of HTTP1<->HTTP2 translation.

> stream. After that, the client starts the usual WebSocket opening handshake.

The client needs to know if server supports WS2. Otherwise, the client could
send ws:// or wss:// URL to server that does not support WS2, and that is
"undefined behaviour" (HTTP 4xx/5xx, stream error, connection error(!) or
the worst of all, HTTP 200(!!!)).

I don't see any point in changing server behaviour based on
SETTINGS_WEBSOCKET_CAPABLE. Clients without WS2 support won't send WS2
handshakes. WS2-supporting clients may send those, to servers known to
support WS2.

Similar considerations apply for intermediaries.
 
> discussed, but IMHO it is less important and can be discussed later.

I can't see how such confusion could happen. Intermediary/endpoint is
known WS2-capable and the stream is designated as WS2 in the opening
handshake.

Or is this about dealing with really buggy intermediaries/endpoints?

-Ilari

---

From: Robert Collins <[email-masked]>
Date: Tue, 30 Sep 2014 10:46:08 +1300

> may be obsolete. Please let me know if you find any flaw.

Cool - I have some questions, below.

> data. The HTTP/2 spec had "MSG_DONE" once and I wanted to use it, but it was
> removed from the spec. Currently I think introducing a new frame type is the
> right way.

I don't understand: DATA is subject to flow control, but I don't see
any prose about buffering (or not buffering) of DATA frames. Unless we
define a new frame time and explicitly prohibit buffering (but we need
to keep it flow controlled), I don't see what would be different. And
more importantly, intermediaries might still choose to buffer a new ws
frame type anyway: we know that what the spec requires and what people
do will differ :). All the implementor discussion I've seen during the
HTTP/2 discussions has focused on how intermediaries want to be
scalable: and buffering is anti-scaling. So - is it a pragmatic
concern, or do we expect DATA stream buffering to take place [outside
of protocol gateways converting to HTTP/1.1 where non upload can
require buffering - and note that such a gateway can't carry ws anyway
unless its aware of it, and if its aware of it, it can make sure it
does not buffer].

> Can you tell me why it is desirable?

Same reason for PUSH_PROMISE of any stream, to avoid setup latency. If
the server is pushing down some javascript libraries that will want a
ws stream back to the same server, it could setup the stream in-line.

Ok, so my questions...

Firstly, I found the 'frame is overloaded, only specify when its
unambiguous' really hard for reading. I'd like to propose that we
fully qualify the frame type everywhere. 'h2 SETTINGS frame', 'ws
RST_STREAM frame' etc: doing otherwise requires a lot more mental
state, at least for me :).

3.2 seems like an antifeature: ALPN is for defining the base protocol
spoke, if you define a new ALPN name for things that speak websockets
over HTTP/2, but it still has all the same framing and flow control
etc, I think it just adds complexity.

There are two alternatives here:
 - ws[s] over TCP, RFC 6355, which using ALPN to select directly could
make sense [though nothing out there does it AFAIK?]
 - ws[s] over HTTP/2. which should be using ALPN to select *HTTP/2*,
since thats the protocol we need.

I say this because if adding ws to HTTP/2 makes a new ALPN, then when
we add a third thing (nt) we'll get:
h2
h2ws
h2nt
h2wsnt

And so on - its problematic :). And SETTING is sufficient to let us know.

Consider this: an h2 capable browser with ws over h2 support is asked
to connect to ws.example.com with ws. If it:
 - tries for h2
 - examines its peer setting for WEBSOCKET_CAPABLE
 - if present tries a HEADERS frame
 - if absent falls back to RFC6455

Then thats pretty sane IMO. If we get this deployed, it will work most
of the time, and when it doesn't, the fallback is there.

3.3
This section seems confused about the semantics of SETTING - see my
new post about clearing that up (whether I need the clearing up, or
the spec does :)).

Anyhow, settings is about advertising, not about negotiating - you
can't negotiate a path through an intermediary graph using it, because
there is no way for the intermediary to signal upstream that just one
client supports it and another doesn't. So the intermediary needs to
signal its support unconditionally. Likewise, the server can't use the
peer's WEBSOCKET_CAPABLE setting to replace the Sec-headers, because
a) a websocket capable browser might still have bad js executing and
trying to attack things, and b) an intermediary may be connecting
multiple different clients to the same server.

Here is an attempt at new prose:

3.3.  WebSocket over HTTP/2 capability

   Servers and intermediaries that can process WebSocket requests over
HTTP/2 MUST advertise the SETTINGS_WEBSOCKET_CAPABLE setting in their
SETTINGS frame.

   Websocket over HTTP/2 MUST NOT be attempted to peers that have not
set SETTINGS_WEBSOCKET_CAPABLE

   Clients do not need to advertise this capability as their making a
valid websockets request signals they are capable.

3.4 secure connection

   If the h2 connection is not secure, wss connections MUST NOT be
intiated over it.
   If the h2 connection is secure, both ws and wss connections MAY be
initiated over it.

3.5 Intermediaries

   Intermediaries that have advertised SETTINGS_WEBSOCKET_CAPABLE may
receive websocket requests which are for origins that do not advertise
SETTINGS_WEBSOCKET_CAPABLE (or may not even support HTTP/2). For
example, it is nearly certain that a forward proxy that speaks HTTP/2
will receive requests for origins that have not yet upgraded to
HTTP/2.

  In such a situation Intermediaries MUST either: initiate a RFC6455
websocket connection with the origin, and translate frames between the
two sides in conformance with both RFC6455 and this RFC. Or they may
return 501 (Not Implemented) to indicate that they cannot forward the
request.

  To illustrate: consider a Client(C), a websocket aware Proxy(P) and
a Server(S).
  P will include SETTINGS_WEBSOCKET_CAPABLE in its SETTINGS frame from P to C.
  C sees that P is capable, and initiates a websocket connection over
HTTP/2 to Origin S.
  P initiates an HTTP/2 connection with S
  S is a baseline HTTP/2 server and does not include
SETTINGS_WEBSOCKET_CAPABLE in its SETTINGS frame from S to P.
  At this point P can either error with 501, signalling that this
particular request cannot be carried, or it can fall back to RFC6455
on behalf of the client.

4.1
You say we can skip "Upgrade, Connection, Sec-WebSocket-Key, and
Sec-WebSocket-Version", because we don't need to do verification. I
think this prose is missing an explanation of why we don't need to do
verification.

There are two failure modes RFC6455 talks about:
A - connections to existing SMTP etc servers
B - submitting data from FORM posts to ws servers

The former is guarded against by looking for a ws specific handshake
from the server.
The latter is guarded against by looking for a ws specific header from
the client *which Javascript APIs do not permit javascript code to
set*.

Your draft defines ws over existing HTTP/2 connections and also new
connections to HTTP/2 endpoints. If we limit ourselves to just ws[s]
over existing HTTP/2 connections, then we maybe we can say:
A) is protected against by RFC6455, and any new connection made for ws
should follow that spec with one exception: If the server negotiates
as a valid HTTP/2 endpoint, then the SETTINGS_WEBSOCKET_CAPABLE
setting from the server is inspected to determine if ws over HTTP/2
can be used - and that supplances the server side calculation that was
used to prove websocket readiness in RFC6455.

B)  the presence of Sec-Websocket-Key and Sec-WebSocket-Version is
used to ensure that a WS endpoint doesn't get form data posted to it.
I see no replacement for that in your draft: we need to keep it,
because its protecting against javascript programming models. (Unless
I've missed something?)

4.2
'"101" or "101 Switching Protocols"' - AIUI in HTTP/2 the reason text
is gone. The status pseudo header is numeric only.

4.2.1 - the ALTSVC draft suggests doing this gracefully - e.g.
openning up the new connection then dropping the old one. We should
include a reason for not following that advice.

5.
There's nothing specified here - neither what frame types we need to
add, nor discussion on the [in]applicability of HTTP/2 DATA. I'd like
to try to use HTTP/2 data I think - the discussion about frame type
compatibility makes me think that we'll be more compatible with
RFC6455 if we just tunnel over the h2 DATA frame: remember that
RFC6455 targetted TCP as a transport, and a series of h2 DATA frames
is most analogous to that. In particular, if we use dedicated control
frames, we could hit out of order behaviour with control frames
forward before DATA frames, because DATA frames are flow controlled:
it will be more complex to specify, and I don't see a benefit.

7
I'm not aware of any equivalent to the masking in HTTP/2, and there is
no discussion of BEAST in the HTTP/2 spec: if we're delegating to
HTTP/2 to solve those issues, I think we need to talk about that now
:)

-Rob

---

From: Ilari Liusvaara <[email-masked]>
Date: Tue, 30 Sep 2014 02:07:21 +0300

> HTTP/2 to solve those issues, I think we need to talk about that now

HTTP/2 uses TLS 1.2, which is not vulernable to BEAST (it was fixed
in 1.1).

-Ilari

---

From: Yutaka Hirano <[email-masked]>
Date: Tue, 30 Sep 2014 16:58:13 +0900

> top
> compatible

"Full RFC6455 compatibility" means that the WebSocket over HTTP/2 can
express any data represented by the native WebSocket. For example, we can't
modify any WebSocket frame boundary if we decide to respect RFC6455. Please
see the long discussion starting from:
http://www.ietf.org/mail-archive/web/hybi/current/msg10266.html .
My opinion is we don't have to keep the full RFC6455 compatibility and
hence we can't have a general translation.

> the

Yeah, In my proposal, the client knows the server's preference via ALPN (or
ALT-SVC). That way, we can create a WebSocket connection without
introducing an additional RTT in typical cases.
The design principal there is
 - We don't want to introduce an additional RTT if the server doesn't
implement WebSocket over HTTP/2.
 - We don't want to introduce an additional RTT in typical cases if the
server implement WebSocket over HTTP/2.
SETTINGS_WEBSOCKET_CAPABLE is needed for the complete communication path
including intermediaries (and for introducing new frame types).

> type

One of lessons learned from the existing WebSocket is that many
intermediaries interpret what they should not interpret. See
http://w2spconf.com/2011/papers/websocket.pdf for example.
As the HTTP/2 spec allows us to create new frame types, doing so looks
safer.

On Tue, Sep 30, 2014 at 4:36 AM, Ilari Liusvaara <
[email-masked]> wrote:

> top
> compatible
> the
> type