Subject: Fwd: New Version Notification for draft-mcmanus-httpbis-h2-websockets-02.txt

From: Patrick McManus <[email-masked]>
Date: Sun, 12 Nov 2017 22:53:16 +0000 (UTC)

fyi
---------- Forwarded message ----------
From: <[email-masked]>
Date: Mon, Nov 13, 2017 at 6:52 AM
Subject: New Version Notification for
draft-mcmanus-httpbis-h2-websockets-02.txt
To: Patrick McManus <[email-masked]>

A new version of I-D, draft-mcmanus-httpbis-h2-websockets-02.txt
has been successfully submitted by Patrick McManus and posted to the
IETF repository.

Name:           draft-mcmanus-httpbis-h2-websockets
Revision:       02
Title:          Bootstrapping WebSockets with HTTP/2
Document date:  2017-11-11
Group:          Individual Submission
Pages:          7
URL:            https://www.ietf.org/internet-drafts/draft-mcmanus-httpbis-
h2-websockets-02.txt
Status:         https://datatracker.ietf.org/doc/draft-mcmanus-httpbis-h2-
websockets/
Htmlized:       https://tools.ietf.org/html/draft-mcmanus-httpbis-h2-
websockets-02
Htmlized:       https://datatracker.ietf.org/doc/html/draft-mcmanus-
httpbis-h2-websockets-02
Diff:           https://www.ietf.org/rfcdiff?url2=draft-mcmanus-httpbis-h2-
websockets-02

Abstract:
   This document defines a mechanism for running the WebSocket Protocol
   [RFC6455] over a single stream of an HTTP/2 connection.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 13 Nov 2017 09:03:27 +0800

Hi Patrick,

Thank you for the updates.

-02 looks very good to me; let me happily retract my comment to the
previous draft suggesting that we should allow use of any method and
use 101 as a signal that the protocol has switched.

The point I had missed is that END_STREAM is used as a signal to
indicate the end of the request under all methods except CONNECT.
OTOH, in WebSockets (or in any other protocol upgraded from HTTP), we
would want to use the stream (without sending END_STREAM) to transfer
the bidirectional communication.

There are three possible solutions: a) use CONNECT (or define a new
method that has the same characteristics as CONNECT; i.e. END_HEADERS
marks end of the request), b) define a new mechanism for detecting the
boundary between end of the HTTP request and the beginning of the
payload of the upgraded bi-directional stream, c) use a new
pseudo-header that instructs the server to consider END_HEADERS as the
end of the request for any given method.

Considering the fact that the only protocol upgraded from HTTP that we
care now is WebSocket which only uses GET, I now agree that option a
is a logical solution.

The remaining question is if we might want to convey HTTP request body
in a future protocol that upgrades from a HTTP/2 stream (or a
QUIC-HTTP stream). We should choose option b if we think that it is
highly probable to see proposals for such protocols. But I am not sure
if that is the case, hence my support for option a (i.e. reuse CONNECT
method).

It’s a pity that you would need to rewrite the method in a reverse
proxy that forwards HTTP/2 to ws-h1. But I think I can cope with that.

2017-11-13 6:53 GMT+08:00 Patrick McManus <[email-masked]>:
> URL:
> https://www.ietf.org/internet-drafts/draft-mcmanus-httpbis-h2-websockets-02.txt
> Status:
> https://datatracker.ietf.org/doc/draft-mcmanus-httpbis-h2-websockets/
> Htmlized:
> https://tools.ietf.org/html/draft-mcmanus-httpbis-h2-websockets-02
> Htmlized:
> https://datatracker.ietf.org/doc/html/draft-mcmanus-httpbis-h2-websockets-02
> Diff:
> https://www.ietf.org/rfcdiff?url2=draft-mcmanus-httpbis-h2-websockets-02
>
> Abstract:

-- 
Kazuho Oku

---

From: Andy Green <[email-masked]>
Date: Mon, 13 Nov 2017 09:37:20 +0800

On 11/13/2017 06:53 AM, Patrick McManus wrote:

> <https://www.ietf.org/internet-drafts/draft-mcmanus-httpbis-h2-websockets-02.txt>

                                            SETTINGS
                                            ENABLE_CONNECT_PROTOCOL = 1

    HEADERS + END_HEADERS
    :method = CONNECT
    :protocol = websocket
    :scheme = https
    :path = /chat
    :authority = server.example.com:443
    sec-websocket-protocol = chat, superchat
    sec-websocket-extensions = permessage-deflate
    sec-websocket-version = 13
    origin = http://www.example.com

                                            HEADERS + END_HEADERS
                                            :status = 200
                                            sec-websocket-protocol = chat

    DATA
    WebSocket Data

                                            DATA + END_STREAM
                                            WebSocket Data

    DATA + END_STREAM
    WebSocket Data

This looks really good to me, I'll be very happy to implement it.

-Andy

---

From: Kari Hurtta <[email-masked]>
Date: Mon, 13 Nov 2017 08:30:44 +0200 (EET)

5.  Using Extended CONNECT To Bootstrap The WebSocket Protocol
https://tools.ietf.org/html/draft-mcmanus-httpbis-h2-websockets-02#section-5

"          The Host information is conveyed as part of the :authority
"   pseudo-header which is required on every HTTP/2 transaction.

8.1.2.3.  Request Pseudo-Header Fields
https://tools.ietf.org/html/rfc7540#section-8.1.2.3

"      To ensure that the HTTP/1.1 request line can be reproduced
"      accurately, this pseudo-header field MUST be omitted when
"      translating from an HTTP/1.1 request that has a request target in
"      origin or asterisk form (see [RFC7230], Section 5.3).  Clients
"      that generate HTTP/2 requests directly SHOULD use the ":authority"
"      pseudo-header field instead of the Host header field.  An
"      intermediary that converts an HTTP/2 request to HTTP/1.1 MUST
"      create a Host header field if one is not present in a request by
"      copying the value of the ":authority" pseudo-header field.

So :authority is not required on *every* HTTP/2 transaction.

/ Kari Hurtta

---

From: Kari Hurtta <[email-masked]>
Date: Mon, 13 Nov 2017 19:16:27 +0200 (EET)

Kari Hurtta <[email-masked]>: (Mon Nov 13 08:30:44 2017)

> So :authority is not required on *every* HTTP/2 transaction.

⇒

"          The Host information is conveyed as part of the :authority
"    pseudo-header which is required on CONNECT -method in HTTP/2.

8.3.  The CONNECT Method
https://tools.ietf.org/html/rfc7540#section-8.3

"   o  The ":authority" pseudo-header field contains the host and port to
"      connect to (equivalent to the authority-form of the request-target
"      of CONNECT requests (see [RFC7230], Section 5.3)).

/ Kari Hurtta

---

From: Kari Hurtta <[email-masked]>
Date: Mon, 13 Nov 2017 19:49:14 +0200 (EET)

This should mention that :status = 200 replaces status code 101
used on RFC 6455, section 1.3.

1.3.  Opening Handshake
https://tools.ietf.org/html/rfc6455#section-1.3

"   The handshake from the server is much simpler than the client
"   handshake.  The first line is an HTTP Status-Line, with the status
"   code 101:
"
"        HTTP/1.1 101 Switching Protocols
"
"   Any status code other than 101 indicates that the WebSocket handshake
"   has not completed and that the semantics of HTTP still apply.  The
"   headers follow the status code.

8.1.1.  Upgrading from HTTP/2
https://tools.ietf.org/html/rfc7540#section-8.3

"   HTTP/2 removes support for the 101 (Switching Protocols)
"   informational status code ([RFC7231], Section 6.2.2).
"
"   The semantics of 101 (Switching Protocols) aren't applicable to a
"   multiplexed protocol.  Alternative protocols are able to use the same
"   mechanisms that HTTP/2 uses to negotiate their use (see Section 3).

/ Kari Hurtta

---

From: Takeshi Yoshino <[email-masked]>
Date: Wed, 15 Nov 2017 19:14:28 +0900

"due to its multiplexing nature" in the introduction section looks
redundant.

Given that we drop the Sec-WebSocket-Key/Sec-WebSocket-Accept header
considering that SETTINGS+:protocol is sufficient to ensure everyone on the
path understands WebSockets (tunneling), I suggest that we also drop the
client-server frame masking while also allowing masking as optional for LBs
that don't want to parse/rebuild frames but forward everything after
upgrading.

---

From: Andy Green <[email-masked]>
Date: Wed, 15 Nov 2017 18:40:52 +0800

On 11/15/2017 06:14 PM, Takeshi Yoshino wrote:
> considering that SETTINGS+:protocol is sufficient to ensure everyone on 
> the path understands WebSockets (tunneling), I suggest that we also drop 
> the client-server frame masking while also allowing masking as optional 
> for LBs that don't want to parse/rebuild frames but forward everything 
> after upgrading.

I think you are right and no point for this expensive, pointless, 
masking when it goes through this tunnel, but it's open to the client, 
ie, typically the browser to either still follow the requirement from 
RFC6455:

5.2
...

    Mask:  1 bit

       Defines whether the "Payload data" is masked.  If set to 1, a
       masking key is present in masking-key, and this is used to unmask
       the "Payload data" as per Section 5.3.  **All frames sent from
       client to server have this bit set to 1. **

...or, the client can just decide to disregard the requirement due to 
its understanding of the tunnel and not set the frame-masked bit on 
outgoing traffic that goes through the h2 CONNECT tunnel and save 4 
bytes per ws frame.

All this tunnel spec can do is make a recommendation the masking is not 
needed, with :protocol websocket it is supposed to be being an *RFC6455* 
tunnel, so it should certainly be able to handle masking if that's what 
the client side decided (not very well) that it wanted to send.

-Andy

---

From: Takeshi Yoshino <[email-masked]>
Date: Wed, 15 Nov 2017 20:06:53 +0900

> the bidirectional communication.

Can this be one of justifications for sticking to CONNECT?

Given that CONNECT + :protocol is already different from CONNECT w/o
:protocol, my rough guess is that this wouldn't help proxy implementors so
much e.g. simplifying logic by using the same logic for CONNECT with and
without :protocol.

Or, more about concern of divergence from RFC 7540? I.e. it's good to rely
on the characteristics already given to CONNECT by RFC 7540.

> end of the request for any given method.

Interesting point!

I share the same feeling as you regarding the demand for asking the peer to
treat some bytes following END_HEADERS as a part of the request and the
rest as not. I can't come up with any use case and leaning toward not
introducing such a mechanism. I might be wrong though.

---

From: Takeshi Yoshino <[email-masked]>
Date: Wed, 15 Nov 2017 20:17:40 +0900

>> that don't want to parse/rebuild frames but forward everything after
After I sent this mail, I noticed that a proxy can skip parsing the
tunneled traffic for the direction from WS/(TLS)/TCP w/ masking to WS/H2
(allowed to be w/o masking), but for the other direction it cannot since
when masking is not made, it needs to apply masking before forwarding to
WS/(TLS)/TCP.

> I think you are right and no point for this expensive, pointless, masking
> when it goes through this tunnel, but it's open to the client, ie,
> typically the browser to either still follow the requirement from RFC6455:
>
> 5.2
> ...
>
>    Mask:  1 bit
>
> ...or, the client can just decide to disregard the requirement due to its
> understanding of the tunnel and not set the frame-masked bit on outgoing
> traffic that goes through the h2 CONNECT tunnel and save 4 bytes per ws
> frame.
>
> tunnel, so it should certainly be able to handle masking if that's what the
> client side decided (not very well) that it wanted to send.

Yeah, maybe.

---

From: Patrick McManus <[email-masked]>
Date: Wed, 15 Nov 2017 23:34:07 +0000 (UTC)

> "due to its multiplexing nature" in the introduction section looks
thanks!

> that don't want to parse/rebuild frames but forward everything after
I don't think we can go there for a couple of reasons

 a] that kind of change to the guts of 6455 is not in scope for httpbis -
to do this work here we need to limit ourselves to how http/2 initiates
websockets

 b] part of the threat model for the masking is an otherwise passive
intermediary seeing a cleartext HTTP fragment (created by ws content) and
acting on it. as neither hybi nor httpbis made their protocols prohibit
cleartext (ugh) that threat isn't really changed by this document.

- but I do think you could very easily do a websocket extension that turned
off masking for wss:// contexts. That would be separate from this work.. I
would expect it to be pretty easy.. maybe dispatch is the right ietf venue
at this point. I think it would be easy to find implementer interest.

---

From: Lucas Pardue <[email-masked]>
Date: Fri, 17 Nov 2017 03:00:49 +0000

Hi Patrick,

This proposal is interesting. One small point, the introduction says:

This tunneled stream will be multiplexed with other regular streams
on the connection and enjoys the normal priority, cancellation, and
flow control features of HTTP/2.

On the surface this sounds sensible, however I think there could be some issues with clients that naively place dependencies on the completion of a WebSocket stream. This assumes that the WebSocket stream is longer lived than the typical request/response stream. Funnies with priority may also surface from pushed streams that are anchored of off the WebSocket stream (is it even possible to push HTTP resources on a WebSocket parent?)

Kind regards
Lucas

From: Patrick McManus [mailto:[email-masked]]
Sent: 12 November 2017 22:53
To: HTTP Working Group <[email-masked]>
Subject: Fwd: New Version Notification for draft-mcmanus-httpbis-h2-websockets-02.txt

fyi
---------- Forwarded message ----------
From: <[email-masked]<mailto:[email-masked]>>
Date: Mon, Nov 13, 2017 at 6:52 AM
Subject: New Version Notification for draft-mcmanus-httpbis-h2-websockets-02.txt
To: Patrick McManus <[email-masked]<mailto:[email-masked]>>

Name:           draft-mcmanus-httpbis-h2-websockets
Revision:       02
Title:          Bootstrapping WebSockets with HTTP/2
Document date:  2017-11-11
Group:          Individual Submission
Pages:          7
URL:            https://www.ietf.org/internet-drafts/draft-mcmanus-httpbis-h2-websockets-02.txt
Status:         https://datatracker.ietf.org/doc/draft-mcmanus-httpbis-h2-websockets/
Htmlized:       https://tools.ietf.org/html/draft-mcmanus-httpbis-h2-websockets-02
Htmlized:       https://datatracker.ietf.org/doc/html/draft-mcmanus-httpbis-h2-websockets-02
Diff:           https://www.ietf.org/rfcdiff?url2=draft-mcmanus-httpbis-h2-websockets-02

Abstract:

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org<http://tools.ietf.org>.

The IETF Secretariat

---

From: Bence Béky <[email-masked]>
Date: Mon, 20 Nov 2017 12:31:17 -0500

Hi y'all,

Patrick, thank you for driving this effort!  A couple of questions from my
team:

1. What happens if the client needs to send a Websocket request to a server
but there is currently no connection open to the the server: is it okay for
the client to open an HTTP/2 connection, and fallback to HTTP/1.1 only if
the server does not support Websockets over HTTP/2?

If so, does the client need to wait for the initial SETTINGS frame to see
if there is an ENABLE_CONNECT_PROTOCOL = 1 parameter in it, or can it send
out the Websocket request right away in order to save a roundtrip (it would
fail anyway if the server has no support)?  Would it make sense to
explicitly allow an HTTP/2 server to process a Websocket request that it
receives before the initial SETTINGS frame is ACKed?  The client would
still need to wait for the server handshake before it can use the Websocket.

2. "On requests bearing the :protocol pseudo-header, the :scheme and :path
pseudo-header fields SHOULD be included."

Is there a use case for a request with :protocol but without :scheme or
:path headers?  In other words, why could this not be a MUST?

3. As I am reading the current specification, there is no restriction on
sending ENABLE_CONNECT_PROTOCOL = 1 in a SETTINGS frame other than the
initial SETTINGS frame.  Is there any scenario where it would make sense
for a server to do this?  If not, would it be overly restrictive to require
this setting to be sent only in the initial SETTINGS frame?

Cheers,

Bence

On Sun, Nov 12, 2017 at 5:53 PM, Patrick McManus <[email-masked]>
wrote:

> URL:            https://www.ietf.org/internet-
> drafts/draft-mcmanus-httpbis-h2-websockets-02.txt
> Htmlized:       https://datatracker.ietf.org/
> doc/html/draft-mcmanus-httpbis-h2-websockets-02
> Diff:           https://www.ietf.org/rfcdiff?
> url2=draft-mcmanus-httpbis-h2-websockets-02
>
> Abstract:

---

From: Patrick McManus <[email-masked]>
Date: Mon, 20 Nov 2017 19:20:44 +0000 (UTC)

Hey Bence - thanks for the review.

On Mon, Nov 20, 2017 at 12:31 PM, Bence Béky <[email-masked]> wrote:

> 1. What happens if the client needs to send a Websocket request to a
> server but there is currently no connection open to the the server: is it
> okay for the client to open an HTTP/2 connection, and fallback to HTTP/1.1
> only if the server does not support Websockets over HTTP/2?
>
>
that's pretty much the client's choice; I would expect it to be dictated by
what the client expects the outcome to be.. for example when you have 6
pending requests for a new origin and no connections do you open 6 conns
assuming h1 or open 1 assuming h2? a little of both? Its a local decision
that might change as server support changes in the market. also caching.

> still need to wait for the server handshake before it can use the Websocket.
the client needs to wait because the draft changes the definition of
CONNECT.. you can't use the new definition against a server that hasn't
opted into it. Note that there is no technical requirement on the server
for a settings ack - the requirement is on when the client can send the
request. This might be 0.5 rtt or not, depending on the style of handshake.
It is the annoying part here driven by the simplicity of the design. I
think that's the right choice.

> :path headers?  In other words, why could this not be a MUST?
ought to be MUST. will change. thanks.

> this setting to be sent only in the initial SETTINGS frame?
we don't normally do that though obviously a good implementation would
announce asap and I don't think has an incentive to do otherwise. I think
its fine for a client to look at an h2 connection where it has received a
settings without enable_connect_protocol and assume it will never be able
to do websockets there.

---

From: Adam Rice <[email-masked]>
Date: Tue, 21 Nov 2017 13:21:35 +0900

> think that's the right choice.

I'm concerned about this "wait an arbitrary amount of time before falling
back" semantic. When we first implement it I expect to be falling back
almost all of the time. In fact, most developers would probably have a
better experience if we never attempted h2 at all. We could race an h2
connection with an h1 connection but that seems suboptimal.

---

From: Andy Green <[email-masked]>
Date: Tue, 21 Nov 2017 13:08:00 +0800

On 11/21/2017 12:21 PM, Adam Rice wrote:
> On 21 November 2017 at 04:20, Patrick McManus <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>     CONNECT.. you can't use the new definition against a server that
>     hasn't opted into it. Note that there is no technical requirement on
>     the server for a settings ack - the requirement is on when the
>     client can send the request. This might be 0.5 rtt or not, depending
>     on the style of handshake. It is the annoying part here driven by
>     the simplicity of the design. I think that's the right choice.
> 
> 
> I'm concerned about this "wait an arbitrary amount of time before 
> falling back" semantic. When we first implement it I expect to be 
> falling back almost all of the time. In fact, most developers would 
> probably have a better experience if we never attempted h2 at all. We 
> could race an h2 connection with an h1 connection but that seems suboptimal.

It's a very common pattern a script is served in HTML from a server, 
only after getting the script does the client try to make the ws 
connection back.

If the ws connection is back to the same server that gave it the script, 
the client knows the connection SETTINGS situation already, as it says 
in RFC7540 6.5

"SETTINGS frames always apply to a connection, never a single stream."

So you can use the existing connection's SETTINGS to know if it's worth 
trying the h2 path for ws, when it's targeting the same server.

If it's targeting a different server you're a bit stuck as you say... it 
will be fairly expensive to set up SSL and try h2 on that, fail to get 
the SETTINGS you want, then retry setting up SSL for h1.

-Andy

---

From: Takeshi Yoshino <[email-masked]>
Date: Wed, 22 Nov 2017 18:36:09 +0900

> to do this work here we need to limit ourselves to how http/2 initiates
OK. I'm fine with making sure that we decouple the effort into building a
general tunneling solution and some amendment to RFC 6455. The latter could
be discussed later.

> cleartext (ugh) that threat isn't really changed by this document.

Ah, yes. Agreed with your suggestion below.

>
> - but I do think you could very easily do a websocket extension that
> turned off masking for wss:// contexts. That would be separate from this
> work.. I would expect it to be pretty easy.. maybe dispatch is the right
> ietf venue at this point. I think it would be easy to find implementer
> interest.
>

---

From: Takeshi Yoshino <[email-masked]>
Date: Wed, 22 Nov 2017 19:01:04 +0900

On Tue, Nov 21, 2017 at 4:20 AM, Patrick McManus <[email-masked]>
wrote:

>> Patrick, thank you for driving this effort!  A couple of questions from
>> my team:
>>
> that's pretty much the client's choice; I would expect it to be dictated
> by what the client expects the outcome to be.. for example when you have 6
Doesn't ALPN help here?

>
>
>> 2. "On requests bearing the :protocol pseudo-header, the :scheme and
>> :path pseudo-header fields SHOULD be included."
>>

The only case I was thinking is suppressing the ENABLE_CONNECT_PROTOCOL
until the main resource of the web app is requested. But I guess there
would be very limited cases where this really works. The client may already
have the main resource (already loaded or just fetched from some cache) and
just want to connect WS.

I don't come up with any situation where we want to route a few to H1 and
then the remaining ones to H2.

Sending ENABLE_CONNECT_PROTOCOL is not expensive. We could just do as
Patrick said.

---

From: Bence Béky <[email-masked]>
Date: Wed, 22 Nov 2017 07:58:19 -0500

Hi,

Thanks for all the responses.

On Mon, Nov 20, 2017 at 2:20 PM, Patrick McManus <[email-masked]> wrote:
>> receives before the initial SETTINGS frame is ACKed?  The client would still
>> need to wait for the server handshake before it can use the Websocket.
>>
>
> opted into it. Note that there is no technical requirement on the server for
> a settings ack - the requirement is on when the client can send the request.
> This might be 0.5 rtt or not, depending on the style of handshake. It is the
> annoying part here driven by the simplicity of the design. I think that's
> the right choice.
>

Since there is no requirement for the server to wait for settings ACK
before accepting Websockets requests, there should be no harm in the
client sending such a request right after sending its initial SETTINGS
on a freshly opened h2 connection.  If the server supports Websockets,
the request will go through.  If the server does not, then it will
reset the frame because of the undefined pseudo-header.  The earlier
the client sends the request, the better latency we get.  The earlier
the client gets a failure, the better latency we get for the
retry-over-h1 case.  Am I missing something here?

Or if this behavior is not desirable, then instead just default to h1
every time there is no open h2 connection with Websockets support to
the server.  As Andy mentioned, most of the cases we expect a warm
connection when a script requests a Websocket anyway.  We can gather
some data about this and try to be clever later.

Bence

---

From: Patrick McManus <[email-masked]>
Date: Wed, 22 Nov 2017 23:45:22 +0000 (UTC)

> retry-over-h1 case.  Am I missing something here?
the opt-in settings is there to allow non-7540 compatible behavior.
otherwise we wouldn't need it at all.

a server that didn't opt-in to this extension must issue a protocol error
upon receipt of the unknown pseudo header. So a client better not send it :)

similarly a CONNECT with :scheme and :path elements must generate a
protocol error on a server that didn't opt-in because they are explicitly
disallowed by 7540 (in the absence of the extension).

an alternative design with a new method and the protocol field stuffed
somewhere else has end-to-end problems because there is no equivalent of
"Connection" in h2 and also practical problems with any non-CONNECT method
that has a fully bidi stream behavior (as opposed to request before
response) in that kind of scenario.

opt-in is annoying, but in this case I don't think its a big deal to move
the ball forward - especially as you say the no-connection case is not
expected to be all that common.