Subject: Re: CONNECT and HTTP/2.0

From: Amos Jeffries <[email-masked]>
Date: Sun, 01 Sep 2013 20:02:25 +1200

On 1/09/2013 2:42 p.m., William Chan (陈智昌) wrote:
> Sorry for the standardese. Please refer to 
> http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-23#section-4.3.6 
> and 
> http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-23#section-5.3. 
>
>
> Copied from p1 section 5.3:
>     authority-form
>
>     The authority-form of request-target is only used for CONNECT
>     requests (Section 4.3.6 of [Part2  <http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-23#ref-Part2>]).  When making a CONNECT request
>     to establish a tunnel through one or more proxies, a client MUST send
>     only the target URI's authority component (excluding any userinfo) as
>     the request-target.  For example,
>
>       CONNECTwww.example.com:80  <http://www.example.com:80>  HTTP/1.1
>
> Cheers.
>

So "example.com:80" is a _path_ in your world?

That kind of explains a few bugs we have with Chrome.

I believe the empty scheme header, empty path header, and filled host 
header (both FQDN and port) to be the accurate representation.

Amos

>
> On Sat, Aug 31, 2013 at 7:38 PM, Ryan Hamilton <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Looks good to me, with one question.  What is the "authority-form
>     of the request-target of CONNECT requests"?
>
>
>     On Sat, Aug 31, 2013 at 6:18 PM, William Chan (陈智昌)
>     <[email-masked] <mailto:[email-masked]>> wrote:
>
>         I suck at editorial stuff so I expect people to object to my
>         wording, but here's some proposed text (I'd be happy to put
>         together a pull request too)? Does this clarify whatever you
>         find muddy?
>
>         =====
>
>         8.3 The CONNECT method
>
>         The HTTP pseudo-method CONNECT [RFC2817] is used to convert an
>         HTTP/1.1 connection into a tunnel to a remote host. CONNECT is
>         primarily used with HTTP proxies to established a TLS session
>         with a server for the purposes of interacting with https
>         resources.
>
>         In HTTP/2.0, the CONNECT method is used to establish a tunnel
>         over a single HTTP/2.0 stream to a remote host. The HTTP
>         header mapping works as mostly as defined in 8.1.2.1, with a
>         few differences. Specifically:
>         * The :method header value is CONNECT
>         * The :scheme header field MUST be omitted.
>         * The :host header MAY be sent. It exists for backwards
>         compatibility with HTTP/1.1 implementations where the Host
>         header may not match the host indicated in the authority
>         specified in the request-target of the CONNECT request.
>         * The :path header value contains the authority-form of the
>         request-target of CONNECT requests.
>
>         =====
>
>         I note that it's tempting to omit the :host header, but my
>         primary concern is that it may introduce backwards
>         compatibility issues in HTTP/1=>HTTP/2 gateways, as I don't
>         know how HTTP/1 implementations currently handle the case
>         where the Host header does not match the host specified in the
>         request-target of CONNECT requests.
>
>
>         On Sat, Aug 31, 2013 at 5:51 PM, James M Snell
>         <[email-masked] <mailto:[email-masked]>> wrote:
>
>             A pull request with edits against the current draft would
>             work too :-)
>
>             On Aug 31, 2013 5:40 PM, "Ryan Hamilton" <[email-masked]
>             <mailto:[email-masked]>> wrote:
>
>                 I implemented the CONNECT over SPDY support in Chrome.
>                  Here are a couple of docs we cooked up at the time
>
>                 http://www.chromium.org/spdy/spdy-proxy-examples
>                 http://www.chromium.org/developers/design-documents/secure-web-proxy
>                 http://www.chromium.org/spdy/spdy-proxy
>
>                 The first is the most concrete.  I could write up a
>                 I-D for this, if that's the right way forward.
>                  However, it seems like the behavior of CONNECT on an
>                 HTTP/2 stream is pretty much identical to the behavior
>                 of CONNECT on an HTTP/1 connection.
>
>                 Cheers,
>
>                 Ryan
>
>
>
>                 On Sat, Aug 31, 2013 at 4:56 PM, James M Snell
>                 <[email-masked] <mailto:[email-masked]>> wrote:
>
>                     If CONNECT support is critical, then my
>                     recommendation would be for
>                     y'all to make a more formal proposal and document
>                     how it's supposed to
>                     work in the form of an I-D. Currently the details
>                     on how this would
>                     work are somewhat muddy.
>
>                     On Sat, Aug 31, 2013 at 4:35 PM, William Chan (陈智昌)
>                     <[email-masked]
>                     <mailto:[email-masked]>> wrote:
>                     > This is fine for now, but FYI I consider this a
>                     blocker for Chromium to
>                     > switch entirely to HTTP/2.0. I note that this is
>                     an existing HTTP feature
>                     > that clients use to tunnel over HTTP proxies. As
>                     far as its use in SPDY,
>                     > it's not merely theoretical, but has a number of
>                     actual uses:
>                     >
>                     > *
>                     >
>                     http://spdylay.sourceforge.net/package_README.html#shrpx-a-reverse-proxy-for-spdy-https
>                     > *
>                     >
>                     https://chrome.google.com/webstore/detail/breakwall-vpn-spdy-proxy/higommoegggcanmkapeoohipckeofpnd
>                     > (3000~ installs)
>                     > *
>                     >
>                     https://chrome.google.com/webstore/detail/spdy-proxy/hhihiednomfhmngipmplmgcngliajdnn
>                     > (4000~ installs)
>                     > * Corporate google.com <http://google.com> VPN
>                     extension (not public) (widely used by Googlers)
>                     >
>                     > I believe these uses demonstrate that this is a
>                     desired use case to support.
>                     > As noted, it is fairly straightforward to define
>                     a mapping of HTTP CONNECT
>                     > over HTTP/2.0. Please see:
>                     http://www.chromium.org/spdy/spdy-proxy-examples
>                     > and http://www.chromium.org/spdy/spdy-proxy.
>                     >
>                     >
>                     > On Fri, Aug 30, 2013 at 10:30 AM, Martin Thomson
>                     <[email-masked]
>                     <mailto:[email-masked]>>
>                     > wrote:
>                     >>
>                     >> Folks might notice that I've added a section on
>                     CONNECT to HTTP/2.0:
>                     >>
>                     >>
>                     http://http2.github.io/http2-spec/index.html#rfc.section.8.3
>                     >>
>                     >> This doesn't close #230, it simply documents
>                     status quo.  If we decide
>                     >> to support CONNECT, the draft will, of course,
>                     be updated to reflect
>                     >> that decision.  This is fairly straightforward
>                     based on the Chromium
>                     >> documentation and the discussion thus far, we
>                     just need to decide if
>                     >> it's valuable enough to do.
>                     >>
>                     >
>
>
>
>
>

---

From: Amos Jeffries <[email-masked]>
Date: Sun, 01 Sep 2013 21:18:19 +1200

On 1/09/2013 5:04 p.m., Martin Thomson wrote:
> On 31 August 2013 18:18, William Chan (陈智昌) <[email-masked]> wrote:
>> I suck at editorial stuff so I expect people to object to my wording, but
>> here's some proposed text (I'd be happy to put together a pull request too)?
>> Does this clarify whatever you find muddy?
> This is exactly the sort of response I wanted to encourage.  The only
> problem I see with your proposed text is that it says nothing about
> what forms the tunnel and its characteristics.  Basically, I think
> that a full edit for this needs to take a good hard look at 2817.
> This is a good start, but there's a bit more required.
>
> Note that changing what colon-headers are required, especially
> prohibiting :scheme is going to be a little bit of a surprise to some.
>   (And it will compress less well.)  Can we just say that its value is
> ignored instead?

I dont think so.

It makes sense to send it with empty value for authority-form URI. Or to 
permit it to contain a scheme name if teh :host header is missing a po

So both of these requests connect a tunne to example.com on port 443:

:method CONNECT
:scheme
:host example.com:443
:path

,
:method CONNECT
:scheme HTTPS
:host example.com
:path

If :path is left to be optional values it permits clients to either omit 
it when mapping HTTP/1->HTTP/2. Or to send a hint at what path the 
tunnel will be fetching. This is useful for the firewalls and gateways 
which currently are forced to spoof and decrypt the SSL before 
allowing/rejecting the traffic. THey will no longer be foreced to touch 
the encryption to manage the authorizations.

Amos

---

From: James M Snell <[email-masked]>
Date: Sun, 1 Sep 2013 08:10:08 -0700

> This is exactly the sort of response I wanted to encourage.  The only

+1 ... proposed spec text is always better than a bunch of links.

My problem with this proposal is that it smells like a hack. If I was
starting this from scratch, I'd likely propose that CONNECT would
morph from it's current http-layer definition and into something that
fits more naturally into the framing layer. For instance, rather than
having a CONNECT method expressed via a normal HTTP flow, there would
be a CONNECT frame type that would establish the tunnel independent of
the regular http semantics. But that's just me and I have to admit
that I don't have any skin in this particular part of the game.
Whatever is done here, we just need to make sure there is clear spec
text written in the draft, not links off to some other website.

- James

> Other issues: this tunneling will limit the size of TLS frames if the
> intent is to have them in a single HTTP/2.0 frame.  That might need
> some mention.

---

From: Martin Thomson <[email-masked]>
Date: Sun, 1 Sep 2013 11:03:26 -0700

On 1 September 2013 08:10, James M Snell <[email-masked]> wrote:
> My problem with this proposal is that it smells like a hack.

That was my first reaction, then I thought about what it would take to
do it right and I concluded that this is close enough to that answer
that I'm happy with it.

---

From: Martin Thomson <[email-masked]>
Date: Sun, 1 Sep 2013 11:07:00 -0700

On 31 August 2013 23:28, William Chan (陈智昌) <[email-masked]> wrote:
> Yeah, I didn't really know how to spec that. Basically, the DATA frames form
> the bidirectional tunnel.

Then that's what you say :)

> I'm not quite sure what else to say here, [...]

The only thing that probably needs looking into is the effect that
tunneling has on what the TLS layer will probably treat like TCP.
That is, note how this potentially differs from TCP.  Frame sizes can
be dealt with easily enough, but you also need to consider how it
closes.  Maybe this leads to a requirement to treat END_STREAM like
FIN (just as RST_STREAM is analogous to RST).

---

From: Ilari Liusvaara <[email-masked]>
Date: Sun, 1 Sep 2013 22:10:35 +0300

On Sat, Aug 31, 2013 at 11:28:05PM -0700, William Chan (陈智昌) wrote:
> <[email-masked]>wrote:
> 
> Yeah, I didn't really know how to spec that. Basically, the DATA frames
> form the bidirectional tunnel. I'm not quite sure what else to say here,
> but I have difficulty because my natural thought process matches what we
> did in SPDY, so I don't know how to discuss this in a way that would clear
> up any other possible tunnel characteristics. Do you have any proposed text
> here?

Regarding :scheme, what about fixing it to 'tcp' (with possiblity of some
other kind of CONNECT be specified later)?

And as I said before (for the part after tunnel has established):

Client <-- HTTP2 stream ---> Proxy <-- TCP connection --> Server

1) If DATA frame is received from the client:

- Send its payload to the server.

2) If END_STREAM is received from the client:

- Flush any buffered to to the server (including this DATA frame
  payload if any).
- Send end-of-data to the server.

3) If stream error is received from the client.

- Send TCP reset to the server.

4) If data is received from the server:

- Send the data to client in DATA frame(s).

5) If end-of-data is received from the server:

- Flush any buffered data to the client as DATA frame(s).
- Send a frame with END_STREAM (possibly on the last DATA frame).

6) If reset is received from the server:

- Send stream error to client.

Basically:

HTTP2 DATA <---> TCP data.
HTTP2 END_STREAM <--> TCP end-of-data.
HTTP2 stream error <--> TCP reset.

But this only works with TCP. If SCTP or something is to be supported, things
get much more complex.

-Ilari

---

From: "Adrien de Croy" <[email-masked]>
Date: Sun, 01 Sep 2013 21:37:38 +0000

in all this didcussion about CONNECT, I presume that references to 
server, are to be interpreted as proxy?

Since CONNECT is for proxies only.

Or is it being proposed this could be useful for end servers?

Adrien

------ Original Message ------
From: "Ilari Liusvaara" <[email-masked]>
To: "William Chan (陈智昌)" <[email-masked]>
Cc: "Martin Thomson" <[email-masked]>; "James M Snell" 
<[email-masked]>; "Ryan Hamilton" <[email-masked]>; "HTTP Working 
Group" <[email-masked]>
Sent: 2/09/2013 7:10:35 a.m.
Subject: Re: CONNECT and HTTP/2.0
>>wording, but
>>  > > here's some proposed text (I'd be happy to put together a pull 
>>request
>>  > too)?

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sun, 1 Sep 2013 15:12:57 -0700

> +1 ... proposed spec text is always better than a bunch of links.

FYI, an I-D or pull request would require following a link :) But I agree
that the proposed spec text was more clearly stated that our previous
documentation on dev.chromium.org.

> that I don't have any skin in this particular part of the game.

FWIW, I don't care too much either. I just object when people say that we
should *remove* an existing feature in HTTP. That will break backwards
compatibility with HTTP/1.X, which will block adoption (I already pointed
out that my browser can't switch fully without this feature).

> text written in the draft, not links off to some other website.

Hah, of course. Doesn't that always go without saying? :P Lol.

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sun, 1 Sep 2013 15:28:42 -0700

On Sun, Sep 1, 2013 at 11:07 AM, Martin Thomson <[email-masked]>wrote:

> form

I was naturally assuming that END_STREAM==FIN in the tunnel. I guess we
need to specify it? How about adding the following paragraph:

=====
After successfully establishing a CONNECT tunnel over a stream, the stream
DATA frames comprise the data of the bidirectional tunnel. Each direction
of the tunnel is closed via the END_STREAM flag, similar to how TCP closes
each direction with its FIN flag. RST_STREAM abnormally terminates the
tunnel, similar to how TCP RSTs abnormally terminate the connection.
=====

---

From: Ilari Liusvaara <[email-masked]>
Date: Mon, 2 Sep 2013 21:18:03 +0300

> Or is it being proposed this could be useful for end servers?

Copying the diagram:

The semantics were written from the point of view of the proxy.

But I think similar semantics could be applied to upgrade,
where:

Client <-- HTTP2 stream --> Server <-- unspecified --> Service

The transport between server and service is unspecified because
it is internal implementation detail not visible to the client.

Of course, that unspecified transport would have to internally
define concepts of end-of-data and reset. But if it can run
from HTTP/1.1, it must anyway.

-Ilari

---

From: William Chan (陈智昌) <[email-masked]>
Date: Mon, 9 Sep 2013 08:36:20 -0700

Any further thoughts on this? Should I put together a pull request?

---

From: Martin Thomson <[email-masked]>
Date: Mon, 9 Sep 2013 11:30:31 -0700

> Any further thoughts on this? Should I put together a pull request?

You know that we always like pull requests.  Better something than
nothing.  Then we can move on to arguing about the details.

---

From: William Chan (陈智昌) <[email-masked]>
Date: Mon, 9 Sep 2013 12:37:00 -0700

OK, I'll put one together. I just like having all the high level issues
sorted out here since the mailing list is way more visible than github.

---

From: William Chan (陈智昌) <[email-masked]>
Date: Tue, 24 Sep 2013 11:02:45 -0700

Sorry for the delay. https://github.com/http2/http2-spec/pull/249.