Subject: :scheme and CONNECT method

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Tue, 13 Aug 2013 23:23:24 +0900

Hi,

draft-05 says that :scheme MUST be present in HTTP request.
If client sends to CONNECT method to the middle box, which value
should be sent as :scheme? http?
For the reference, the current SPDY proxy (aka secure proxy) omits
:scheme. For example, CONNECT to remote example.org:443 via
proxy, chrome sends the followings:

:method => CONNECT
:host => example.org
:path => example:org:443
:version => HTTP/1.1

:version is not omitted in HTTP/2.0, so it can be ignored.

Best regards,

Tatsuhiro Tsujikawa

---

From: William Chan (陈智昌) <[email-masked]>
Date: Tue, 13 Aug 2013 16:42:12 +0200

Great catch again! You're describing behavior that we've implemented in
Chromium (
https://code.google.com/p/chromium/codesearch#chromium/src/net/spdy/spdy_proxy_client_socket.cc&q=spdy_proxy_client_socket.cc&sq=package:chromium&l=368)
to get SPDY proxies to work with CONNECT, but we never wrote out in the
SPDY spec, since we've been bad people and have forgotten to write
everything down. I think we may need to separately discuss HTTP CONNECT
mapping to HTTP/2. I suggest opening an issue for this.

---

From: Roberto Peon <[email-masked]>
Date: Tue, 13 Aug 2013 08:21:11 -0700

++

On Tue, Aug 13, 2013 at 7:42 AM, William Chan (陈智昌)
<[email-masked]>wrote:

> On Tue, Aug 13, 2013 at 4:23 PM, Tatsuhiro Tsujikawa <
> [email-masked]> wrote:
>
>> Hi,
>>

---

From: Martin Thomson <[email-masked]>
Date: Tue, 13 Aug 2013 18:35:56 +0100

On 13 August 2013 15:42, William Chan (陈智昌) <[email-masked]> wrote:
> I suggest opening an issue for this.

https://github.com/http2/http2-spec/issues/230

Just spit-balling, but I think that the '*' form is a better way to
deal with this, i.e.,

:method = CONNECT
:scheme https
:host example.org:443
:path *

The trick here is that the DATA frames that form the "request body"
constitute a separate channel that is reliable, in-ordered, etc...
This channel can be used to perform a TLS handshake and run any
protocol you like.

---

From: William Chan (陈智昌) <[email-masked]>
Date: Tue, 13 Aug 2013 19:53:59 +0200

> https://github.com/http2/http2-spec/issues/230

Thanks!

> Just spit-balling, but I think that the '*' form is a better way to

All in all, I don't particularly care. That said, I don't know if it's a
big deal, but I think there might be information loss that way. AIUI, the
Host header may be present even for CONNECT. So my only concern is possible
compatibility issues, which would be silly, but maybe possible.

> This channel can be used to perform a TLS handshake and run any

Yep, sorry for not spelling this out before.

---

From: Ilari Liusvaara <[email-masked]>
Date: Tue, 13 Aug 2013 22:24:33 +0300

> This channel can be used to perform a TLS handshake and run any

Here are some semantics one could apply to it:

DATA <---> Data TX/RX.
END_STREAM <----> Half-close
RST_STREAM <---> Reset connection   

Possibly some connection control messages using HEADERS?

As note, this sort of stuff could also be applicable to upgrade
(internally emulate TCP connection)?

-Ilari

---

From: Martin Thomson <[email-masked]>
Date: Tue, 13 Aug 2013 21:40:18 +0100

> END_STREAM <----> Half-close

That reminds me.  Treating this as an approximation for a TCP
connection isn't perfect if you permit the stream to remain half
closed for a significant period.  I think that you have to treat
END_STREAM as being equivalent to a TCP FIN.

> (internally emulate TCP connection)?

I see no reason to support upgrade.  We're providing plenty of other
hooks for upgrading prior to commencing HTTP/2.0.  You can use ALPN or
HTTP/1.1 Upgrade.

---

From: Ilari Liusvaara <[email-masked]>
Date: Wed, 14 Aug 2013 00:06:21 +0300

> END_STREAM as being equivalent to a TCP FIN.

At least for some things running over TCP one has to be able to half-
close the connection and precive peer's half-close or it just plain
won't work.

> hooks for upgrading prior to commencing HTTP/2.0.  You can use ALPN or

I don't mean X -> HTTP/2.0 upgrade, but upgrading a stream from HTTP/2.0
to something else.

E.g. Full-blown websockets implementation (as opposed to potentially
simplified one built into HTTP/2.0).

-Ilari

---

From: James M Snell <[email-masked]>
Date: Tue, 13 Aug 2013 14:14:54 -0700

While I'm certainly not going to spend any time arguing this one much,
my vote is to deprecate CONNECT in HTTP/2 with the recognition that
doing so removes a feature that currently exists in HTTP/1. Later, a
better approach for handling the use case can be determined.

On Tue, Aug 13, 2013 at 7:42 AM, William Chan (陈智昌)
<[email-masked]> wrote:
> Chromium
> (https://code.google.com/p/chromium/codesearch#chromium/src/net/spdy/spdy_proxy_client_socket.cc&q=spdy_proxy_client_socket.cc&sq=package:chromium&l=368)
> to get SPDY proxies to work with CONNECT, but we never wrote out in the SPDY
> spec, since we've been bad people and have forgotten to write everything
> down. I think we may need to separately discuss HTTP CONNECT mapping to
> HTTP/2. I suggest opening an issue for this.
>
>
> On Tue, Aug 13, 2013 at 4:23 PM, Tatsuhiro Tsujikawa <[email-masked]>
> wrote:
>>
>> Hi,
>>

---

From: Martin Thomson <[email-masked]>
Date: Tue, 13 Aug 2013 22:18:56 +0100

> simplified one built into HTTP/2.0).

For the most part, that really isn't much different to what we've
described for CONNECT.

Upgrade implies something else, that is: I want to move from the
current protocol (HTTP/2.0) to some other protocol (WS).  That's
non-trivial in a multiplexed protocol, even if HTTP/2.0 isn't really
multiplexed in a fundamental way.

---

From: Martin Thomson <[email-masked]>
Date: Tue, 13 Aug 2013 22:20:27 +0100

> better approach for handling the use case can be determined.

That's certainly a valid answer.  But we have folks here with some
experience with CONNECT in SPDY.  Maybe their experience will provide
enlightenment.

---

From: William Chan (陈智昌) <[email-masked]>
Date: Tue, 13 Aug 2013 23:22:09 +0200

As far as I'm concerned, you're welcome to do whatever you want with a
CONNECT tunnel. If you want to layer WebSockets on top of it, go for it.
There are lots of other considerations with regard to that (see
https://groups.google.com/forum/#!topic/spdy-dev/rwOh5dH4ibU for some
previous discussion). I think this is probably out of scope for HTTP/2
discussion, at least at this early stage.

---

From: William Chan (陈智昌) <[email-masked]>
Date: Tue, 13 Aug 2013 23:31:46 +0200

James, can you explain why you prefer deprecating CONNECT? It's a fairly
widely used HTTP feature. Browsers tunnel SSL to the origin server over
HTTP proxies using CONNECT. It seems rather bad to remove this capability.
Or do you mean you just think we should file an issue (already done) and
address it later?

Note that Chromium indeed already supports this functionality and it's been
in use in production for awhile now, at least by Googlers using Chrome.

---

From: Ilari Liusvaara <[email-masked]>
Date: Wed, 14 Aug 2013 00:33:01 +0300

> CONNECT tunnel. If you want to layer WebSockets on top of it, go for it.

There is a fundamential difference between CONNECT and upgrade: CONNECT
designates a target host, whereas upgrade does not.

Also, existing websockets runs using upgrade, not CONNECT...

-Ilari

---

From: William Chan (陈智昌) <[email-masked]>
Date: Tue, 13 Aug 2013 23:41:20 +0200

I read this email and was like...yes...those are factual statements...why
are you saying them...

Then I re-read the earlier emails and realized you haven't been talking
about CONNECT at all, you want to do HTTP Upgrade within a stream. My bad
for misunderstanding you earlier.

I do believe it's doable...although I'm not sure it's desirable :)

---

From: James M Snell <[email-masked]>
Date: Tue, 13 Aug 2013 14:51:26 -0700

I mean that, at some point, we ought to sit down and figure out a
better approach to handling this case because using CONNECT as
currently defined is really not a great fit and I'd rather not try to
just shoe horn something in with a bunch of new exception cases. If
Google has this working, please provide a detailed description on list
of what's being done so we can evaluate it.

---

From: Roberto Peon <[email-masked]>
Date: Tue, 13 Aug 2013 14:54:41 -0700

It would be changing the semantics of UPGRADE to be stream instead of
connection, but yes.
The main source of complexity here is that one cannot jettison the HTTP/2
parsing, etc. state machine when one gets an UPGRADE, but must concatenate
the state machines.

I can understand use-cases for wanting to do these kinds of things (e.g. if
TLS 1.3 doesn't happen or doesn't have an encrypted handshake and proxies
are mucking with the list of negotiated protocols).

Alternately, it could mean for the entire connection, but that has its own
uglyness-- the server would likely want to hold off on doing the upgrade
until after it has served the responses to the requests...
Anyway you cut it, there is some uglyness here.
-=R

---

From: William Chan (陈智昌) <[email-masked]>
Date: Wed, 14 Aug 2013 00:05:22 +0200

http://www.chromium.org/spdy/spdy-proxy

The CONNECT tunnel is implemented as Tatsuhiro-san described (the mapping
is admittedly unclean). There are some complexities when you tunnel SPDY
over SPDY due to multiplexing and flow control, but otherwise I think the
rest is relatively straightforward, and indeed there are open source SPDY
proxy implementations that support this like Tatsuhiro-san's.

---

From: James M Snell <[email-masked]>
Date: Tue, 13 Aug 2013 15:13:43 -0700

> http://www.chromium.org/spdy/spdy-proxy

Internet-Draft, please?

- James

> The CONNECT tunnel is implemented as Tatsuhiro-san described (the mapping is
> admittedly unclean). There are some complexities when you tunnel SPDY over
> SPDY due to multiplexing and flow control, but otherwise I think the rest is
> relatively straightforward, and indeed there are open source SPDY proxy
> implementations that support this like Tatsuhiro-san's.
>

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Wed, 14 Aug 2013 16:54:58 +0900

> Just spit-balling, but I think that the '*' form is a better way to
I'm fine with these values as long as it is clearly defined in the spec..
but proxy may not use the value of scheme in this case because
the tunneled protocol may be unknown to the proxy.

> This channel can be used to perform a TLS handshake and run any

This is what the SPDY proxy currently does as far as I know.

Best regards,

Tatsuhiro Tsujikawa

---

From: Martin Thomson <[email-masked]>
Date: Wed, 14 Aug 2013 17:39:57 +0100

> the tunneled protocol may be unknown to the proxy.

Yeah, on further consideration, I don't think that :scheme is necessary.

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Sun, 18 Aug 2013 10:11:37 +0200

> better approach for handling the use case can be determined.

That would be mightily nice. A lot of interop problems are directly caused
by endpoints trying to bypass security rules via CONNECT, while
intermediaries try to maintain some sanity

-- 
Nicolas Mailhot