Subject: :scheme, was: consensus on :query ?

From: Greg Wilkins <[email-masked]>
Date: Thu, 24 Jul 2014 14:35:26 +1000

While we are talking about decomposing the uri into it's component
parts.....

why are we sending :scheme?

It's not something that I would trust from a client anyway.

If the connection is not TLS and the request says https, then I'm not going
to believe it.  The only way I'll upgrade a request to https is with some
secret handshake with my SSL offloader via a special privileged port that
will probably nail all requests to https regardless of what the header says.

If the connection is TLS and the scheme says http, then I guess that tells
me something... that it is not TLS end to end, but then I don't know if I'm
meant to be trusting the hop or the end to end.    It's landing on my
server as https... so I guess it is.

Or is scheme meant to be optional, as in h1 allowing an absolute URL to be
sent in the request line?

-- 
Greg Wilkins <[email-masked]>
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.

---

From: "Adrien de Croy" <[email-masked]>
Date: Thu, 24 Jul 2014 05:02:00 +0000

so that when a client sends the http2 equivalent of

GET ftp://ftp.somewhere.com/file /HTTP/1.1

to a proxy, we can do it.

------ Original Message ------
From: "Greg Wilkins" <[email-masked]>
To: "Matthew Kerwin" <[email-masked]>
Cc: "Adrien de Croy" <[email-masked]>; "Zhong Yu" 
<[email-masked]>; "Martin Thomson" <[email-masked]>; 
"HTTP Working Group" <[email-masked]>
Sent: 24/07/2014 4:35:26 p.m.
Subject: :scheme, was: consensus on :query ?

>If the connection is not TLS and the request says https, then I'm not 
>going to believe it.  The only way I'll upgrade a request to https is 
>with some secret handshake with my SSL offloader via a special 
>privileged port that will probably nail all requests to https 
>regardless of what the header says.
>
>If the connection is TLS and the scheme says http, then I guess that 
>tells me something... that it is not TLS end to end, but then I don't 
>know if I'm meant to be trusting the hop or the end to end.    It's 
>landing on my server as https... so I guess it is.
>
>Or is scheme meant to be optional, as in h1 allowing an absolute URL to 
>be sent in the request line?
>
>--

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 24 Jul 2014 15:15:44 +1000

> to a proxy, we can do it.

That's a reasonable usage.

But distinguishing between http and https is not.   Can't we just not send
the scheme for HTTP and if we do then it should just be http and never
https.      At the very least we should point out that :scheme is not a
trusted value and just because it says https does not mean the request is
secure.

There is plenty of code out that that implements the equivalent of

 boolean isSecure() { return "https".equals(getScheme()); }

https://www.google.com.au/search?q=isSecure+%22https%22.equals+getScheme%28%29

---

From: Zhong Yu <[email-masked]>
Date: Thu, 24 Jul 2014 00:19:50 -0500

If a request self-claims that it is HTTPS, I think the server should
just take its word for it. If a client lies about the scheme, the
client does it at its own peril, and it should have the freedom to do
so. If an intermediary (possibly a man-in-the-middle) lies about the
scheme, there's not much the server can do about it.

The problem in HTTP/1 is that the server has no reliable way to know
whether the request was originated as HTTPS at the client end,
(assuming client/intermediaries are all honest), because the request
could have gone through multiple intermediaries that alternate
TLS/PLAIN connections.

There is an unofficial header "X-Forwarded-Proto" to record the
upstream scheme, which the server should take into account when it's
present (e.g. for Request.isSecure() API). However it is very unclear,
from reading proxy/load-balancer documents, how would the header
record multiple entries, and how are these entries corresponding to
entries in the "X-Forwarded-For" header.

Zhong Yu
bayou.io

On Wed, Jul 23, 2014 at 11:35 PM, Greg Wilkins <[email-masked]> wrote:
> meant to be trusting the hop or the end to end.    It's landing on my server
> as https... so I guess it is.
>

---

From: Yutaka Hirano <[email-masked]>
Date: Thu, 24 Jul 2014 14:20:16 +0900

WebSocket over HTTP/2 will use :scheme: ws and :scheme: wss

On Thu, Jul 24, 2014 at 2:02 PM, Adrien de Croy <[email-masked]> wrote:

> Cc: "Adrien de Croy" <[email-masked]>; "Zhong Yu" <[email-masked]>;
> "Martin Thomson" <[email-masked]>; "HTTP Working Group" <
> [email-masked]>
> going to believe it.  The only way I'll upgrade a request to https is with
> some secret handshake with my SSL offloader via a special privileged port
> that will probably nail all requests to https regardless of what the header
> says.
>

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 24 Jul 2014 08:11:47 +0200

> sent in the request line?

In my opinion it's not used to know if the connection was secure or not, but :

  1) for proxies, what scheme to use on the other side (remember the
     discussions we had about secure proxies to which we could send
     http:// requests over SSL)

  2) for application servers, guess whether links in pages should be
     built with http:// or https:// schemes so that the page displays
     consistently in the browsers despite the presence of an SSL offloader
     in front of the server. Currently this is handled using X-Forwarded-Proto
     or X-SSL: True or whatever header field an offloader might add to inform 
     the server.

Thus I'd say that it only reflects the scheme the client desires to
use, nothing more.

Regards,
Willy

---

From: Amos Jeffries <[email-masked]>
Date: Thu, 24 Jul 2014 19:54:00 +1200

On 24/07/2014 5:15 p.m., Greg Wilkins wrote:
> But distinguishing between http and https is not.

Yes it is. There are non-browser clients out there today which rely on
the proxy performing TLS. These clients send URI begining with https://
instead of http:// to the proxy.

Amos

---

From: Amos Jeffries <[email-masked]>
Date: Thu, 24 Jul 2014 19:58:06 +1200

> could have gone through multiple intermediaries that alternate

scheme is not about front-end. It is about what *backend* protocol the
proxy / second-to-last hop should use to contact the origin server.

Amos

---

From: "Martin Nilsson" <[email-masked]>
Date: Thu, 24 Jul 2014 14:34:48 +0200

> entries in the "X-Forwarded-For" header.

The Forwarded header (RFC 7239) combines the different X-Forwarded-*  
headers into one list to keep them synchronized.

/Martin Nilsson

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

---

From: Erik Nygren <[email-masked]>
Date: Thu, 24 Jul 2014 08:51:56 -0400

This is a key part of:

    http://tools.ietf.org/html/draft-ietf-httpbis-http2-encryption-00

In particular, if a connection comes in over TLS it could now be
either http or https.
The :scheme header is a way to know which it is, especially as the
scheme is a critical part
of the URI and Origin tuple.

Note that many implementations make assumptions about TLS == https, so
much of the work
in draft-ietf-httpbis-http2-encryption will be (safely!) decoupling these.

Perhaps this doc needs some clarifying text to make this explicit?

     Erik

---

From: Patrick McManus <[email-masked]>
Date: Thu, 24 Jul 2014 09:32:25 -0400

> why are we sending :scheme?
because its part of the resource name. http://example/foo and
https://example/foo are different resources - scheme disambiguates them.

> It's not something that I would trust from a client anyway
>

then why do you trust :path?

Mostly, I jest.

But just as you check the security context against the :path, you also
check the security context against :scheme.. and sure, receiving https
without tls is something 7230 says is an error. I think 6455 says the same
thing about wss. However just because TLS is present doesn't mean https is
the only acceptable scheme.

---

From: Martin Thomson <[email-masked]>
Date: Thu, 24 Jul 2014 06:47:03 -0700

> Perhaps this doc needs some clarifying text to make this explicit?

https://github.com/httpwg/http-extensions/issues/12

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 24 Jul 2014 10:12:26 -0700

On Jul 23, 2014, at 10:15 PM, Greg Wilkins wrote:
> But distinguishing between http and https is not.   Can't we just not send the scheme for HTTP and if we do then it should just be http and never https.      At the very least we should point out that :scheme is not a trusted value and just because it says https does not mean the request is secure.

Actually, people use many different schemes to access HTTP services.
You don't see them because it can't be communicated yet.

>  boolean isSecure() { return "https".equals(getScheme()); }

Yep, and they are all guesses.  It doesn't actually work in all cases,
but there is no alternative in HTTP/1.  It would be a shame not to fix a
known bug in HTTP/1.

....Roy

---

From: Erik Nygren <[email-masked]>
Date: Thu, 24 Jul 2014 14:21:40 -0400

> https://github.com/httpwg/http-extensions/issues/12

I'd been under the assumption that http-scheme-over-TLS would only be
allowed over HTTP/2? We should make this crystal clear. For example,
if Alt-Svc "h2" is used to move http up to TLS then if the TLS fails
to negotiate HTTP/2 then the client must fall-back to HTTP/1.1 over
clearntext.
Similarly, it should be crystal clear that https scheme must never be
allowed over cleartext (h2c or http/1.1).

Does this cover the issues above here?

---

From: Martin Thomson <[email-masked]>
Date: Thu, 24 Jul 2014 11:33:41 -0700

On 24 July 2014 11:21, Erik Nygren <[email-masked]> wrote:
> allowed over HTTP/2?

I'll open that issue.  We currently have no explicit restriction that
prevents this.  I don't think that we have any reason to say
HTTP/2-only.  I also don't think that we need a specific exclusion for
HTTP/1.1, which is the other way we might cut this (so that we could
retain the feature for some theorized HTTP/5, which may or may not be
in active development for some major browser).

That said, Mozilla doesn't plan to use oppsec for HTTP/1.1, at least
in the short to medium term.

---

From: Erik Nygren <[email-masked]>
Date: Thu, 24 Jul 2014 14:53:12 -0400

I think the requirement is that http-scheme-over-TLS must only be done
in a way where the client and server agree on the scheme in a way that
works hop-by-hop and also works with legacy clients.  In http/2 the
:scheme makes this clear.  In prior versions (eg, http/1.1) it's not
clear there's a sane way (eg, new headers) that unaware intermediaries
can't be made confused by an adversary on the client or server side?

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 25 Jul 2014 12:14:31 +1200

> can't be made confused by an adversary on the client or server side?

Why would intermediaries be confused?

http-scheme-over-TLS is only useful when communicating to an explicit
proxy. So the request URI is required to be in absolute-form where the
scheme: is explicitly sent as http:// regardess of the TLS connection it
arrives on.

FYI: Squid has been doing this for TLS between proxies and from
supporting UA for over a decade with zero problems. If this restriction
becomes a "MUST NOT" then you just made FastCGI, Perl, lynx, Chrome,
Squid, curl, wget, and any number of other non-browser implementations
non-compliant.

The only confusion seems to be in certain browser implementers minds as
a whole lot of FUD comes back for reasons not to implement every time
the end users request better security on proxy connections. The only
real technical problem I have heard in the whole debate is that changing
the generic proxy config UI on Windows is a hard problem due to re-use.

Amos

> 
> On Thu, Jul 24, 2014 at 2:33 PM, Martin Thomson wrote:
>> On 24 July 2014 11:21, Erik Nygren wrote:

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 25 Jul 2014 10:48:21 +1000

> the only acceptable scheme.

OK that makes sense.  I'll take this to the servlet expert group as I think
we should require that isSecure does more than check the scheme and makes
some effort to check context.

cheers

---

From: Erik Nygren <[email-masked]>
Date: Thu, 24 Jul 2014 23:29:58 -0400

With AltSvc, http-scheme-over-TLS is highly relevant to client-to-origin as
well.  What is the behavior of non-proxy origins to getting absolute http://
request URIs over TLS with HTTP/1.1?  Good point that this is normal for
proxies, but I'd guess that many non-proxy origins would be confused by an
absolute http:// request URI over TLS?  With HTTP/2 this is expected to be
a normal/typical to have browsers/clients send http-scheme-over-TLS to
origins after an AltSvc.

       Erik

> scheme: is explicitly sent as http:// regardess of the TLS connection it
...

---

From: Roland Zink <[email-masked]>
Date: Fri, 25 Jul 2014 09:12:51 -0400

I guess this is similar to the origin server getting a :authority it is not serving. The server can check it or simple minded just ignore it.

Roland

> Am 24.07.2014 um 23:29 schrieb Erik Nygren <[email-masked]>:
> 
> With AltSvc, http-scheme-over-TLS is highly relevant to client-to-origin as well.  What is the behavior of non-proxy origins to getting absolute http:// request URIs over TLS with HTTP/1.1?  Good point that this is normal for proxies, but I'd guess that many non-proxy origins would be confused by an absolute http:// request URI over TLS?  With HTTP/2 this is expected to be a normal/typical to have browsers/clients send http-scheme-over-TLS to origins after an AltSvc.
> 
>        Erik
> 
> 
>

---

From: Martin Thomson <[email-masked]>
Date: Fri, 25 Jul 2014 07:39:57 -0700

On 25 July 2014 06:12, Roland Zink <[email-masked]> wrote:
> I guess this is similar to the origin server getting a :authority it is not
> serving. The server can check it or simple minded just ignore it.

The server had better check its inputs.