Subject: 2.2. Interaction with "https" URIs | Re: Op-sec simplification

From: Kari Hurtta <[email-masked]>
Date: Tue, 1 Nov 2016 19:22:02 +0200 (EET)

> | https://github.com/httpwg/http-extensions/pull/254
> | 
> | The main changes:
> | 
> |  - the .well-known resource is a flat list of origins
> 
> ( no comment about that yet. )
> 

On earlier version there was concern that if
same connection was used both on "http" 
and "https" scheme then these are
confused by server.

Therefore there was requirement that clients must 
not send "http" requests on a connection that would
ordinarily be used for "https" requests.

There was  "mixed-scheme" flag what disabled that
requirement.

Now there is no that requirement:

https://tools.ietf.org/html/draft-ietf-httpbis-http2-encryption-08#section-2.2

| 2.2.  Interaction with "https" URIs
|
|   When using alternative services, requests for resources identified by
|   both "http" and "https" URIs might use the same connection, because
|   HTTP/2 permits requests for multiple origins on the same connection.
|   Because of the potential for server confusion about the scheme of
|   requests (see Section 4.4), clients MUST NOT send "http" requests on
|   a connection prior to successfully retrieving a valid http-
|   opportunistic resource that contains the origin (see Section 2.3).
|   The primary purpose of this check is to provide a client with some
|   assurance that a server understands this specification and has taken
|   steps to avoid being confused about request scheme.

What is wanted ?  Options

	• No concern; that check that http://{...}/.well-known/http-opportunistic
          succeed tells enough that there is no confusion

        • Forbid using same connection for "http" requests
	  which would ordinarily be used for "https" requests.

	• Forbid using same connection for "http" requests
          which would ordinarily be used for "https" requests
	  but allow it if there is "mixed-scheme" -member (or
	  some other flag).

        • Forbid using same connection for "http" requests
	  which would ordinarily be used for "https" requests
	  unless same connection which was tested
	  for http://{...}/.well-known/http-opportunistic
	  are also after that tested:

		∘ That https requests for /.well-known/http-opportunistic
		  on same host {...} does not return same object

		∘ Note that if that test fails, then that connection
		  is no longer be good for http requests either.
		
/ Kari Hurtta

---

From: Mike Bishop <[email-masked]>
Date: Tue, 1 Nov 2016 17:33:25 +0000

I think the case for TBD2 is that the client sent an "ambiguous" request -- that is, connecting over port 443 and not specifying http:// or https://, but just sending e.g. GET /resource.

The problem is, if that server is also an alternate for https://www.example.com, that's not an ambiguous request -- it's a clear request for an https:// resource.  The server can't detect that unless the https:// origin isn't supported there, in which case you're likely to get an existing error (probably 400 for our current code, but a good argument could be made for 421 again).  If the https:// origin is supported there, the client will get a response and never know it asked for (and got) the wrong thing.  The server simply can't fix a broken client, and TBD2 doesn't provide a useful mechanism because the server can't distinguish.

That makes it all the more important that clients know whether the server can support scheme-mixing *and then do it.*  Unless we want to require servers to have distinct scheme-always-required ports, I don't think this is a problem solvable server-side.

-----Original Message-----
From: Martin Thomson [mailto:[email-masked]] 
Sent: Monday, October 31, 2016 10:54 PM
To: Kari Hurtta <[email-masked]>
Cc: Mark Nottingham <[email-masked]>; HTTP working group mailing list <[email-masked]>
Subject: Re: Op-sec simplification

On 1 November 2016 at 16:25, Kari Hurtta <[email-masked]> wrote:
> That may be good idea. (This spec requires scheme and http/1.1 spec 
> does not allow scheme to be used. )

I have tried to capture this information in a PR:
  https://github.com/httpwg/http-extensions/pull/257

> |   | TBD1  | Scheme Not Allowed            | Section 2.2 |

We can probably avoid doing that on the basis that we have 421.

> |   | TBD2  | Scheme Required               | Section 2.1 |

The case for this seems weak.  You have to have a resource that is only available on the cleartext version of the site, and you have to use opp-sec, and the client has to be very silly.  I would prefer to use 404 here.  That is, assume that the client asked for a secure resource (https://example.com/http-only) which doesn't exist; rather than asking for the unsecured resource (http://example.com/http-only) which might.

---

From: Kari Hurtta <[email-masked]>
Date: Tue, 1 Nov 2016 19:35:37 +0200 (EET)

Martin Thomson <[email-masked]>: (Tue Nov  1 07:54:29 2016)
> The case for this seems weak.  You have to have a resource that is
> only available on the cleartext version of the site, and you have to
> use opp-sec, and the client has to be very silly.  I would prefer to
> use 404 here.  That is, assume that the client asked for a secure
> resource (https://example.com/http-only) which doesn't exist; rather
> than asking for the unsecured resource (http://example.com/http-only)
> which might.

TBD2 Scheme Required   

may happen when listener is for op-sec only -- it expects only
http requests over TLS and listener does not serve https requests.

Scheme is required then because it is what is
required for op-sec. 

Or that is 421 (Misdirected Request) also (as you suggested for
TBD1 Scheme Not Allowed).

/ Kari Hurtta

---

From: Martin Thomson <[email-masked]>
Date: Wed, 2 Nov 2016 11:52:35 +1100

>           succeed tells enough that there is no confusion

I think that this is sufficient.

The reason that I think this is OK, and the reason for removing a
mixed-scheme flag was that the risk comes from confusion.  That
confusion exists as soon as an http:// request is made over a TLS
connection.

It's not that much worse when things are mixed.  The bad examples of
routing based on the first request are bad for many reasons.  They are
simply cause to NOT provide the .well-known resource.

>           which would ordinarily be used for "https" requests.

We've already established that this is difficult to define properly.
And it's unnecessarily limiting.

>           but allow it if there is "mixed-scheme" -member (or

As above.

>                   is no longer be good for http requests either.

As above, just more complicated :)

---

From: Martin Thomson <[email-masked]>
Date: Wed, 2 Nov 2016 11:54:10 +1100

> I think the case for TBD2 is that the client sent an "ambiguous" request -- that is, connecting over port 443 and not specifying http:// or https://, but just sending e.g. GET /resource.

I think my rationale could be restated more simply as: "there is
always a scheme, just that HTTP/1.1 requires that it be implicit".

BTW, Kari's example of an opportunistic-only server can be handled by 421.

---

From: Kari Hurtta <[email-masked]>
Date: Wed, 2 Nov 2016 07:48:04 +0200 (EET)

Martin Thomson <[email-masked]>: (Wed Nov  2 02:52:35 2016)
> simply cause to NOT provide the .well-known resource.

Except of course on case where that .well-known was the first request.

In these cases on these bad examples that http: -probe determined
routing.  I guess that bad examples are NOT concern for op-sec, but it 
may be concern for browser (some secure cookie is then served
to http: -routing for example when broser sent it to for
https: -scheme).

In theses case they do not mess http: prosessing.

I asked earlier that

https://lists.w3.org/Archives/Public/ietf-http-wg/2016OctDec/0104.html

was that routing based on the first request  only concern:

| Is reason that
| 	∘ confusion happens if "http" follows "https", or 
|	∘ scheme is determined only on first request ?

You answered 
https://lists.w3.org/Archives/Public/ietf-http-wg/2016OctDec/0105.html

| The point of this is to cover off any problems that might arise from
| connection reuse.  It's clumsy. 

and

| we're incapable of detecting all cases where the server decides to do
| crazy things - I'm sure that I can devise a server architecture that
| will fail for any solution we devise - we have to instead take steps
| that we think are reasonable.

/ Kari Hurtta

---

From: Martin Thomson <[email-masked]>
Date: Wed, 2 Nov 2016 17:13:38 +1100

> to http: -routing for example when broser sent it to for

I'm willing to say that (contrary to previously-held opinions), that
this is a risk that is worth taking.  If we find that the probe
triggers a bad route AND that bad route responds favourably to that
probe, THEN we have to assume that the bad route is smart enough to
handle requests with a slightly odd scheme.

Let's say that we have a gateway that routes based on the first
request, there's not really much that the gateway does that will
prevent the origin server from receiving a request for the .well-known
resource.  If the origin server is still stupid enough to provide a
positive response - for an origin that it shouldn't be expecting to be
serving requests for - then I don't think there is any way that they
can be helped.

If a gateway routes on path, and .well-known goes to a server that
advertises op-sec, and other resources don't, that's pathological.

If the gateway routes on almost every request, but occasionally gets
lazy, that's pathological.

---

From: Kari Hurtta <[email-masked]>
Date: Wed, 2 Nov 2016 19:24:22 +0200 (EET)

( was off-list )

Martin Thomson <[email-masked]>: (Wed Nov  2 17:37:55 2016)
> On 2 November 2016 at 18:39, Kari Hurtta <[email-masked]> wrote:
> > Hmm. Is there requirement that authority must not serve
> > /.well-known/http-opportunistic with https -origin unless
> > http and https -origins provide identical resourses and processing ?
> 
> The draft currently doesn't say anything about the
> https://.../.well.known/http-opportunistic resource.  If the resource
> happens to say something about https origins, that's kinda
> nonsensical.  Maybe I should point that out.  Do you think that this
> would help?
> 
>   https://github.com/httpwg/http-extensions/pull/261
> 
> (To paraphrase what I added: this is "http" opportunistic, not "https"
> opportunistic.)
> 
> I mean, we could insist that the https:// resource not exist, but I
> don't think we need that on the basis that we already have a pretty
> strong signal, and it keeps things simple.

https://github.com/httpwg/http-extensions/pull/261/files/dc24113b22b545ba112ebbc707b38ef3396c9f8a?short_path=fd50b7c#diff-fd50b7c5883e57d650fa3ac7f47c12f9

| This document does not define semantics for "http-opportunistic" resources 
| on an https origin, nor does it define semantics if 
| the resource includes https origins.

That is OK, but it is mostly orthogonal about that
what I was meaning. 

https://lists.w3.org/Archives/Public/ietf-http-wg/2016OctDec/0367.html

|>         • No concern; that check that http://{...}/.well-known/http-opportunistic
|>           succeed tells enough that there is no confusion
|
| I think that this is sufficient.
|
| The reason that I think this is OK, and the reason for removing a
| mixed-scheme flag was that the risk comes from confusion.  That
| confusion exists as soon as an http:// request is made over a TLS
| connection.
| 
|  It's not that much worse when things are mixed.  The bad examples of
| routing based on the first request are bad for many reasons.  They are
| simply cause to NOT provide the .well-known resource.

Except if .well-known resource exists also on https: -origin
then bad routing still provides .well-known resource.

So strong signal is that server tells that it supports Opportunistic HTTP 
Security.

Reading http://{authority}/.well-known/http-opportunistic does not
test that there is no confusion. 

( Also it is useless for http-client to check non-existence of 
  https://{authority}/.well-known/http-opportunistic
  because existence of 
  https://{authority}/.well-known/http-opportunistic
  is OK as far op-sec is considered. )

( If http: and https: origins are identical then there is no
  bad routing; just one routing. )

Maybe that is enough.

/ Kari Hurtta

---

From: Erik Nygren <[email-masked]>
Date: Wed, 2 Nov 2016 16:02:38 -0400

> handle requests with a slightly odd scheme.

It's not just the "confusion" factor.  There are other reasons why a server
operator may not want mixed-scheme (ie, mixed origin) on the same
connection.  Clients must at least expect that a server will 421 for
mixed-scheme on a connection, and the perf impact and bug risk from this
could be a blocker to some using Opp Sec.

An example of why this could be bad would be a CDN server that terminates
both HTTP and HTTPS over TLS but demuxes them such that HTTPS requires TLS
to content origin but HTTP is allowed to go cleartext to content origin.
When a single TLS connection demuxes to a mixture of TLS and cleartext
traffic, this feels like asking for increased trouble and attack surfaces.
Prohibiting mixed-scheme on the incoming connection makes this feel much
safer.

Another example would be client cert authentication for HTTPS requests
against a TLS connection.  Having these also apply to HTTP requests feels
"weird" somehow (and could be another attack surface).

      Erik

---

From: Kari Hurtta <[email-masked]>
Date: Thu, 3 Nov 2016 20:31:23 +0200 (EET)

Erik Nygren <[email-masked]>: (Wed Nov  2 22:02:38 2016)
> "weird" somehow (and could be another attack surface).

Hmm.  

Simplest /.well-known/http-opportunistic response, which
includes that functionlity, contains object as root.

Member names are origins.  Members have string
either "mixed-scheme" or "distinct-scheme" as value.

If origin member have value "mixed-scheme" then 
client may use same connection for "http" and
"https" requests.

If origin member have value "distinct-connection"
then client must reserve distict connection for
http requests of Opportunistic Security. Where
that connection is not used for other purposes
(for example normal "https" requests).

If origin member have some other value, then
this specification does not define semantic
for it. Client should not use opportunic
security for that origin unless client
does not know semantic of that origin member
value.

Example is

   GET http://example.com/.well-known/http-opportunistic HTTP/1.1
   Host: example.com

   HTTP/1.1 200 OK
   Content-Type: application/json
   Connection: close

   {
     "http://www.example.com": "mixed-scheme",
     "http://example.com": "distinct-connection"
   }

/ Kari Hurtta

---

From: Martin Thomson <[email-masked]>
Date: Fri, 4 Nov 2016 11:16:35 +1100

> Prohibiting mixed-scheme on the incoming connection makes this feel much

I am almost inclined to say that you don't get to use the feature if
you are concerned about this causing issues of that sort.  Or, as some
of us have discussed, a new h2 setting that prohibits coalescing might
be a simpler option.

Kari's solution works, though it opens other possibilities, and I'm
concerned we're off down the rabbit hole again:

{ "http://...": "mixed-scheme", --> open season
  "http://...": "single-scheme", --> only one scheme per connection
  "http://...": "dedicated-connection" } --> only one origin per connection

---

From: Kari Hurtta <[email-masked]>
Date: Fri, 4 Nov 2016 06:23:36 +0200 (EET)

Martin Thomson <[email-masked]>: (Fri Nov  4 02:16:35 2016)
>   "http://...": "dedicated-connection" } --> only one origin per connection

Only one origin per connection ("dedicated-connection") is 
better to use some other mechanism because that look 
someting which is not limited to Opportunistic HTTP Security.
It looks like something which is wanted also for "https".

"single-scheme" looks like rougly same than "distinct-connection" 
on my suggestion.  "distinct-connection" was 'own connection for 
Opportunistic HTTP Security' and that imply that only scheme used 
on that connection is "http". 

And "Opportunistic HTTP Security" really does not want say
usage of other connections, which are not used by it, so 
"single-scheme" does not say more than "distinct-connection".

/ Kari Hurtta

---

From: Martin Thomson <[email-masked]>
Date: Sun, 6 Nov 2016 08:51:10 +1100

> It looks like something which is wanted also for "https".

Yeah, I'm wondering if this isn't a) enough for this mixed scheme
concern, and b) enough for the origin frame.

---

From: Kari Hurtta <[email-masked]>
Date: Thu, 10 Nov 2016 07:07:15 +0200 (EET)

[ was
  To: Kari Hurtta <[email-masked]>
  cc: Erik Nygren <[email-masked]>, 
                HTTP working group mailing list <[email-masked]>
]

Martin Thomson <[email-masked]>: (Sat Nov  5 23:51:10 2016)
> concern, and b) enough for the origin frame.

I do not know.

If that does not is like:

« We can't support/use "Opportunistic HTTP Security"
  because concern XXX. Resolution of concern XXX
  requires support of "other mechanism" (not on
  "Opportunistic HTTP Security") and if our client
  supports "Opportunistic HTTP Security" 
  that is no indication support of "other mechanism". »

I'm also just wondering.

/ Kari Hurtta