Subject: Re: Benjamin Kaduk's Discuss on draft-ietf-httpbis-messaging-16: (with DISCUSS and COMMENT)

From: Mark Nottingham <[email-masked]>
Date: Fri, 2 Jul 2021 18:09:16 +1000

Hi Ben,

> On 19 Jun 2021, at 11:17 am, Benjamin Kaduk <[email-masked]> wrote:
> 
> My complaint was
> attempting to focus specifically on the procedure for converting the
> request-target to the target URI, since in this one specific case of the
> absolute-form request target the procedure is that "the target URI is
> defined to have the same value as the request-target" and there seem to be
> some edge cases relating to the scheme selection.
> 
> At a high level, my point is that if a client sends a request with
> request-target of https://www.example.com over TCP-without-TLS, I expect
> the client to get back an HTTP 4xx response over that TCP channel, or maybe
> just a TCP RST.  In particular, I do not expect it to get a non-4xx
> response, and definitely not 200 with content from the "https" origin.
> There are actually quite a few places in the flow of processing the message
> where this could be done, across both -messaging and -semantics, but I
> don't remember a specific place that gives a clear direction to the server
> implementation to handle this scenario.

The server reconstructs the target URI and identifies the associated resource, as per 7.1. If the scheme is HTTPS but the transport is not TLS, that resource won't be found (at least in HTTP/1.1), so you'll get a 4xx, probably a 421 on a modern server. Some quick testing with Apache gives a 301 back to the same URI, which also works (I suppose).

>> Martin commented:
>> 
>>> I think that this question doesn't really read on -messaging, but more 
>>> on -semantics.  However, I am not seeing any requirement on the server 
>>> to ensure that the response it generates is secured.
>>> 
>>> s 4.2.2 of semantics -- 
>>> https://httpwg.org/http-core/draft-ietf-httpbis-semantics-latest.html#https.uri
>>> 
>>>> A client MUST ensure that its HTTP requests for an "https" resource are secured, prior to being communicated, and that it only accepts secured responses to those requests. Note that the definition of what cryptographic mechanisms are acceptable to client and server are usually negotiated and can change over time.
>>> 
>>> No similar requirement is made of the server.  It would be trivial to 
>>> impose a similar requirement on servers in that same paragraph.
>> 
>> I don't think that helps, at least in the case of HTTP/1.1. There, the server is responsible for setting the correct scheme for the target URI when a request is received; the security properties of the request and response follow from that. Effectively, it's not under attacker control.
> 
> (This is the part where I'm most confused.)
> Where is the requirement specified that the server set the correct scheme
> for the target URI?  If the procedure to set the taret URI is "use the
> scheme the client gave you", that seems like it's clearly under attacker
> control, so I don't understand how you reached the conclusion in the final
> sentence.  Were you maybe talking about h2 at that point and not HTTP/1.1?

Aha - I see what's happened. In 3.3 of 1.1 <https://httpwg.org/http-core/draft-ietf-httpbis-messaging-latest.html#reconstructing.target.uri> I was skimming by the initial text:

> The target URI is the request-target when the request-target is in absolute-form.

... whereas you obviously picked up on it. I agree that there's an issue here.

The decisions about absolute-form requests were made *way* back when. My reading of the archive (circa September and October 1995 -- I'm sure those that were there will correct me) is that Host headers were added to address the multiple-hosts-on-an-IP problem in a way that was backwards-compatible with HTTP/1.0, but because some folks wanted to enable the use of URNs, proxies were required to support and use the absolute form, so that URNs could be (theoretically) resolvable through them. That didn't happen, but it is possible to use e.g., FTP through a HTTP proxy as a result (last I looked).

I think the solution here is to restrict the statement above so that it only applies to proxies, and to add a requirement for origin servers (including gateways) to specifically check absolute-form URIs for alignment regarding the scheme.

Does that make sense to everyone (especially Roy, who has the most history here)?

>> However, I don't see any equivalent mechanism regarding :scheme in http/2 bis  or http/3. Off the cuff, I tend to think that security considerations about this probably belong on both of those specs.
> 
> That makes sense to me, as does spinning it off to a separate issue for
> h2-bis.  I will note that I didn't raise a concern for h3 and :scheme
> processing since h3 always requires secure transport and the failure
> case didn't seem very significant.

See https://github.com/httpwg/http2-spec/issues/867

Cheers,

--
Mark Nottingham   https://www.mnot.net/

---

From: Benjamin Kaduk <[email-masked]>
Date: Tue, 6 Jul 2021 17:45:33 -0700

Hi Mark,

> The server reconstructs the target URI and identifies the associated resource, as per 7.1. If the scheme is HTTPS but the transport is not TLS, that resource won't be found (at least in HTTP/1.1),…

That's reassuring to hear (and the 421 is intuitive, while 301 also makes
some kind of sense now that it's pointed out).  I'm not entirely sure where
the requirement for "if the scheme is HTTPS but the transport is not TLS"
comes from, though.  But perhaps I should keep reading in this mail...

> ... whereas you obviously picked up on it. I agree that there's an issue here.

Yes, exactly, that's what stuck out at me.  Sorry for not quoting it more
explicitly from the start.

> Does that make sense to everyone (especially Roy, who has the most history here)?

That makes perfect sense to me, but I expect that we should make strenuous
efforts to hear from others who have more knowledge of the history before
proceeding with it.

Thanks,

Ben

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 7 Jul 2021 16:48:01 +1000

Unless I missed it, we didn't create an issue to track this, so:
  https://github.com/httpwg/http-core/issues/896
... and created a PR to implement the solution outlined below.

Could folks please take a look and leave feedback (there or here)?

Cheers,

> On 7 Jul 2021, at 10:45 am, Benjamin Kaduk <[email-masked]> wrote:
>>

---

From: Julian Reschke <[email-masked]>
Date: Wed, 7 Jul 2021 11:03:14 +0200

Am 02.07.2021 um 10:09 schrieb Mark Nottingham:
> ... > The decisions about absolute-form requests were made *way* back when.
My reading of the archive (circa September and October 1995 -- I'm sure
those that were there will correct me) is that Host headers were added
to address the multiple-hosts-on-an-IP problem in a way that was
backwards-compatible with HTTP/1.0, but because some folks wanted to
enable the use of URNs, proxies were required to support and use the
absolute form, so that URNs could be (theoretically) resolvable through
them. That didn't happen, but it is possible to use e.g., FTP through a
HTTP proxy as a result (last I looked).
> ...

Not really.

My understanding is that support for "absolute-form" in request lines
has been a "MUST" level requirement for ages.

Do you want to change that? If not, we'll still have to define how to
compute the target URI from the request line, so special-casing the
support for proxies breaks that.

Best regards, Julian

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 7 Jul 2021 16:49:19 +0200

> has been a "MUST" level requirement for ages.

Yes, for me it appeared in RFC2068 as a MUST already.

> support for proxies breaks that.

Also, let's keep in mind that proxies can suddenly become servers when
they intercept some well-known URIs (internal management ones, etc).
Let's not start to special-case anything. And gateways are commonly
placed in front of proxies, having to deal with their absoluteURI
format as well.

Speaking about the ability to reconstruct the absolute URI, from within
a server there will always be some ambiguity left depending on the
knowledge about the connection: some tunnels that offload TLS can be
installed in front of them without touching any header field, and the
server doesn't see a secure connection in this case. If the port is
dedicated that may be OK, if it's not, after all, as long as the
authority remains the same... that's basically the same place :-/

And the announced port is not a discriminant. One interesting trick to
play with that I came across a week ago was to direct my browser to
http://127.0.0.1:443/ and https://127.0.0.1:80/. In such cases the
port is explicitly present in the Host field since it doesn't match
the scheme used, but if this passes through an SSL offloader the info
is definitely lost.

In short there has always been some ambiguity in that area and I don't
see how it could be addressed 24 years after RFC2068 has settled
everything in stone anyway.

Just my two cents,
Willy

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 8 Jul 2021 11:49:14 +1000

> support for proxies breaks that.

No, I don't want to change it. I don't see how it breaks anything, though.

IMO that absolute URIs in the start-line don't really help (and probably hurt) interoperability or security beyond that use case -- URNs failed to take off (thankfully), so the only practical use case for them is proxies effectively acting as gateways to non-HTTP protocols. I know that in theory HTTP supports other schemes, but in practice it doesn't (outside of that use case).

If we want to preserve this unused* capability in the protocol, we'll need to modify Semantics to do a check on the connection properties -- see proposed text at:
  https://github.com/httpwg/http-core/pull/898

Cheers,

* If someone wants to assert otherwise, please refer to an even moderately used server implementation that supports serving non-HTTP URIs over HTTP as an origin server.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 8 Jul 2021 12:34:15 -0700

On Jun 16, 2021, at 8:15 PM, Benjamin Kaduk <[email-masked]> wrote:
> 
> On Thu, Jun 17, 2021 at 12:32:21PM +1000, Martin Thomson wrote:
>> Restoring the CC list (sorry about that).
>> 
>> On Thu, Jun 17, 2021, at 12:13, Martin Thomson wrote:
>>> On Thu, Jun 17, 2021, at 11:53, Mark Nottingham wrote:
>>>>> On 17 Jun 2021, at 11:36 am, Benjamin Kaduk via Datatracker <[email-masked] <mailto:[email-masked]>> wrote:
>>>>> Let's discuss whether the currently specified procedures for
>>>>> reconstructing the target URI
>>> 
> I think that's a good insight that this question exposed something like a
> gap in -semantics.
> 
>>> s 4.2.2 of semantics -- 
>>> https://httpwg.org/http-core/draft-ietf-httpbis-semantics-latest.html#https.uri <https://httpwg.org/http-core/draft-ietf-httpbis-semantics-latest.html#https.uri>
>>> 
> I think at some formal level, adding analogous requirements for the server
> to only send secured responses on https resources would close off the
> harmful edge case I had in mind when balloting.  It seems like adding such
> a requirement would be a good idea, at least given the little thought I've
> put in so far.
> 
> My initial unease as I was reading -messaging stems roughly from a sense
> that it describes a scenario where the server is acting on "untrusted
> input" in a mode with "elevated privilege" (stretching the terminology,
> obviously), brought about by a potential mismatch between the in-band
> protocol element and the request's context.  Looking a bit more closely
> into the analogous parts of h3 is assauging that concern to a large extent,
> though, as h3 is explicitly not restricted to URIs with scheme http/https.
> That makes me feel more confident that the abstract model of an HTTP server
> needs to be paying significant attention to the scheme vs just replying on
> whatever transport the request came in on.
> 
>>> 
>>> s 4.3.3 of semantics is another place you might look for this: 
>>> https://httpwg.org/http-core/draft-ietf-httpbis-semantics-latest.html#https.origin <https://httpwg.org/http-core/draft-ietf-httpbis-semantics-latest.html#https.origin> 
> 
> Looking quickly, it seems that the discussion of server behavior mostly
> presumes that the server is acting in an https role, since the origin and
> its namespace are identified by the request target's host and port value.
> 
> Do you think anything would go in §7.4 of -semantics?  For reference, it
> currently has:
> 
>> Before performing a request, a server decides whether or not to provide
>> service for the target URI via the connection in which the request is
>> received. For example, a request might have been misdirected,
>> deliberately or accidentally, such that the information within a received
>> Host header field differs from the connection's host or port.
> 
> It doesn't seem like much of a stretch to add "or have been sent over a
> connection with attributes that are incompatible with the requirements of
> the requested scheme".
> 
> Thanks,
> 
> Ben

This discussion went off in the weeds and started talking about absolute URIs
and changes to basic proxy functionality. None of that is necessary.

The above recognition that there is a missing/implied requirement (that origin
servers obey their own identifier requirements) is a good one, and I agree that
the right fix is to add a specific requirement to that section 7.4 of Semantics
which matches the existing (passive) requirements on the identifier scheme.
Note that this doesn't change the protocol at all -- it reinforces the existing
requirements on https.

We are working on the text now in 

Cheers,

....Roy