Subject: Issue #356: Form-encode Expect-CT report bodies?

From: Emily Stark <[email-masked]>
Date: Tue, 6 Jun 2017 09:45:42 -0700

I'm looking for some feedback on https://github.com/httpwg/
http-extensions/issues/356.

Expect-CT violation reports are currently specified as POST requests with
JSON bodies. When implemented in a web browser, such reports should
arguably send CORS preflights, because the content type is not
CORS-safelisted (https://www.w3.org/TR/cors/#simple-header).

But sending CORS preflights for these requests doesn't really make sense
from a web browser architecture perspective: CT compliance is checked as
part of certificate verification and connection setup, divorced from the
context one needs to send a CORS preflight (such as an Origin header). This
is not just a theoretical layering issue; implementing preflights for
Expect-CT reports in Chrome would be pretty challenging.

I only see two options to resolve this, both of which seem bad to me:

a) Leave the reporting part of the spec as it currently is, and leave it up
to the UA to decide whether further operations such as CORS preflights are
needed for sending reports. This would basically leave it to us in Chrome
to decide either that Expect-CT reports should not be subject to CORS
restrictions, or that we should not ship reporting. (I'm uncomfortable with
this option because it somewhat jeopardizes the only active implementation
effort that I know of.)

b) The disgusting option: disguise Expect-CT reports as form submissions,
which are not subject to CORS preflighting. This would mean the report body
is sent as hostname=blah&port=443&... or we could even just send
expect-ct-report=<stringified JSON blob>.

Thanks,
Emily

---

From: Patrick McManus <[email-masked]>
Date: Wed, 7 Jun 2017 12:23:05 +0200

my gut reaction: why would an expect-ct report be more subject to cors than
something like OCSP? (assuming it wasn't driven directly from content)

On Tue, Jun 6, 2017 at 6:45 PM, Emily Stark <[email-masked]> wrote:

> I'm looking for some feedback on https://github.com/httpwg/h
> ttp-extensions/issues/356.
>
> a) Leave the reporting part of the spec as it currently is, and leave it
> up to the UA to decide whether further operations such as CORS preflights
> are needed for sending reports. This would basically leave it to us in
> Chrome to decide either that Expect-CT reports should not be subject to
> CORS restrictions, or that we should not ship reporting. (I'm uncomfortable
> with this option because it somewhat jeopardizes the only active
> implementation effort that I know of.)
>

---

From: Martin Thomson <[email-masked]>
Date: Wed, 7 Jun 2017 15:39:46 +0200

Can we go to first principles?

The URL is controlled by a site.  Though it doesn't include ambient
authority (I hope), it can still be used to poke at things on the same
side of the firewall as a client.  That would seem to qualify it for
CORS.

On 7 June 2017 at 12:23, Patrick McManus <[email-masked]> wrote:
>> I'm looking for some feedback on
>> https://github.com/httpwg/http-extensions/issues/356.
>>
>> JSON bodies. When implemented in a web browser, such reports should arguably
>> send CORS preflights, because the content type is not CORS-safelisted
>> (https://www.w3.org/TR/cors/#simple-header).
>>
>> Chrome to decide either that Expect-CT reports should not be subject to CORS

---

From: Patrick McManus <[email-masked]>
Date: Wed, 7 Jun 2017 17:36:15 +0200

> Can we go to first principles?
I can learn, forget, and relearn cors more times in a day than I care to
count :)

fwiw, it seems an inconsistent state of affairs here..

csp report-uri is similar.. afaict the current spec applies fetch (and
therefore cors) rules to it after mandating no credentials.. but also
afaict that's not the reality of implementations. The OCSP example has a
lot of similarities too, though the uri is less controllable (but after a
few layers from the PKI root who really knows..). when talking through the
basic cors principles (credential leakage, read-side leaks, write-side
triggers) I didn't really see a problem with a non content UA request doing
this based on a policy decision to implement a spec but I can certainly see
how every UA might come to a different conclusion. Indeed I went back and
forth a few times. So Emily, I certainly see where your use of
"jeopardizes" comes from with option a. This IETF forum is only likely to
muddy the waters further, I'm afraid as we aren't the keepers of cors.

which kind of brings us to the more httpbis parts of your issue - the self
described option b. Those are indeed gross. Perhaps more a little less
offensive, but still non ideal, workarounds would be:
 * an empty string origin request header and an
access-control-allow-origin: * response
 * content-type: text/plain (its not right, but seems strictly better than
a form)

hth. (I'm prepared to be told those are bogus too.)

-P

> arguably
> CORS

---

From: Emily Stark <[email-masked]>
Date: Wed, 7 Jun 2017 09:04:58 -0700

Re: CSP and OCSP, yes, those are open issues as well. Indeed, the Expect-CT
issue spawned a long and as-yet-unresolved debate about OCSP:
https://github.com/whatwg/fetch/issues/530

For CSP, the plan is to move reporting to the in-progress Reporting API,
which was intended to send CORS preflights but doesn't currently in either
spec or implementation. The spec issue is
https://github.com/WICG/reporting/issues/41, and it's still an open
question how preflights would be implemented in Chrome.

HPKP reporting has the same issue. Whatever we do for Expect-CT would most
likely make sense to apply for HPKP reports as well.

I would be okay with either of your proposed workarounds. I lean slightly
towards text/plain because I suspect the empty-Origin-header proposal will
lead to implementors doing special-purpose implementations of a subset of
CORS for this purpose, skipping preflight caching and such.

---

From: Emily Stark <[email-masked]>
Date: Fri, 9 Jun 2017 07:38:35 -0700

Does anyone else have an opinion? If not, I'll probably go with text/plain.

On Wed, Jun 7, 2017 at 9:04 AM, Emily Stark <[email-masked]> wrote:

> Re: CSP and OCSP, yes, those are open issues as well. Indeed, the
> Expect-CT issue spawned a long and as-yet-unresolved debate about OCSP:
> spec or implementation. The spec issue is https://github.com/WICG/
> reporting/issues/41, and it's still an open question how preflights would
> be implemented in Chrome.
>
>> which kind of brings us to the more httpbis parts of your issue - the
>> self described option b. Those are indeed gross. Perhaps more a little less
>>  * content-type: text/plain (its not right, but seems strictly better
>> than a form)
>>

---

From: Martin Thomson <[email-masked]>
Date: Fri, 9 Jun 2017 16:42:23 +0200

> Does anyone else have an opinion? If not, I'll probably go with text/plain.

After considering this, I would prefer to have this added to the CORS
exception list in the same way that CSP reporting is.  It is better to
have an accurate MIME type with an exception and accompanying analysis
of why it is safe to send these payloads than it is to just have the
spec try to route around the problem (tempting and easy as that might
be).

---

From: Emily Stark <[email-masked]>
Date: Fri, 9 Jun 2017 07:53:45 -0700

> spec try to route around the problem (tempting and easy as that might

CSP reporting isn't added to the CORS whitelist. It's been in violation of
CORS for years and there are some vague plans to fix it by sending
preflights, but adding it to the whitelist hasn't really been discussed.
Anne has said that he prefers not to add more to the whitelist, which I
think is a reasonable stance. (see
https://lists.w3.org/Archives/Public/public-webappsec/2017Feb/0009.html --
though to be fair, the same text/plain idea is rejected in that thread as
well)

In addition to the fact that there's not really any principled reason for
expanding the whitelist, it would mean that, say, an XHR can send the new
header value, which shouldn't really be allowed.

---

From: Martin Thomson <[email-masked]>
Date: Fri, 9 Jun 2017 17:01:26 +0200

> header value, which shouldn't really be allowed.

Ahh, I remembered that discussion, but failed to get that critical
detail.  My point is that if you want to avoid a preflight, then make
sure that you have an analysis to back it up, don't just dodge the
issue by using a whitelisted MIME type.

If that means using a preflight, then great.  If we go back to first
principles, the "POST to intranet site" case would seem to suggest
that some preflighting is warranted.

Ultimately, I want the same answer for this and for CSP reports.  I
would rather not add this to the pile of violating mechanisms though.

---

From: Emily Stark <[email-masked]>
Date: Fri, 9 Jun 2017 08:10:43 -0700

> > https://lists.w3.org/Archives/Public/public-webappsec/2017Feb/0009.html

These are quite different scenarios though. With CSP, sending preflights is
totally doable and makes sense, except for the fact of the widely deployed
reporting servers that would break if we suddenly started requiring them to
respond to preflights. Expect-CT and HPKP are done as part of certificate
verification and it's not clear that they should be governed by CORS any
more than OCSP requests or any request made by the OS in the course of
loading a webpage. I agree with your "first principles" argument that if
they are cross-origin requests triggered in the course of loading a web
page, then they can be used by malicious web content and should be subject
to CORS... but at the same time I'm not sure that it's practical to require
that any request at any layer of the system triggered during the course of
loading a web page should go through Fetch and send preflights if needed.

---

From: Martin Thomson <[email-masked]>
Date: Fri, 9 Jun 2017 16:22:30 +0100

The "first principles" notion also refers to the fact that the URL is
attacker-controlled.  Being hard to do isn't a great reason not to
preflight, though it is something to consider.

---

From: Emily Stark <[email-masked]>
Date: Fri, 9 Jun 2017 08:42:27 -0700

The URL can be attacker-controlled in the OCSP case too. (
https://github.com/bifurcation/expect-ct/issues/18#issuecomment-295867109)

As I said in my first message, implementing true preflights would violate
very core architectural principles in Chrome, and would jeopardize our
ability to ship an implementation. Maybe there are other implementors who
would like to chime in to the contrary, but as of now I'm not very inclined
to specify something that can't realistically be implemented.

So that leaves the null-origin preflight that Patrick suggested as the only
option on the table. Do you prefer that, Martin?

On Fri, Jun 9, 2017 at 8:22 AM, Martin Thomson <[email-masked]>
wrote:

> > On Fri, Jun 9, 2017 at 8:01 AM, Martin Thomson <[email-masked]
> >
> > wrote:
> >>
> >> > CSP reporting isn't added to the CORS whitelist. It's been in
> violation
> >> > of
> >> > https://lists.w3.org/Archives/Public/public-webappsec/
> 2017Feb/0009.html
> >> > --

---

From: Martin Thomson <[email-masked]>
Date: Sat, 10 Jun 2017 09:55:35 +0100

For those who are reading this and eating popcorn, this is probably
worth reading also... <https://github.com/whatwg/fetch/issues/530>

> to specify something that can't realistically be implemented.

I wanted to poke at that a little.  Expect-CT is a header field, sent
by a particular origin.  You store the tuple of origin, the expect CT
mode (none, report, require), and the report URI somewhere.  Then when
you connect to an origin you retrieve any stored tuple and compare
what you get with what you expect.  I don't see how a stack would be
unable to preflight at that point.  It has the information it needs
for a preflight check.

I recognize that a particular piece of software might be constructed
in a way that makes this difficult, but it can't be impossible.

Question: how do you manage the checks when you are using alternative
services?  I expect that you need to store a target origin for the
connection attempt, rather than using the host and port or SNI and
port that you are connecting to.  (This doesn't complicate things
regarding the question at hand, but I don't see any text on this
point.)

---

From: Patrick McManus <[email-masked]>
Date: Sat, 10 Jun 2017 11:19:32 +0200

First, I think it would be good for the expect-ct spec to give reporting
servers a little non normative advice to be ready for cors and preflights.
Something complementary to the client mention now.

second, cors clearly makes some distinction between UA content and
content-content.. request headers for example. The notion being that the UA
can effectively make at least some decisions about what will botch things
up compared to what arbitrary JS might do. A whole request body formatted
by the UA, such as done by expect-ct reporting, must fall on this spectrum
somewhere (though I'm not certain where) in between. I agree with Emily
(and others in the thread cited) that not everything is subject to cors and
I'm not sure that "content controls the uri" is the sole determining factor.

But, as interesting as this discussion is, httpbis doesn't make cors
decisions. In that spirit the null origin stm the 'right way to use http'
if you can't figure out the real origin.. but text/plain (while being a
less true to the type'd nature of http) goes down the road of the "the UA
believes it has the right to declare this content safe enough to not
require cors" which also seems kinda reasonable overall if less pure http.

---

From: Emily Stark <[email-masked]>
Date: Sat, 10 Jun 2017 09:29:05 -0700

> unable to preflight at that point.  It has the information it needs

What's missing is the origin that triggered the request: in Fetch language,
the request's client's origin, which populates the `Origin` header. Hence
the suggestion for a preflight with a null Origin header and
Access-Control-Allow-Origin: *. Which would be possible, and I think safe,
but a bit hacky. Probably not more hacky than any of the other solutions
under discussion, though. :)

Having the request's client's origin available at connection setup time
would require quite some gymnastics/re-design of the Chrome network stack,
which late-binds sockets and does not know which request(s) a particular
socket will be used for at connect time.

> regarding the question at hand, but I don't see any text on this

That's a good question, I have to think about that a bit.