Subject: Re: site-wide headers

From: Martin Thomson <[email-masked]>
Date: Sat, 1 Oct 2016 17:27:14 +1000

I agree that this is a benefit that it picks up all the warts and
wrinkles that come with using HTTP header fields.  I just wanted to
make clear that this isn't 100% positive.

On 1 October 2016 at 16:12, Willy Tarreau <[email-masked]> wrote:
> Hi Martin,
>
> On Wed, Sep 28, 2016 at 09:00:05PM +1000, Martin Thomson wrote:
>> (https://tools.ietf.org/html/draft-nottingham-site-wide-headers-00)
>>
>> I like this approach because it is more obviously composable into an
>> existing system at the consuming end.  I especially like that the
>> format is without opinion about its contents.  That makes it quite
>> powerful.
>>
>> I dislike this approach (in contrast to the JSON-based
>> origin-policy[1]) because it uses header fields.  Of course that makes
>> it better suited to HTTP.
>
> In fact that's what I find powerful here. I know *many* places where
> these headers are set by the front reverse-proxy, simply because it
> ensures that they're uniform across all the servers. But it also
> happens that there are exceptions (eg: for static some servers or
> certain unrelated applications). With this mechanism, there's almost
> nothing to change in the way it works. The admin will just have to
> add "HS" to the responses instead of adding all these header fields,
> when the reverse-proxy notices that the client provided the valid SM
> field. And it also ensures the proxy an simply remove HS: and replace
> it with all appropriate headers when it comes from a server where it's
> not appropriate at all (you know, some application developers like to
> copy-paste when they don't know).
>
> So in fact, it supports everything already supported today but the
> smarter way. It's really nice in my opinion.
>
> Cheers,
> Willy

---

From: Eitan Adler <[email-masked]>
Date: Sat, 1 Oct 2016 11:11:32 +0300

> (https://tools.ietf.org/html/draft-nottingham-site-wide-headers-00)

a) Strong +1 to using rfc5785 for site-wide items. A couple of concerns though:

b) We should mention something about headers on the site-headers file
itself. For example how long should this file be cached, etc.

c) I don't understand why we have HS or SM tags at all. So long as the
site-headers file returns 200, has contents, and has the correct media
type those headers should be used.

d) Do we want to create a whitelist of headers that should exist in
site-headers and have user agents validate it? At the moment the draft
lists a small number of blacklisted items.

e) If a single page injects additional headers do they override
site-headers? For example can
https://example.com/~user/evil/page.html send
   Strict-Transport-Security: max-age=0 ; includeSubDomains

and win?

-- 
Eitan Adler

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 1 Oct 2016 11:18:22 +0200

> type those headers should be used.

Simply so that :
  1) the server knows whether or not it should use this method instead
     of sending all the relevant headers

  2) the client knows whether the server assumed the client is going to
     use them or if the server doesn't expect the client to use them

> lists a small number of blacklisted items.

It may do more harm than good because everyone has a different use case.
Suggestions and examples are more appropriate here (that's what the draft
does by the way).

Regards,
Willy

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 4 Oct 2016 19:01:40 +1100

Hey Martin,

CC:ing Mike to make sure he sees this. Mike, there's been a few messages on the HTTP WG list, you might want to have a look.

> On 28 Sep. 2016, at 9:00 pm, Martin Thomson <[email-masked]> wrote:
> 
> it better suited to HTTP.  I dislike that the format is without
> opinion about its contents.  That makes it quite powerful.

Mike and I have been talking about the proposals in back channel a bit. A modified origin-policy could take yet another path -- it could not try to convey headers at all, but instead just describe site-wide policy in JSON, defining new syntax to supplant existing site-wide headers where necessary. 

(the downside here being a nasty period where sites have to set up .well-known files *and* send response headers, and not mess it up)

I'm not yet convinced which approach is better, but I'm fairly convinced that it should be one or the other ("just headers" or something new), not both.

One of the advantages of doing this as "just headers" is that, as others on the thread have noted, it's easier for a hoster / CDN / etc. to insert/enforce policy in a way that's more purely mechanical.

I'm probably going to continue working on the site-wide draft as well as send Mike some friendly pull requests; hopefully as we discuss this problem area, the right approach will become more apparent.

> On balance, I think that this is a distinct improvement.

Please disambiguate "this"?

> One thing that this can't do but the origin-policy does is do
> something to manage the downside of CORS.  The idea that you might
> give out a pass to avoid CORS preflight is very appealing.  As far as
> I can tell, this proposal cannot address that problem.  It would be
> justifiable to say that this is a completely different problem that
> might build on this work, but it's a very appealing problem to look
> at.

Yes, I'd intended anything like that to be separately defined, as a new HTTP header, with the site-wide approach.

> (It's tempting to suggest that you could include a label that just
> applies to preflight requests, but I don't know how to solve the
> origin enumeration problem.  origin-policy seems to punt on that.)
> 
> 
> ---Nits and suggestions
> 
> I think that this needs a little more rigour in the definition of the
> format and the algorithm.  They should at least match.  It's unclear
> from the algorithm how blank lines (CRLF CRLF) are handled.

Agreed, if it goes forward.

> The character set for labels could be expanded a little to include
> [0-9\-_] so that you can base64url things you might have lying around
> to produce the label (or just keep the label very short).

Nod.

> You should also note another reason to keep things out of the h2
> header table: any change to the table eventually pushes entries out,
> necessitating re-creation.  This is more manageable because it is
> directly under control.

Yeah, that's not explicitly stated, but it's a strong motivation.

> The draft should note that these advantages come with a cost in memory
> to clients and that clients that receive unreasonably large header
> sets can/should pretend that they don't exist.

Nod. The nice thing is that they're per-origin, not per-connection, but it does have tradeoffs. 

> [1] https://wicg.github.io/origin-policy/

--
Mark Nottingham   https://www.mnot.net/

---

From: Martin Thomson <[email-masked]>
Date: Tue, 4 Oct 2016 20:53:38 +1100

> I'm not yet convinced which approach is better, but I'm fairly convinced that it should be one or the other ("just headers" or something new), not both.

This is the part that I care about most.  I'm glad you are slowing
working towards a coherent strategy here. :)

> Please disambiguate "this"?

This draft, and more generally the direction that you and Mike are
exploring.  We've done a lot to "optimize" HTTP over time, and this
could do good things for both performance AND operational use.

(BTW, I'm very interested in having a good transition strategy here.
Expect me to give you heaps of flack over anything that requires
careful matching of two policies.  Especially if that synchronization
doesn't involve copy-paste; I expect tools in a wide array of
languages to accompany any proposal that looks like that.)