Subject: Re: Migrating some high-entropy HTTP headers to Client Hints.

From: Mike West <[email-masked]>
Date: Mon, 7 Jan 2019 10:14:07 +0100

Hey folks!

This is something I think Chrome will be able to implement in Q1. To that
end, I'd like to come to some sort of consensus about where we'd like to
see the standardization work take place.

My personal preference would be to have distinct documents that describe
the rationale, give use cases/examples, and define the header together in
one package so that developers get a reasonable understanding of the thing
in itself. But I don't feel so strongly about it that I wouldn't take
Ilya's advice to work with folks in the WHATWG to integrate these hints
into Fetch. Likewise, I'd be happy to take Mark's advice, and define them
here (either in Client Hints or in distinct document) and merely reference
them from Fetch. I'm sure there are other constellations that might make
sense.

Since this group is currently responsible for both `User-Agent` and
`Accept-Language`, I'd appreciate y'all's advice on how you'd like me to
proceed.

Thanks!

-mike

On Wed, Dec 5, 2018 at 3:08 PM Ilya Grigorik <[email-masked]> wrote:

> Hey folks.
>
> On Sat, Dec 1, 2018 at 7:48 PM Mark Nottingham <[email-masked]> wrote:
>
>> Speaking personally --
>>
>> I could see splitting out the various task-specific bits into separate
>> documents (e.g., images).
>>
>> However, it's not good to ship a framework like this without having it
>> actually working for a real-world use case, so even if we split out all of
>> the various CHs into separate docs, I think we'd need to hold the "main"
>> document until at least one of those is ready.
>>
>> Just a thought -- replacing User-Agent is probably suitable for that test
>> case (in addition to the image-focused stuff, or perhaps instead of it). If
>> we do decide to do that, it might be suitable to put it in the core
>> document, since that seems like it's pretty central to what's going on here
>> (the current claims in the document about not replacing UA notwithstanding).
>>
>
> Just to clarify, we do have a shipping implementation of image hints
> (Viewport-Width, Width, DPR), as well as network related hints (ECT, RTT,
> Downlink). The latter set is defined in NetInfo [1] spec and the proposal
> is to spec processing definitions for image hints within Fetch+HTML to
> resolve current UA implementation and processing gaps — this would resolve
> and prevent questions similar to [2], [3]. On that note, the basic
> integration for image hints is already in Fetch (see steps 6+7 and related
> plumbing in [4]) but instead of linking to the IETF draft for definitions,
> the goal is to define those concepts directly against relevant HTML+Fetch
> concepts.
>
> Which is to say, this is editorial shuffling and it doesn't change or
> affect current shipping implementation of either sets of hints. I'm excited
> by Mike's proposal but I don't think we need to block the framework on that
> set of hints: we already have two existing shipping sets active in the wild.
>
> [1] https://wicg.github.io/netinfo/
> [2] https://github.com/httpwg/http-extensions/issues/698
> [3] https://github.com/httpwg/http-extensions/issues/697
> [4] https://fetch.spec.whatwg.org/#fetching
>
>
>> Chair hat on -- what I did notice was that when the update for CH was
>> read in Bangkok, *many* WG participants expressed surprise at the direction
>> you were taking it in; most people seemed to think that this document was
>> almost done in its current form, and there was concern that forming WG
>> consensus on that was being disregarded. So whatever you do here, please
>> make sure you get buy-in on the list, and make sure you coordinate with the
>> chairs. Continuing this discussion and moving towards a common idea of what
>> the doc(s) should include, when we should ship them, etc. sounds like a
>> good start.
>>
>
> Yup, fair feedback and will do. The change in course is based on recent
> discussions and issues that were raised as we were trying to shepherd the
> document through last stages.
>
> On Mon, Dec 3, 2018 at 1:49 AM Mike West <[email-masked]> wrote:
>
>> +Anne van Kesteren <[email-masked]> for the Fetch discussion. Hi, Anne!
>>
>> On Fri, Nov 30, 2018 at 6:00 PM Ilya Grigorik <[email-masked]>
>> wrote:
>>
>>> +Mike West <[email-masked]> I like your proposal for User-Agent and
>>> Accept-Language. Most of the CH plumbing is already in place (or close to)
>>> in HTML spec, WDYT of defining those hints directly in the HTML spec?
>>>
>>
>> For `User-Agent`: HTML currently relies on Fetch's "default `User-Agent`
>> value <https://fetch.spec.whatwg.org/#default-user-agent-value>" (for `
>> navigator.userAgent
>> <https://html.spec.whatwg.org/#dom-navigator-useragent>`), and Fetch is
>> also responsible for setting the `User-Agent` value (in step 5.11 of
>> https://fetch.spec.whatwg.org/#http-network-or-cache-fetch). That seems
>> like the right direction for the dependency, so I could imagine defining
>> the entire mechanism in Fetch if this group is willing to point off in that
>> direction in a future version of RFC7231 that deprecates `User-Agent`.
>>
>
> +Mike West <[email-masked]> yep, defining these in Fetch makes sense to
> me. I'll defer to the group on the deprecation.
>
> ig
>

---

From: Anne van Kesteren <[email-masked]>
Date: Mon, 7 Jan 2019 14:16:15 +0100

Heya, happy new year,

On Mon, Jan 7, 2019 at 10:14 AM Mike West <[email-masked]> wrote:
> Since this group is currently responsible for both `User-Agent` and `Accept-Language`, I'd appreciate y'all's advice on how you'd like me to proceed.

I don't really care strongly, but "reference from Fetch" seems too
weak as there's a fair number of details that matter here that are
still lingering for Client Hints. E.g., can they be set by developers,
can they be observed by service workers, and how do they interact with
CORS and the same-origin policy? Then for User-Agent and
Accept-Language it would be nice if the primitives upon which those
are built are defined in a central place so they can be reused by
other consumers of that information. For this defining something in
IETF-space is less than ideal as RFCs lend themselves much less to the
inevitable refactoring you want to be doing. It's probably okay and
you could maybe leave it up to some kind of integration standard to
define "system language", "system architecture", ... (Which would be
Fetch for browsers and browser-like user agents.)

Hope that helps,

Anne

---

From: Ilya Grigorik <[email-masked]>
Date: Mon, 7 Jan 2019 21:30:54 -0800

Having worked with trying to shepherd a few of these (hints) through
implementation, my recommendation would be to:

   1. Spec the framework in the IETF draft: how to declare which hints you
   want to receive, how those prefs are stored, expected and recommended cache
   behaviors
   2. Spec the individual hints alongside relevant implementation specs
   (HTML, Fetch, or feature specific specs like NetInfo)
      - This eliminates all the corner+edge cases that Anne highlighted and
      allows us to iterate and define new hints as necessary

On that note, I think we're ~70% of the way there already. We have
in-flight PR's to update all the necessary plumbing in HTML and Fetch, we
already pulled out network related hints into NetInfo, and we can integrate
remaining hints into the HTML spec itself, which will also clarify all the
outstanding CH questions we have on GitHub. For User-Agent specifically, we
can+should define it directly in Fetch.

WDYT, reasonable?

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 8 Jan 2019 16:33:31 +1100

From an HTTP WG perspective - does anyone object to the plan that Ilya lays out below for Client Hints?

> On 8 Jan 2019, at 4:30 pm, Ilya Grigorik <[email-masked]> wrote:
> 
> Having worked with trying to shepherd a few of these (hints) through implementation, my recommendation would be to: 
> 	• Spec the framework in the IETF draft: how to declare which hints you want to receive, how those prefs are stored, expected and recommended cache behaviors
> 	• Spec the individual hints alongside relevant implementation specs (HTML, Fetch, or feature specific specs like NetInfo)
> 		• This eliminates all the corner+edge cases that Anne highlighted and allows us to iterate and define new hints as necessary
> On that note, I think we're ~70% of the way there already. We have in-flight PR's to update all the necessary plumbing in HTML and Fetch, we already pulled out network related hints into NetInfo, and we can integrate remaining hints into the HTML spec itself, which will also clarify all the outstanding CH questions we have on GitHub. For User-Agent specifically, we can+should define it directly in Fetch.
> 
> WDYT, reasonable?
> 

--
Mark Nottingham   https://www.mnot.net/

---

From: Martin Thomson <[email-masked]>
Date: Tue, 08 Jan 2019 16:50:02 +1100

No real objection, though a question that might be important:

"the" IETF draft, or just "an" IETF draft?  I say this because client hints seems to be in some sort of perpetual limbo and I don't want to extend that unnecessarily.

---

From: Yoav Weiss <[email-masked]>
Date: Tue, 8 Jan 2019 13:50:42 +0100

The intent is to modify the current draft and to move the bits that are
related to specific hints and their web-related processing model to HTML
and Fetch.

On Tue, Jan 8, 2019 at 8:47 AM Martin Thomson <[email-masked]> wrote:

> "the" IETF draft, or just "an" IETF draft?  I say this because client
> hints seems to be in some sort of perpetual limbo and I don't want to
> extend that unnecessarily.
>
> > >     • Spec the framework in the IETF draft: how to declare which hints
> you want to receive, how those prefs are stored, expected and recommended
> cache behaviors
> > >     • Spec the individual hints alongside relevant implementation
> specs (HTML, Fetch, or feature specific specs like NetInfo)
> > >             • This eliminates all the corner+edge cases that Anne
> highlighted and allows us to iterate and define new hints as necessary

---

From: Martin J. Dürst <[email-masked]>
Date: Sun, 13 Jan 2019 11:56:37 +0000

Hello Mike, others,

On 2018/11/29 19:22, Mike West wrote:
> Hey folks,
> 
> Section 9.7 of RFC7231 <https://tools.ietf.org/html/rfc7231#section-9.7>
> rightly notes that some of the content negotiation headers user agents
> deliver in HTTP requests create substantial fingerprinting surface. I think
> it would be beneficial if we took steps to reduce their prevalence on the
> wire, and Client Hints looks like a reasonable infrastructure on top of
> which to build.

Sorry to be very late, and with a rather basic question:

The point about substantial fingerprinting is definitely important. But 
what's the difference, in terms of fingerprinting, between the following 
two alternatives?

a) The browser sending out Accept-Language,... to a server interested in 
fingerprinting.

b) A server interested in fingerprinting sending out an Accept-CH header 
with the equivalent information, even if the server doesn't need e.g. 
language information for serving the request.

Regards,   Martin.

---

From: Mike West <[email-masked]>
Date: Mon, 14 Jan 2019 06:26:46 -0800

> language information for serving the request.

Hey Martin, good question!

In short, the client hints infrastructure places a number of limitations on
the ways in which hints can be enabled for a given request. Some of those
are documented in
https://tools.ietf.org/html/draft-west-lang-client-hint-00#section-3. I'll
hit the highlights below:

1.  No plaintext delivery; hints can be delivered over encrypted channels,
which means that we'll substantially reduce the scope of leakage to network
attackers.

2.  Client hints are moving to a delegation model whereby "first-party"
responses (top-level navigations, etc) can opt-into receiving hints, but
subresource requests cannot. That is, assuming the server you're talking
about above lives at `https://fingerprinter.com/`, it can obtain access
when the user directly navigates to `https://fingerprinter.com/`, but can
only obtain access in third-party contexts (e.g. when embedded as a frame
on `https://publisher.com/`) if the top-level domain explicitly delegates
the privilege to it. Note that there's some ongoing work (described in
https://tools.ietf.org/html/draft-west-lang-client-hint-00#section-3.2) to
bring the various specifications up to date with this decision. +Ilya
Grigorik <[email-masked]>, +Yoav Weiss <[email-masked]>, and
friends are working through those.

3.  Because this data is no longer broadcast by default, but must be
explicitly requested, the onus falls upon the requestor to justify the
request. The explicitness makes it easier for researchers to dig into data
usage, and, in the best case, brings abuses to light.

Does that answer your question?

-mike

---

From: Martin J. Dürst <[email-masked]>
Date: Tue, 15 Jan 2019 10:15:22 +0000

Hello Mike,

Many thanks for your explanations.

> Does that answer your question?

Partially. But let me be more specific about the threat scenario I'm 
thinking about. Web sites use all kinds of third party services, some of 
the main ones being advertising and analytics. All these services come 
with installation instructions. My (easy, I'd say) guess is that these 
installation instructions will include instructions to activate the 
necessary third-party opt-ins for the server in question for those 
third-party services that are interested in fingerprinting.

Given that many third-party services are interested in fingerprinting, 
and that many Web administrators follow instructions carefully, I'd 
guess that most sites will end up with fingerprinting third-party 
services anyway. Those sites not interested in fingerprinting didn't 
analyse the Accept... headers to begin with.

Regards,   Martin.

---

From: Ilya Grigorik <[email-masked]>
Date: Mon, 28 Jan 2019 17:07:18 -0600

> lays out below for Client Hints?

I'll take silence as lack of vocal objections and proceed as described..
Unless I hear evidence to the contrary by EOW / Feb 1st. :)

> analyse the Accept... headers to begin with.

Hi Martin.

To echo what Mike highlighted before, I don't think we're suggesting that
CH will eliminate fingerprinting. To your point, yes 3P providers will
likely request that site owners grant access to these hints, but that in
itself is already a significant step forward: hints are restricted to
secure transports (significantly reduced fingerprinting surface area for
unencrypted traffic); 1P must explicitly state what hints they want to
receive (auditing); 1P must explicitly delegate permission to 3Ps (auditing
and permission based access). Compare that to status quo today, where there
are no signals on what data is being requested and used by whom, and most
1P's being entirely unaware of which 3P's are collecting what data from
their users.

CH is not a magic bullet but the constraints it introduces — I think —
offer significant accountability and transparency improvements over status
quo.

ig

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 29 Jan 2019 11:06:51 +0900

> On 29 Jan 2019, at 8:07 am, Ilya Grigorik <[email-masked]> wrote:
> 
> I'll take silence as lack of vocal objections and proceed as described.. Unless I hear evidence to the contrary by EOW / Feb 1st. :)

I think that's a safe assumption; thanks.

> 
> On Tue, Jan 15, 2019 at 4:15 AM Martin J. Dürst <[email-masked]> wrote:
> To echo what Mike highlighted before, I don't think we're suggesting that CH will eliminate fingerprinting. To your point, yes 3P providers will likely request that site owners grant access to these hints, but that in itself is already a significant step forward: hints are restricted to secure transports (significantly reduced fingerprinting surface area for unencrypted traffic); 1P must explicitly state what hints they want to receive (auditing); 1P must explicitly delegate permission to 3Ps (auditing and permission based access). Compare that to status quo today, where there are no signals on what data is being requested and used by whom, and most 1P's being entirely unaware of which 3P's are collecting what data from their users.
> 
> CH is not a magic bullet but the constraints it introduces — I think — offer significant accountability and transparency improvements over status quo.
> 
> ig
> 
> 
> 
> 
>

---

From: Martin J. Dürst <[email-masked]>
Date: Tue, 29 Jan 2019 07:34:43 +0000

Hello Ilya,

> offer significant accountability and transparency improvements over status

Thanks for the explanations. If something similar can go into the 
relevant draft, in a "Privacy Considerations" section if there is one, 
and if not as part of the "Security Considerations" section or in some 
other appropriate place, then that would be great.

Regards,   Martin.

---

From: Ilya Grigorik <[email-masked]>
Date: Tue, 29 Jan 2019 08:55:45 -0600

> other appropriate place, then that would be great.

We do have one [1] but we don't explicitly spell out what we discussed here
and I agree that we should. I'll take an AI to add this when we tackle the
refactor.

thanks!

[1]
https://httpwg.org/http-extensions/draft-ietf-httpbis-client-hints.html#security-considerations