Subject: Migrating some high-entropy HTTP headers to Client Hints.

From: Mike West <[email-masked]>
Date: Thu, 29 Nov 2018 11:22:27 +0100

Hey folks,

Section 9.7 of RFC7231 <https://tools.ietf.org/html/rfc7231#section-9.7>
rightly notes that some of the content negotiation headers user agents
deliver in HTTP requests create substantial fingerprinting surface. I think
it would be beneficial if we took steps to reduce their prevalence on the
wire, and Client Hints looks like a reasonable infrastructure on top of
which to build.

`User-Agent` and `Accept-Language` seem like particularly tasty and
low-hanging fruit, and I've sketched out two proposals as proofs of concept:

*   `User-Agent` could be represented as ~four distinct hints: `UA`,
`Model`, `Platform`, and `Arch`: https://github.com/mikewest/ua-client-hints is
a high-level explainer, and
https://tools.ietf.org/html/draft-west-ua-client-hints a sketchy ID for the
new headers.

*   `Accept-Language` could be represented as a `Lang` hint:
https://github.com/mikewest/lang-client-hint is a high-level explainer,
https://tools.ietf.org/html/draft-west-lang-client-hint an equally sketchy
ID for the new header.

I'd appreciate y'all's feedback. Thanks!

-mike

---

From: Thomas Peterson <[email-masked]>
Date: Thu, 29 Nov 2018 12:08:05 +0000

I would propose that all Accept* headers are included in Client Hints as 
all can be used for some level of fingerprinting, e.g. Accept can used 
to distinguish between desktop browsers (which typically have html/xml 
MIME types) and cURL/wget which by default have '*/*'. Many user agents 
also do their own guess work on response bodies anyway (such as looking 
at the magic number) to determine content type or encoding, so the 
impact of a "failed negotiation" of content can be limited.

Also, Is there a particular reason why Sec-CH-Lang omits Quality Values?

Regards

On 29/11/2018 10:22, Mike West wrote:
> Hey folks,
>
> Section 9.7 of RFC7231 
> <https://tools.ietf.org/html/rfc7231#section-9.7> rightly notes that 
> some of the content negotiation headers user agents deliver in HTTP 
> requests create substantial fingerprinting surface. I think it would 
> be beneficial if we took steps to reduce their prevalence on the wire, 
> and Client Hints looks like a reasonable infrastructure on top of 
> which to build.
>
> `Model`, `Platform`, and `Arch`: 
> https://github.com/mikewest/ua-client-hints is a high-level explainer, 
> and https://tools.ietf.org/html/draft-west-ua-client-hints a sketchy 
> ID for the new headers.
>
> https://github.com/mikewest/lang-client-hint is a high-level 
> explainer, https://tools.ietf.org/html/draft-west-lang-client-hint an 
> equally sketchy ID for the new header.
>

---

From: Mike West <[email-masked]>
Date: Thu, 29 Nov 2018 13:54:53 +0100

Thanks for the feedback!

On Thu, Nov 29, 2018 at 1:08 PM Thomas Peterson <[email-masked]>
wrote:

> MIME types) and cURL/wget which by default have '*/*'.

The philosophy in https://tools.ietf.org/html/draft-west-ua-client-hints is
that it's reasonable to expose basic information about the user agent (e.g.
it's Firefox, not cURL). That level of information seems quite difficult to
hide (given differences in behavior, network stacks, etc.) and quite
valuable to developers, which tips the balance for me towards exposing
brand and major version by default.

With that in mind, `Accept` and `Accept-Encoding` seem to be fairly static
in their relationship to the UA brand and version. Chrome more or less
hard-codes `Accept` and `Accept-Encoding` based on the kind of resource
being asked for, for instance (see
https://cs.chromium.org/chromium/src/net/url_request/url_request_http_job.cc?g=0&l=666
and
places like
https://cs.chromium.org/chromium/src/media/blink/resource_multibuffer_data_provider.cc?rcl=e53d19f7befd7927b6b9727dc88b9ee295c6fa05&l=110
 and
https://cs.chromium.org/chromium/src/content/renderer/loader/web_url_loader_impl.cc?rcl=e53d19f7befd7927b6b9727dc88b9ee295c6fa05&l=672
).

With the caveat that I'm sometimes prone to a myopic view of the world from
the standpoint of a web browser: `User-Agent` and `Accept-Language` seem to
contain significantly more entropy, and therefore feel like the right place
to start. I certainly wouldn't suggest that that's where we ought to stop.
:)

> Many user agents

https://tools.ietf.org/html/draft-west-lang-client-hint-00#section-4.3
addresses this. In a nutshell, it seems like cruft, and some widely-used
user agents (I spot-checked Chrome and Firefox) implement the weighting
mechanism as a function of the list order. That semantic makes sense to me,
and more doesn't seem to be necessary.

I might well be missing use cases here, which I'd be thrilled to hear about!

-mike

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 29 Nov 2018 14:44:52 +0000

Hiya,

> https://tools.ietf.org/html/draft-west-ua-client-hints a sketchy ID for the

It'd be good to see some progress along these lines, so thanks
for writing that.

If doing so, and in addition to greasing, could it be useful
to define some fixed strings that mean "NOT SAYING" for each
of the new headers?

I'm not sure it'd be a major win, but if those were defined,
then that might help a bit for whatever various UAs do when
they're supposedly in a more privacy-friendly mode - it'd
allow/encourage such UAs to look that little bit more alike.

Cheers,
S.

---

From: Martin Thomson <[email-masked]>
Date: Fri, 30 Nov 2018 10:22:20 +1100

I think maybe I was predisposed to not like this, but I do like it.
Not saying that I'm hugely enthusiastic about doing the CH part, but
the bit were User-Agent becomes fixed is really appealing.

One thing we might consider, if the timing works out, is having user
agents register their UA strings with us.  We can bake them into the
QPACK static table so that we can save bits.  I don't want to
privilege particular clients overly, so that only works if we have
broad acceptance of the plan.
On Thu, Nov 29, 2018 at 9:26 PM Mike West <[email-masked]> wrote:
>
> Hey folks,
>
> Section 9.7 of RFC7231 rightly notes that some of the content negotiation headers user agents deliver in HTTP requests create substantial fingerprinting surface. I think it would be beneficial if we took steps to reduce their prevalence on the wire, and Client Hints looks like a reasonable infrastructure on top of which to build.
>
> `User-Agent` and `Accept-Language` seem like particularly tasty and low-hanging fruit, and I've sketched out two proposals as proofs of concept:
>
> *   `User-Agent` could be represented as ~four distinct hints: `UA`, `Model`, `Platform`, and `Arch`: https://github.com/mikewest/ua-client-hints is a high-level explainer, and https://tools.ietf.org/html/draft-west-ua-client-hints a sketchy ID for the new headers.
>
> *   `Accept-Language` could be represented as a `Lang` hint: https://github.com/mikewest/lang-client-hint is a high-level explainer, https://tools.ietf.org/html/draft-west-lang-client-hint an equally sketchy ID for the new header.
>

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 30 Nov 2018 11:29:47 +1100

I, for one, welcome our new Client Hint overlords.

Personally, I'd like to see these integrated into the current CH document, rather than as separate drafts. CH still needs some work, so it's not like we're going to get it out the door tomorrow.

However, it seems like Ilya wants to go in a different direction, based upon the notes we received for Bangkok.

Ilya, your thoughts?

> On 29 Nov 2018, at 9:22 pm, Mike West <[email-masked]> wrote:
> 
> Hey folks,
> 

--
Mark Nottingham   https://www.mnot.net/

---

From: Mike West <[email-masked]>
Date: Fri, 30 Nov 2018 09:32:02 +0100

Hey Stephen!

On Thu, Nov 29, 2018 at 3:44 PM Stephen Farrell <[email-masked]>
wrote:

> https://github.com/mikewest/ua-client-hints is

We could certainly such a constant, but it's not clear how it would help. I
would expect user agents to simply not send the relevant hints, rather than
sending a hint that says "I'm not sending you a hint." Do you think it's
likely that some user agents might make difference choices?

-mike

---

From: Mike West <[email-masked]>
Date: Fri, 30 Nov 2018 09:37:57 +0100

> the bit were User-Agent becomes fixed is really appealing.

I am glad the concept overcame your potential predispositions! I think it's
a pretty natural extension of the CH infrastructure, and I think it's a
pretty reasonable compromise that gives us the breathing room to freeze the
UA string while not freezing out developers.

> broad acceptance of the plan.

That seems pretty reasonable to me, though I'm not sure how much space we
have to work with in the static table. Chrome's current user agent string
comes in at around ~150 characters, depending on platform. Taking that as a
baseline, how many strings could we reasonably encode? Can the table expand
to an extend that would allow it to cover enough of the UA market?

-mike

> > Section 9.7 of RFC7231 rightly notes that some of the content
> negotiation headers user agents deliver in HTTP requests create substantial
> fingerprinting surface. I think it would be beneficial if we took steps to
> reduce their prevalence on the wire, and Client Hints looks like a
> reasonable infrastructure on top of which to build.
> >
> and https://tools.ietf.org/html/draft-west-ua-client-hints a sketchy ID
> for the new headers.
> >
> https://tools.ietf.org/html/draft-west-lang-client-hint an equally
> sketchy ID for the new header.
> >

---

From: Mike West <[email-masked]>
Date: Fri, 30 Nov 2018 09:44:59 +0100

On Fri, Nov 30, 2018 at 1:30 AM Mark Nottingham <[email-masked]> wrote:

> Personally, I'd like to see these integrated into the current CH document,
> rather than as separate drafts. CH still needs some work, so it's not like
> we're going to get it out the door tomorrow.
>

These hints seem pretty clearly separable from the infrastructure upon
which they're built. I'd prefer to split them out into things-in-themselves
that we can point developers towards independently, giving ourselves the
opportunity to explain the rationale and background more coherently than I
think we'll be able to if we bury these in a subsection of the larger
document.

I'll defer to the group as to how y'all would like to handle these, but I'd
prefer several short and focused docs as a reader.

-mike

However, it seems like Ilya wants to go in a different direction, based
> upon the notes we received for Bangkok.
>
> Ilya, your thoughts?
>
>
>

---

From: Yoav Weiss <[email-masked]>
Date: Fri, 30 Nov 2018 11:10:24 +0100

On Fri, Nov 30, 2018 at 9:47 AM Mike West <[email-masked]> wrote:

>> Personally, I'd like to see these integrated into the current CH
>> document, rather than as separate drafts. CH still needs some work, so it's
>> not like we're going to get it out the door tomorrow.
>>
>
On my list, I want to remove the specific image-related features and move
them to their own specification, with a well defined browser processing
model.
Anything else that's needed to get CH infra "out the door tomorrow"? :)

> think we'll be able to if we bury these in a subsection of the larger

Similarly, I'd prefer clear distinctions between "CH as infrastructure" and
"Features that use the CH infrastructure".
We've had a lot of confusion and resistance to "CH the infrastructure" due
to some of the features that rely on it, and clearly separating the two
will enable implementations and user-agents to say "I support the CH
infrastructure, and certain features relying on it, but not feature X".

>From a procedural perspective, we wouldn't want every added feature to
delay "CH as infrastructure" to advance.

>
> I'll defer to the group as to how y'all would like to handle these, but
> I'd prefer several short and focused docs as a reader.
>
> -mike
>

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 30 Nov 2018 10:35:38 +0000

Hiya,

> likely that some user agents might make difference choices?

Not sending anything is better yes. Each UA deciding to
send something slightly different would be worse though,
so it may make sense to define it even if one recommends
it not be sent.

S.

---

From: Ilya Grigorik <[email-masked]>
Date: Fri, 30 Nov 2018 09:00:09 -0800

I agree with Yoav on the direction to separate the infrastructure from the
features.

We started by integrating the two within one spec, but overtime realized
that it's hard to cleanly and crisply define many of the concepts without
pulling in and having to replicate a whole lot of existing concepts and
plumbing from HTML, Fetch, and other specs. This problem also only gets
harder when we want to spec browser implementation. Hence the reason why we
started pulling out individual hints (e.g. Downlink, RTT, ECT) into NetInfo
spec, where those concepts and implementations are defined — bonus,
everything is in one place for consistency — and I think it makes sense to
do the same for DPR and remaining hints in current spec.

+Mike West <[email-masked]> I like your proposal for User-Agent and
Accept-Language. Most of the CH plumbing is already in place (or close to)
in HTML spec, WDYT of defining those hints directly in the HTML spec?

On Fri, Nov 30, 2018 at 2:10 AM Yoav Weiss <[email-masked]> wrote:

> Similarly, I'd prefer clear distinctions between "CH as infrastructure"
> and "Features that use the CH infrastructure".