Subject: Re: Migrating some high-entropy HTTP headers to Client Hints.

From: Mark Nottingham <[email-masked]>
Date: Sun, 2 Dec 2018 14:48:27 +1100

Speaking personally --

I could see splitting out the various task-specific bits into separate documents (e.g., images).

However, it's not good to ship a framework like this without having it actually working for a real-world use case, so even if we split out all of the various CHs into separate docs, I think we'd need to hold the "main" document until at least one of those is ready.

Just a thought -- replacing User-Agent is probably suitable for that test case (in addition to the image-focused stuff, or perhaps instead of it). If we do decide to do that, it might be suitable to put it in the core document, since that seems like it's pretty central to what's going on here (the current claims in the document about not replacing UA notwithstanding).

Chair hat on -- what I did notice was that when the update for CH was read in Bangkok, *many* WG participants expressed surprise at the direction you were taking it in; most people seemed to think that this document was almost done in its current form, and there was concern that forming WG consensus on that was being disregarded. So whatever you do here, please make sure you get buy-in on the list, and make sure you coordinate with the chairs. Continuing this discussion and moving towards a common idea of what the doc(s) should include, when we should ship them, etc. sounds like a good start.

Cheers,

> On 30 Nov 2018, at 9:10 pm, Yoav Weiss <[email-masked]> wrote:
> 
> 
> 
> On Fri, Nov 30, 2018 at 9:47 AM Mike West <[email-masked]> wrote:
> On Fri, Nov 30, 2018 at 1:30 AM Mark Nottingham <[email-masked]> wrote:
> I, for one, welcome our new Client Hint overlords.
> 
> Personally, I'd like to see these integrated into the current CH document, rather than as separate drafts. CH still needs some work, so it's not like we're going to get it out the door tomorrow.
> 
> On my list, I want to remove the specific image-related features and move them to their own specification, with a well defined browser processing model.
> Anything else that's needed to get CH infra "out the door tomorrow"? :)
>  
> 
> These hints seem pretty clearly separable from the infrastructure upon which they're built. I'd prefer to split them out into things-in-themselves that we can point developers towards independently, giving ourselves the opportunity to explain the rationale and background more coherently than I think we'll be able to if we bury these in a subsection of the larger document.
> 
> Similarly, I'd prefer clear distinctions between "CH as infrastructure" and "Features that use the CH infrastructure".
> We've had a lot of confusion and resistance to "CH the infrastructure" due to some of the features that rely on it, and clearly separating the two will enable implementations and user-agents to say "I support the CH infrastructure, and certain features relying on it, but not feature X".
> 
> From a procedural perspective, we wouldn't want every added feature to delay "CH as infrastructure" to advance.
>   
> 
> I'll defer to the group as to how y'all would like to handle these, but I'd prefer several short and focused docs as a reader.
> 
> -mike
> 
> However, it seems like Ilya wants to go in a different direction, based upon the notes we received for Bangkok.
> 
> Ilya, your thoughts?
> 
> 
> 
> > On 29 Nov 2018, at 9:22 pm, Mike West <[email-masked]> wrote:
> > 
> > Hey folks,
> > 
> > Section 9.7 of RFC7231 rightly notes that some of the content negotiation headers user agents deliver in HTTP requests create substantial fingerprinting surface. I think it would be beneficial if we took steps to reduce their prevalence on the wire, and Client Hints looks like a reasonable infrastructure on top of which to build.
> > 
> > `User-Agent` and `Accept-Language` seem like particularly tasty and low-hanging fruit, and I've sketched out two proposals as proofs of concept:
> > 
> > *   `User-Agent` could be represented as ~four distinct hints: `UA`, `Model`, `Platform`, and `Arch`: https://github.com/mikewest/ua-client-hints is a high-level explainer, and https://tools.ietf.org/html/draft-west-ua-client-hints a sketchy ID for the new headers.
> > 
> > *   `Accept-Language` could be represented as a `Lang` hint: https://github.com/mikewest/lang-client-hint is a high-level explainer, https://tools.ietf.org/html/draft-west-lang-client-hint an equally sketchy ID for the new header.
> > 
> > I'd appreciate y'all's feedback. Thanks!
> > 
> > -mike
> 
> --
> Mark Nottingham   https://www.mnot.net/
> 

--
Mark Nottingham   https://www.mnot.net/

---

From: Mike West <[email-masked]>
Date: Mon, 3 Dec 2018 10:48:49 +0100

+Anne van Kesteren <[email-masked]> for the Fetch discussion. Hi, Anne!

On Fri, Nov 30, 2018 at 6:00 PM Ilya Grigorik <[email-masked]> wrote:

> +Mike West <[email-masked]> I like your proposal for User-Agent and
> Accept-Language. Most of the CH plumbing is already in place (or close to)
> in HTML spec, WDYT of defining those hints directly in the HTML spec?
>

For `User-Agent`: HTML currently relies on Fetch's "default `User-Agent`
value <https://fetch.spec.whatwg.org/#default-user-agent-value>" (for `
navigator.userAgent <https://html.spec.whatwg.org/#dom-navigator-useragent>`),
and Fetch is also responsible for setting the `User-Agent` value (in step
5.11 of https://fetch.spec.whatwg.org/#http-network-or-cache-fetch). That
seems like the right direction for the dependency, so I could imagine
defining the entire mechanism in Fetch if this group is willing to point
off in that direction in a future version of RFC7231 that deprecates
`User-Agent`.

For `Accept-Language`, HTML defines `navigator.language{.s}
<https://html.spec.whatwg.org/#language-preferences>` with some
recommendations about what data to expose, and Fetch sets the header in
step 1.4 of https://fetch.spec.whatwg.org/#fetching. Again, it seems
reasonable for the header work to be exposed in Fetch rather than pulling
it into HTML.

If we agree that these hints are interesting enough to build and ship, then
I'd be perfectly happy defining them in Fetch route, or doing something
like the drafts I sketched out above. The former approach would require
this group to point elsewhere when deprecating `Accept-Language` and
`User-Agent` in a future iteration of RFC7231, the latter requires some
discussion about which HTTPbis document they'd live in (see Mark's
suggestion to merge into the client hints draft itself). I have a personal
preference for small, focused, and separate documents (and I think Fetch is
already growing some features I'd have preferred to split out), but I also
don't have much context on the client hints discussion here thus far. I'm
happy to defer to y'all.

-mike

---

From: Daniel Stenberg <[email-masked]>
Date: Mon, 3 Dec 2018 11:05:06 +0100 (CET)

On Mon, 3 Dec 2018, Mike West wrote:

> the entire mechanism in Fetch if this group is willing to point off in that 
> direction in a future version of RFC7231 that deprecates `User-Agent`.

I would be most curious on *how* we would go ahead and actually deprecate 
User-Agent for real on the web. Just saying it in a document like that won't 
be enough.

I'm sure that removing that header in requests will break the experience on N% 
of the world's web sites, which I doubt browsers would like to impose on their 
users. I expect that nobody will remove this header until such an action is 
likely to only cause an insignificant amount of user pain, and the same time 
the world's web sites with all their user-agent sniffing logic are unlikely to 
actually swich to CH as long as the vast majority of the browsers keep sending 
User-Agent headers... The good old depcrecating things on the web dilemma.

Or are there reasons to believe this time will be different?

-- 

  / daniel.haxx.se

---

From: Mike West <[email-masked]>
Date: Mon, 3 Dec 2018 11:21:00 +0100

> User-Agent for real on the web. Just saying it in a document like that

https://github.com/mikewest/ua-client-hints#a-proposal tries to lay out a
story: in short, we take another stab at Safari's attempt to freeze the UA
string, with the client hint serving as an escape valve for developers who
have legitimate need for the existing string.

-mike

---

From: Martin Thomson <[email-masked]>
Date: Wed, 5 Dec 2018 04:12:18 +1100

On Fri, Nov 30, 2018 at 7:38 PM Mike West <[email-masked]> wrote:
> That seems pretty reasonable to me, though I'm not sure how much space we have to work with in the static table. Chrome's current user agent string comes in at around ~150 characters, depending on platform. Taking that as a baseline, how many strings could we reasonably encode? Can the table expand to an extend that would allow it to cover enough of the UA market?

The cost is in static code size only, so in theory it could be
modestly large.  (Those concerned with static code size can replace
the strings with symbols representing the semantics without preserving
the syntax - those 150+ characters could be just be a static
IS_CHROME)

---

From: Ilya Grigorik <[email-masked]>
Date: Wed, 5 Dec 2018 06:08:07 -0800

Hey folks.

On Sat, Dec 1, 2018 at 7:48 PM Mark Nottingham <[email-masked]> wrote:

> However, it's not good to ship a framework like this without having it
> actually working for a real-world use case, so even if we split out all of
> the various CHs into separate docs, I think we'd need to hold the "main"
> document until at least one of those is ready.
>
> Just a thought -- replacing User-Agent is probably suitable for that test
> case (in addition to the image-focused stuff, or perhaps instead of it). If
> we do decide to do that, it might be suitable to put it in the core
> document, since that seems like it's pretty central to what's going on here
> (the current claims in the document about not replacing UA notwithstanding).
>

Just to clarify, we do have a shipping implementation of image hints
(Viewport-Width, Width, DPR), as well as network related hints (ECT, RTT,
Downlink). The latter set is defined in NetInfo [1] spec and the proposal
is to spec processing definitions for image hints within Fetch+HTML to
resolve current UA implementation and processing gaps — this would resolve
and prevent questions similar to [2], [3]. On that note, the basic
integration for image hints is already in Fetch (see steps 6+7 and related
plumbing in [4]) but instead of linking to the IETF draft for definitions,
the goal is to define those concepts directly against relevant HTML+Fetch
concepts.

Which is to say, this is editorial shuffling and it doesn't change or
affect current shipping implementation of either sets of hints. I'm excited
by Mike's proposal but I don't think we need to block the framework on that
set of hints: we already have two existing shipping sets active in the wild.

[1] https://wicg.github.io/netinfo/
[2] https://github.com/httpwg/http-extensions/issues/698
[3] https://github.com/httpwg/http-extensions/issues/697
[4] https://fetch.spec.whatwg.org/#fetching

> Chair hat on -- what I did notice was that when the update for CH was read
> in Bangkok, *many* WG participants expressed surprise at the direction you
> were taking it in; most people seemed to think that this document was
> almost done in its current form, and there was concern that forming WG
> consensus on that was being disregarded. So whatever you do here, please
> make sure you get buy-in on the list, and make sure you coordinate with the
> chairs. Continuing this discussion and moving towards a common idea of what
> the doc(s) should include, when we should ship them, etc. sounds like a
> good start.
>

Yup, fair feedback and will do. The change in course is based on recent
discussions and issues that were raised as we were trying to shepherd the
document through last stages.

On Mon, Dec 3, 2018 at 1:49 AM Mike West <[email-masked]> wrote:

> navigator.userAgent
> <https://html.spec.whatwg.org/#dom-navigator-useragent>`), and Fetch is
> also responsible for setting the `User-Agent` value (in step 5.11 of
> https://fetch.spec.whatwg.org/#http-network-or-cache-fetch). That seems
> like the right direction for the dependency, so I could imagine defining

+Mike West <[email-masked]> yep, defining these in Fetch makes sense to
me. I'll defer to the group on the deprecation.

ig