Subject: Variants and Client Hints

From: Jeffrey Yasskin <[email-masked]>
Date: Fri, 1 Jun 2018 16:21:15 -0700

Hey Mark,

I'm working on a matching algorithm for bundled exchanges (
https://github.com/WICG/webpackage/issues/201), and I realized that a CDN's
use of variants will need a similar algorithm.
https://httpwg.org/http-extensions/draft-ietf-httpbis-variants.html#cache
is fairly clear about how to pick a "best" Variant-Key for the Accept-*
headers, but the introduction also mentions Client Hints, where matching is
less about string equality.

For example, if the client sends Viewport-Width: 350, and the server has
images that are ideal for Viewport-Widths of 320 or 400, which one gets
sent? I can imagine a couple options:

   - Only certain kinds of clients are supported efficiently. 🤢
   - The available-values are single numbers, and the spec says that a
   request for 350 matches the next smaller number.
   - The syntax of available-values defines ranges in some way, maybe using
   a syntax like CSS Media Queries or inventing a new syntax.

What do folks think this should look like?

Thanks,
Jeffrey

On Wed, May 2, 2018 at 12:14 AM <[email-masked]> wrote:

>
> A New Internet-Draft is available from the on-line Internet-Drafts
> directories.
> This draft is a work item of the Hypertext Transfer Protocol WG of the
> IETF.
>
>         Title           : HTTP Representation Variants
>         Author          : Mark Nottingham
>         Filename        : draft-ietf-httpbis-variants-01.txt
>         Pages           : 19
>         Date            : 2018-05-02
>
> Abstract:
>    This specification introduces an alternative way to communicate a
>    secondary cache key for a HTTP resource, using the HTTP "Variants"
>    and "Variant-Key" response header fields.  Its aim is to make HTTP
>    proactive content negotiation more cache-friendly.
>
>
> The IETF datatracker status page for this draft is:
> https://datatracker.ietf.org/doc/draft-ietf-httpbis-variants/
>
> There are also htmlized versions available at:
> https://tools.ietf.org/html/draft-ietf-httpbis-variants-01
> https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-variants-01
>
> A diff from the previous version is available at:
> https://www.ietf.org/rfcdiff?url2=draft-ietf-httpbis-variants-01
>
>
> Please note that it may take a couple of minutes from the time of
> submission
> until the htmlized version and diff are available at tools.ietf.org.
>
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
>
>
>

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 4 Jun 2018 14:31:32 +0200

Hi Jeffrey,

> On 2 Jun 2018, at 1:21 am, Jeffrey Yasskin <[email-masked]> wrote:
> 
> Hey Mark,
> 
> I'm working on a matching algorithm for bundled exchanges (https://github.com/WICG/webpackage/issues/201), and I realized that a CDN's use of variants will need a similar algorithm. https://httpwg.org/http-extensions/draft-ietf-httpbis-variants.html#cache is fairly clear about how to pick a "best" Variant-Key for the Accept-* headers, but the introduction also mentions Client Hints, where matching is less about string equality.
> 
> For example, if the client sends Viewport-Width: 350, and the server has images that are ideal for Viewport-Widths of 320 or 400, which one gets sent? I can imagine a couple options:
> 	• Only certain kinds of clients are supported efficiently. 🤢
> 	• The available-values are single numbers, and the spec says that a request for 350 matches the next smaller number.

This is kind of what I anticipated, but maybe Ilya has some thoughts?

Cheers,

> 	• The syntax of available-values defines ranges in some way, maybe using a syntax like CSS Media Queries or inventing a new syntax.
> A New Internet-Draft is available from the on-line Internet-Drafts directories.
> This draft is a work item of the Hypertext Transfer Protocol WG of the IETF.
> 
> Please note that it may take a couple of minutes from the time of submission

--
Mark Nottingham   https://www.mnot.net/

---

From: Ilya Grigorik <[email-masked]>
Date: Wed, 6 Jun 2018 11:14:38 -0700

+Yoav Weiss <[email-masked]>

On Mon, Jun 4, 2018 at 5:31 AM Mark Nottingham <[email-masked]> wrote:

> https://github.com/WICG/webpackage/issues/201), and I realized that a
> CDN's use of variants will need a similar algorithm.

Can you elaborate on why this would be the case and what your definition of
"efficiently" is?

> >       • The available-values are single numbers, and the spec says that
> a request for 350 matches the next smaller number.

I think this is something we should defer to the implementers. Some sites
have very stringent requirements about image quality and would not accept
upscaling (e.g. think product images), whereas others might be open to it
and might explicitly choose this route when save-data hint is present.

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 8 Jun 2018 09:31:21 +0200

On 6 Jun 2018, at 8:14 pm, Ilya Grigorik <[email-masked]> wrote:
> I think this is something we should defer to the implementers. Some sites have very stringent requirements about image quality and would not accept upscaling (e.g. think product images), whereas others might be open to it and might explicitly choose this route when save-data hint is present.

In Variants, the definition of a particular content negotiation mechanism is responsible for defining the algorithm for selecting a result (or results); see <https://httpwg.org/http-extensions/draft-ietf-httpbis-variants.html#define>. Defering to the implementers would mean that each cache would behave differently, so no interop and content folks would be (justifiably) grumpy about this.

---

From: Yoav Weiss <[email-masked]>
Date: Fri, 8 Jun 2018 10:55:12 +0200

On Fri, Jun 8, 2018 at 9:31 AM Mark Nottingham <[email-masked]> wrote:

> > >       • The available-values are single numbers, and the spec says
> that a request for 350 matches the next smaller number.
> > I think this is something we should defer to the implementers. Some
> sites have very stringent requirements about image quality and would not
> accept upscaling (e.g. think product images), whereas others might be open
> to it and might explicitly choose this route when save-data hint is present.
>
> In Variants, the definition of a particular content negotiation mechanism
> is responsible for defining the algorithm for selecting a result (or
> results); see <
> https://httpwg.org/http-extensions/draft-ietf-httpbis-variants.html#define>.
> Defering to the implementers would mean that each cache would behave
> differently, so no interop and content folks would be (justifiably) grumpy
> about this.
>
>
If we go back to Jeffrey's example, there are a few possible answers that
are all valid:
* Pick the smaller image (so 320 in the example) - that could be a
preference for servers which prefer data saving at the expense of image
quality.
* Pick the larger image (so 400) - a legitimate preference for a service
which prefers quality
* None (and go back to origin) - a service that wants to serve pixel
perfect images, which only the origin (or upper layer cache) can produce

Since there are several legitimate choices here, it doesn't make sense to
arbitrarily codify one of them in the RFC. It also doesn't make sense to
delegate that decision to an intermediate cache which may not have direct
relationship with the content. (managed by That Other Team™, shared between
different parts of the site with different requirements, etc)
Maybe we need a signal header (`Variants-Choice`?) that can inform the
algorithm of the content's preference?

To make matters even more complex, multiple client hints preferences can
and should interact with those calculations and influence them (DPR, Width,
save-data & ECT all come to mind).
The server can consider them more variants of stored responses, but there
may be (many) duplicates in terms of content (e.g. viewport 350 + save data
can return the same response as viewport 300 without save data. Same goes
for DPR and all the rest)

Would it be possible to somehow define all the client hints headers as a
single content negotiation mechanism that takes all those values into
account?

P.S. The spec says that con-neg mechanisms should start with `Accept-`, but
CH doesn't conform to that (and it's probably a bit late to change at this
point...)

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 8 Jun 2018 11:12:50 +0200

Hi Yoav,

On 8 Jun 2018, at 10:55 am, Yoav Weiss <[email-masked]> wrote:
> 
> If we go back to Jeffrey's example, there are a few possible answers that are all valid:
> * Pick the smaller image (so 320 in the example) - that could be a preference for servers which prefer data saving at the expense of image quality.
> * Pick the larger image (so 400) - a legitimate preference for a service which prefers quality
> * None (and go back to origin) - a service that wants to serve pixel perfect images, which only the origin (or upper layer cache) can produce
> 
> Since there are several legitimate choices here, it doesn't make sense to arbitrarily codify one of them in the RFC.

For this specific case, I'm going to push back here. Keeping in mind that the whole point here of CH is to support multiple image quality "buckets" and find the closest match in the available buckets, is it really the case that such fine-grained control is necessary?

For example, let's say that we say the rule here is "next smallest." A server that prefers quality can take that into account and up the quality of their images, relative to the sizes they make available. If they want to go back to the origin, they won't use Variants; they'll fall back to Vary.

I don't think adding more complexity to the protocol is justified here; people can meet their use cases (at least as far as you outline above) with just one algorithm.

> It also doesn't make sense to delegate that decision to an intermediate cache which may not have direct relationship with the content. (managed by That Other Team™, shared between different parts of the site with different requirements, etc)

Of course.

> Maybe we need a signal header (`Variants-Choice`?) that can inform the algorithm of the content's preference?

My thinking to date has been that extra information of this sort could be encoded in the available-value if necessary -- but it'd be nice to have a worked example. Note that the syntax of available-value opened up in the latest draft; if more characters would help, we can figure that out.

> 
> To make matters even more complex, multiple client hints preferences can and should interact with those calculations and influence them (DPR, Width, save-data & ECT all come to mind). 
> The server can consider them more variants of stored responses, but there may be (many) duplicates in terms of content (e.g. viewport 350 + save data can return the same response as viewport 300 without save data. Same goes for DPR and all the rest)
> 
> Would it be possible to somehow define all the client hints headers as a single content negotiation mechanism that takes all those values into account?

This feels a lot like this discussion that came up in Key:
  https://github.com/httpwg/http-extensions/issues/104

I.e., high fidelity to the server's intent comes at the cost of considerable complexity.

That said, the syntax of Variant-Key now allows a response to match multiple keys, which at least helps in the duplication issue; see the last part of:
  https://httpwg.org/http-extensions/draft-ietf-httpbis-variants.html#variant-key

> P.S. The spec says that con-neg mechanisms should start with `Accept-`, but CH doesn't conform to that (and it's probably a bit late to change at this point...)

Yep. That's why it's a SHOULD.

---

From: Yoav Weiss <[email-masked]>
Date: Fri, 8 Jun 2018 11:33:24 +0200

On Fri, Jun 8, 2018 at 11:12 AM Mark Nottingham <[email-masked]> wrote:

> > Since there are several legitimate choices here, it doesn't make sense
> to arbitrarily codify one of them in the RFC.
>
> For this specific case, I'm going to push back here. Keeping in mind that
> the whole point here of CH is to support multiple image quality "buckets"
> and find the closest match in the available buckets, is it really the case
> that such fine-grained control is necessary?
>
> For example, let's say that we say the rule here is "next smallest." A
> server that prefers quality can take that into account and up the quality
> of their images, relative to the sizes they make available. If they want to
> go back to the origin, they won't use Variants; they'll fall back to Vary.
>

High quality but upscaled images will end up upsetting everyone (as they'd
end up blurry and bloated).

The inverse (i.e. a "next largest" algorithm) might be a better choice, but
I suspect it won't satisfy some people's use cases.

> I don't think adding more complexity to the protocol is justified here;
> people can meet their use cases (at least as far as you outline above) with
> just one algorithm.
>

I'm not sure I see how. Could you outline an example?

>
> > It also doesn't make sense to delegate that decision to an intermediate
> cache which may not have direct relationship with the content. (managed by
> That Other Team™, shared between different parts of the site with different
> requirements, etc)
>
> Of course.
>
> My thinking to date has been that extra information of this sort could be
> encoded in the available-value if necessary -- but it'd be nice to have a
> worked example. Note that the syntax of available-value opened up in the
> latest draft; if more characters would help, we can figure that out.
>

Are you suggesting that origins will communicate in the available-value
e.g. the ranges of viewports that an image can satisfy? That may work if we
ignore the potential cross-conneg-mechanism interactions.

> > The server can consider them more variants of stored responses, but
> there may be (many) duplicates in terms of content (e.g. viewport 350 +
> save data can return the same response as viewport 300 without save data.
> Same goes for DPR and all the rest)
> >

I agree this keeps coming up, but doesn't mean that there's no need for it
:)

>
> That said, the syntax of Variant-Key now allows a response to match
> multiple keys, which at least helps in the duplication issue; see the last
> part of:
>
I agree that's better, but still, the response for `DPR: 2, Viewport: 300`,
`DPR:1, Viewport: 600` and `DPR: 2, Viewport: 600, Save-Data: on` could be
one and the same. Avoiding that cache duplication can be an important
factor.

> > P.S. The spec says that con-neg mechanisms should start with `Accept-`,
> but CH doesn't conform to that (and it's probably a bit late to change at
> this point...)
>

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 13 Jun 2018 15:27:44 +1000

> On 8 Jun 2018, at 11:33 am, Yoav Weiss <[email-masked]> wrote:
> 
> High quality but upscaled images will end up upsetting everyone (as they'd end up blurry and bloated).
> 
> The inverse (i.e. a "next largest" algorithm) might be a better choice, but I suspect it won't satisfy some people's use cases.

Hm. A little more?

> I'm not sure I see how. Could you outline an example?

See below.

> Are you suggesting that origins will communicate in the available-value e.g. the ranges of viewports that an image can satisfy? That may work if we ignore the potential cross-conneg-mechanism interactions. 

That's one possibility, At the end of the day, available-value is just a string under control of the spec, and the algorithm can do pretty much anything as long as it produces appropriate outputs (although if it relies on inputs not listed, it's going to raise the bar for implementation). The tricky bit is likely to be making it succinct. 

> I agree that's better, but still, the response for `DPR: 2, Viewport: 300`, `DPR:1, Viewport: 600` and `DPR: 2, Viewport: 600, Save-Data: on` could be one and the same. Avoiding that cache duplication can be an important factor.

Right, but that response would have on it something like:

Variants: DPR;1;2, Viewport;600;1200, Save-Data;on;off
Variant-Key: 1;2, 600, on;off

If that's not expressive enough, maybe we should be talking about a Variant-Key syntax that lets you specify alternative sets of values, e.g.,

Variants: DPR;1;2, Viewport;300;600, Save-Data;on;off
Variant-Key: 2;300;off, 1;600;off, 2;600;on