Subject: Call for Adoption: Cache Digests for HTTP/2

From: Mark Nottingham <[email-masked]>
Date: Wed, 22 Jun 2016 16:52:55 +1000

<https://tools.ietf.org/html/draft-kazuho-h2-cache-digest-01>

This draft has been discussed both out in the community and at our meetings, and there seems to be a decent amount of interest in it. So, this is a Call for Adoption to add it to our set of working documents.

Please state whether you support adoption, and ideally why. Expressions of interest in implementation would also be very helpful.

We'll wait at least one week before making a decision.

N.B. Because I'm co-author on the draft, Natasha Rooney has kindly agreed to judge consensus on the CfA, and act as Document Shepherd if we do adopt it.

Cheers,

--
Mark Nottingham   https://www.mnot.net/

---

From: Martin Thomson <[email-masked]>
Date: Wed, 22 Jun 2016 08:26:21 +0100

I think that we should adopt this.

But yikes, it got complicated while I wasn't looking.  FOUR flags,
none of which are very well justified in the text.  The implications
of those aren't very well explored either; does this mean that there
are multiple Bloom filters?  How do these flags interact?  Simple
example: what do I do with the contents of a frame that has the RESET
flag?

That needs some significant work.  I'm sure that there are reasonable
reasons for each of these, but given that I was concerned that it was
too complicated to implement in the original form, this really steps
it up a notch.

---

From: Cory Benfield <[email-masked]>
Date: Wed, 22 Jun 2016 08:26:59 +0100

> On 22 Jun 2016, at 07:52, Mark Nottingham <[email-masked]> wrote:
> 

I support adoption of this draft. I will also commit to building a partial implementation: that is, I will add support for handling and emitting the CACHE_DIGEST frame to my Python libraries, but I will not commit to updating any Python server or client to respecting or emitting CACHE_DIGEST by default.

The Cache Digest draft strikes me as an interesting way to resolve a very real tension in HTTP/2. HTTP/2 has a serious focus on reducing the number of network round-trips that are needed for a client to obtain all the resources, but it provides no obvious way for a client to inform a server or intermediary that certain resources are not required. RST_STREAM, as this draft points out, is usable in this role, but certainly not ideal.

For me, then, Cache Digest seems like a useful extension to HTTP/2. It’s certainly not mandatory, and its existence doesn’t hint at an oversight in RFC 7540, but it’s the perfect example of what a HTTP/2 extension can do: provide better information sharing between the client and the server to improve their efficiency.

Cory

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 22 Jun 2016 17:44:26 +1000

> On 22 Jun 2016, at 5:26 PM, Martin Thomson <[email-masked]> wrote:
> 

You clear any state you have for the origin and then process the digest in the frame. If that isn't clear (and it *should* be when reading "Client Behavior"), it's easy enough to fix.

Regardless, the flags are all about the semantics of the digest itself; 
 - If STALE is false, it's fresh in cache, and no push is necessary for matching URLs
 - if STALE is true, they're stale, and matching URLs need to get something pushed
 - if VALIDATORS is true, it indicates that ETags were used to compute the hashes, so the server can decide to push 304 or 200 for a STALE hit
 - COMPLETE lets the client indicate to the server that what it's sent to date represents everything in cache for the origin, so it knows it can make decisions about pushes confidently
 - RESET allows state management.

There are a bunch of different ways to factor those, and personally I'm happy to talk through the different designs. Part of the reason for the flexibility is that it's not yet clear what patterns clients will want to send digests in, and what will be most useful to servers (in terms of fresh, stale, complete or not, etc.). I'm hoping the WG discussion will draw that out.

> too complicated to implement in the original form, this really steps

What made you think it was too complicated originally?

One thing that I'd like to see feedback from client (browser and otherwise) vendors is whether the overhead of generating a digest at the beginning of each connection (and it has to be then, if you want the digest to be complete and to cover both fresh and stale cache entries) is a concern.

---

From: Alcides Viamontes E <[email-masked]>
Date: Wed, 22 Jun 2016 10:28:13 +0200

Hello,

I'm sorry to have to intervene now, since as part of a publicly financed
project my company is trying to build forecasts on how cache digests would
work in practice. We are extrapolating data from a few scanned websites.
The goal is to get likely bounds for the sizes of cache digests.

We also have some data from using a cookie implementation of cache digests
in ShimmerCat that follows this draft closely. We are satisfied with how it
works but we have identified a few things that can be improved.

We expressed some of our concerns in January, and we are glad to see that
some of them have been addressed, i.e. the inclusion of validators.

A further concern was scoping: as it stands now, the only scoping mechanism
for digests is to use different origins.

This is bad for several reasons. AFAIK, sites don't have either a way to
ask the browser to prematurely evict an expired representation that the
browser would otherwise consider fresh. These two things together could
allow a cache digest to grow indefinitely. Wouldn't that have a degrading
effect on performance?

We turn on cache digests during development, and we have seen the cache
digests grow due to the accumulation of different versions of assets.
Because we are using a cookie implementation, we can have the server reset
the cookie when it gets too big. Would that be possible  with the current
draft?

Also related to scoping is the following. Cache digests have been devised
so far for a single HTTP/2 Push use scenario: pushing a few assets which
are critical to the first render of a page. For that scenario, it is a good
idea to keep the digests short. In other words, to not include all assets
from the origin that the browser has in cache. Scoping is good here! But
there are other HTTP/2 Push scenarios. PUSH_PROMISEs can be sent later on
the lifetime of a connection. Since HTTP/2 recommends to not bundle assets,
HTTP/2 Push is also handy to push hierarchies of small Javascript modules.
Digests are also useful here, but with different assets than the digests at
the beginning of a site fetch.

Another concern we expressed on January was that digests as an HTTP/2 frame
are something that suits well large and well-equipped infrastructure
operators but that becomes almost invisible and hard to debug for web
developers and sysadmins with less resources. However we can swipe that
concern under the rug by hoping that browser tooling will catch up
eventually.

Another concern we expressed on January was that there was no way to switch
off the digest frame. All in all, we think that the current draft is a
really good step, and we understand that it doesn't need to be perfect. But
if it is not going to be good enough for all scenarios, it would be nice if
it can be switched off. Since digests are only used from the second visit
to a site, the browser could just remember some hint from the site on
previous visits and abstain from using digests.

For what it counts, and in short, here is our response to the call for
adoption:

- Should be this draft adopted in its current form?:    No.

- What would be the minimum requirement to revert that response?: A way to
switch off digests, so that operators opting for different implementations
don't need to pay for it.

On Wed, Jun 22, 2016 at 8:52 AM, Mark Nottingham <[email-masked]> wrote:

> This draft has been discussed both out in the community and at our
> meetings, and there seems to be a decent amount of interest in it. So, this
> is a Call for Adoption to add it to our set of working documents.
>
> Please state whether you support adoption, and ideally why. Expressions of
> interest in implementation would also be very helpful.
>
> N.B. Because I'm co-author on the draft, Natasha Rooney has kindly agreed
> to judge consensus on the CfA, and act as Document Shepherd if we do adopt
> it.
>
> Cheers,
>
>
> --

---

From: Cory Benfield <[email-masked]>
Date: Wed, 22 Jun 2016 11:43:40 +0100

> On 22 Jun 2016, at 09:28, Alcides Viamontes E <[email-masked]> wrote:
> 
> This is bad for several reasons. AFAIK, sites don't have either a way to ask the browser to prematurely evict an expired representation that the browser would otherwise consider fresh. These two things together could allow a cache digest to grow indefinitely. Wouldn't that have a degrading effect on performance?

This is presumably true of caching to begin with, correct? If the browser doesn’t consider the cached representation stale it is welcome to not emit a request for it at all, and simply to serve it from its cache. This means that the cache digest can only grow as large as the client cache allows it to grow, which I should certainly hope is not indefinitely large!

However, it may be sensible to consider providing a SETTINGS field that allows servers to flag a maximum size on a cache digest that it is willing to accept.

> Also related to scoping is the following. Cache digests have been devised so far for a single HTTP/2 Push use scenario: pushing a few assets which are critical to the first render of a page. For that scenario, it is a good idea to keep the digests short. In other words, to not include all assets from the origin that the browser has in cache. Scoping is good here! But there are other HTTP/2 Push scenarios. PUSH_PROMISEs can be sent later on the lifetime of a connection. Since HTTP/2 recommends to not bundle assets, HTTP/2 Push is also handy to push hierarchies of small Javascript modules. Digests are also useful here, but with different assets than the digests at the beginning of a site fetch.

Can you elaborate on this concern, please? For browsers, at least, the only meaningful use of PUSH_PROMISE at this time is to perform cache priming, and the draft allows sending multiple CACHE_DIGEST frames on the connection, which helps keep the server up to date on cache state.

> Another concern we expressed on January was that there was no way to switch off the digest frame. All in all, we think that the current draft is a really good step, and we understand that it doesn't need to be perfect. But if it is not going to be good enough for all scenarios, it would be nice if it can be switched off. Since digests are only used from the second visit to a site, the browser could just remember some hint from the site on previous visits and abstain from using digests.
> 
> For what it counts, and in short, here is our response to the call for adoption:
> 
> - What would be the minimum requirement to revert that response?: A way to switch off digests, so that operators opting for different implementations don't need to pay for it.

Operators using other implementations shouldn’t have to pay for it. RFC 7540 § 5.5 says:

> Implementations MUST discard frames that have unknown or unsupported types.

This means that if a CACHE_DIGEST frame is sent, well-written server implementations will simply ignore that frame. Additionally, servers that do support CACHE_DIGEST will want to be back-compatible with clients that do not emit it for any reason, and so will presumably continue to function as they do now in the absence of CACHE_DIGEST.

That means that “switch it off” amounts to two different things: either don’t emit them as a client (servers can’t make you), or ignore them as a server (client’s can’t think it’s a PROTOCOL_ERROR if a CACHE_DIGEST frame is ignored). So I don’t believe this problem exists in the draft as provided: any implementation that fails in the face of a CACHE_DIGEST frame is violating RFC 7540 by doing so.

Cory

---

From: Alcides Viamontes E <[email-masked]>
Date: Wed, 22 Jun 2016 13:39:39 +0200

Hello Cory,

It's good to have your feedback. Below are answers to your comments, but I
do expect to use this conversation to fill my gaps.

On Wed, Jun 22, 2016 at 12:43 PM, Cory Benfield <[email-masked]> wrote:

> This is presumably true of caching to begin with, correct? If the browser
> doesn’t consider the cached representation stale it is welcome to not emit
> a request for it at all, and simply to serve it from its cache. This means
> that the cache digest can only grow as large as the client cache allows it
> to grow, which I should certainly hope is not indefinitely large!
>

Yes, this is related to caching in general. And it is the reason people
have to add query strings for doing cache busting. This problem is a
separate issue, but it interacts with cache digests in that old version of
assets are kept in the cache and  therefore in the cache digest and the
origin have no way of removing it. The origin can only create a new URL
(say, via a new query string) that gets added to the cache and the cache
digest.

>
> However, it may be sensible to consider providing a SETTINGS field that
> allows servers to flag a maximum size on a cache digest that it is willing
> to accept.
>

But this leaves the server without any control about which things are made
part of the cache digest. That's why we think scoping and an explicit
eviction mechanism are better long term solutions.

>
> > Also related to scoping is the following. Cache digests have been
> devised so far for a single HTTP/2 Push use scenario: pushing a few assets
> which are critical to the first render of a page. For that scenario, it is
> a good idea to keep the digests short. In other words, to not include all
> assets from the origin that the browser has in cache. Scoping is good here!
> But there are other HTTP/2 Push scenarios. PUSH_PROMISEs can be sent later
> on the lifetime of a connection. Since HTTP/2 recommends to not bundle
> assets, HTTP/2 Push is also handy to push hierarchies of small Javascript
> modules. Digests are also useful here, but with different assets than the
> digests at the beginning of a site fetch.
>
> Can you elaborate on this concern, please? For browsers, at least, the
> only meaningful use of PUSH_PROMISE at this time is to perform cache
> priming, and the draft allows sending multiple CACHE_DIGEST frames on the
> connection, which helps keep the server up to date on cache state.
>

Yes, I'm talking about cache priming. Just at different times: early during
the fech you really want to prime the cache for critical assets, like CSS
and webfonts. Priming the cache with a hierarchy of small Javascript
modules can be done much later. But here latency is also an issue and
HTTP/2 Push with cache digests is also useful. The standard, HTTP/1.1
recommendation is to take all those little Javascript modules and making a
huge .js file out of them. We wanted to deprecate that with HTTP/2.

Also think about the effectivity of cache digests for a Wordpress user
hosting a huge number of cached images in the same origin where the theme
.css is hosted.

Ideally, we would like the browser to say first "hey, this is the cache
digest for the CSS and other things that I need like really right now... it
is a really short digest" and later, when the page starts loading
javascript modules or other resources, the browser would say "hey, I
already have a lot of this  stuff, here is a fat digest with all of them".

If I understand correctly, the point of having cache digests is for the
browser to advertise/broadcast which assets it has in cache *as early as
possible*, so to avoid unneeded HTTP/2 asset push by the server. So if you
have a lot of assets in your domain which are needed at a later time, you
will receive digest bytes for all of them also very early. The size of the
digest is proportional to how many assets from this domain the browser has
in cache, not to how many assets the browser needs right now, nor even how
many assets the browser needs for rendering this page.

Scoping can be achieved with the current draft by having different origins.
It means that some people will need to get back to domain sharding for
static assets. But even then, the possibility of disabling digests would be
priceless.

> > Another concern we expressed on January was that there was no way to
> switch off the digest frame. All in all, we think that the current draft is
> a really good step, and we understand that it doesn't need to be perfect.
> But if it is not going to be good enough for all scenarios, it would be
> nice if it can be switched off. Since digests are only used from the second
> visit to a site, the browser could just remember some hint from the site on
> > - What would be the minimum requirement to revert that response?: A way
> to switch off digests, so that operators opting for different
> implementations don't need to pay for it.
>

Sorry for not being clear enough, I'm talking about the "bytes cost". As
the draft stands now, a browser implementing cache digests will transmit a
number of bytes we have no control of with the cache digest, therefore
taking bandwidth resources. A website operator may very well be of the
opinion that those bytes are better used for something else, e.g. a poor
man, slimmer cache digest bottled either in a cookie or in a custom
put-by-service-worker header.

./Alcides.

---

From: Kazuho Oku <[email-masked]>
Date: Thu, 23 Jun 2016 06:29:47 +0900

Hello,

Thank you for your insights. I am eager to see your experiment results
once it gets ready.

2016-06-22 20:39 GMT+09:00 Alcides Viamontes E <[email-masked]>:
>> doesn’t consider the cached representation stale it is welcome to not emit a
>> request for it at all, and simply to serve it from its cache. This means
> Yes, this is related to caching in general. And it is the reason people have
> to add query strings for doing cache busting. This problem is a separate
> issue, but it interacts with cache digests in that old version of assets are
> kept in the cache and  therefore in the cache digest and the origin have no
> way of removing it. The origin can only create a new URL (say, via a new
> query string) that gets added to the cache and the cache digest.

I share your concern (though I might disagree on how critical the issue is).

Actually, the draft provides a way to update fresh responses _if_ the
client and server agree.

In the current version of the draft, we have intentionally defined
VALIDATOR and STALE as separate flags, to allow a client to send a
digest of fresh resources with their validators taken into account. A
server can use such digest (i.e. a digest with VALIDATOR flag set and
STALE flag not set) for pushing an updated version of the resource.

So if the client is to accept push of a fresh response to replace an
already-existing fresh resource (the HTTP/2 spec. does not specify if
it should or not), the mechanism can be used, and in such case, cache
busting would no longer be necessary.

> eviction mechanism are better long term solutions.

We could extend the current draft if scoping would be an issue, and it
would be possible to do so with keeping backward compatibility. For
example, we could add a new flag that indicates that the scope of the
digest is limited to certain mime-type and that the mime-type is
stored in the frame in addition to the digest-value.

That said, does your concern about the size of the digest dissolve
without adding a method for a server to notify the client the maximum
permitted size of the digest (e.g. a SETTINGS field as suggested by
Cory)?

>> > which are critical to the first render of a page. For that scenario, it is a
>> > good idea to keep the digests short. In other words, to not include all
> and webfonts. Priming the cache with a hierarchy of small Javascript modules
> can be done much later. But here latency is also an issue and HTTP/2 Push
> with cache digests is also useful. The standard, HTTP/1.1 recommendation is
> to take all those little Javascript modules and making a huge .js file out
> of them. We wanted to deprecate that with HTTP/2.
>
> is a really short digest" and later, when the page starts loading javascript
> modules or other resources, the browser would say "hey, I already have a lot
> of this  stuff, here is a fat digest with all of them".
>
>> > But if it is not going to be good enough for all scenarios, it would be nice
>> > if it can be switched off. Since digests are only used from the second visit
> Sorry for not being clear enough, I'm talking about the "bytes cost". As the
> draft stands now, a browser implementing cache digests will transmit a

-- 
Kazuho Oku

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 23 Jun 2016 09:32:11 +1000

Hi,

> On 22 Jun 2016, at 6:28 PM, Alcides Viamontes E <[email-masked]> wrote:
> 

[...]

> A further concern was scoping: as it stands now, the only scoping mechanism for digests is to use different origins. 
> 

It depends on how digests are used. The current idea (reflected in various parts of the draft) is that a client will typically send a cache digest at the beginning of a connection, right after the first request (e.g., for /index.html), but not much afterwards; it's up to the server to track things that it's already sent on the current connection (and indeed some are already doing that). 

If the client caches other responses for the origin during the lifetime of the connection (e.g., on another connection), it can inform the server of this by sending another digest, which does not replace the current one; it's queried alongside any previous digests send on the same connection.

Also, since cache digests are scoped to the lifetime of the connection they're sent within, they can't grow indefinitely, in practical terms. And, of course, servers can discard digests if they don't want to keep the state.

Finally, clients aren't required to represent the complete state of their cache in a digest; they're allowed to choose what representations to include, so (for example) a very large cache can use a heuristic to include only the most likely candidates in a digest, to limit its size.

> We turn on cache digests during development, and we have seen the cache digests grow due to the accumulation of different versions of assets. Because we are using a cookie implementation, we can have the server reset the cookie when it gets too big. Would that be possible  with the current draft?

Yes; see the RESET flag.

> Also related to scoping is the following. Cache digests have been devised so far for a single HTTP/2 Push use scenario: pushing a few assets which are critical to the first render of a page. For th…

Yes. As mentioned above, the set of digests on the server can be added to non-destructively.

One of the reasons we have a fair amount of flexibility in how the format works -- and thus so many flags (Hi, Martin), is that it's difficult to predict all of the different ways they'll be used, and we want to assure that they can address a variety of scenarios. With some implementation experience and wider deployment, we can refine what's there, of course.

> Another concern we expressed on January was that digests as an HTTP/2 frame are something that suits well large and well-equipped infrastructure operators but that becomes almost invisible and hard to debug for web developers and sysadmins with less resources. However we can swipe that concern under the rug by hoping that browser tooling will catch up eventually. 
> 

Hmm. We'd effectively need a site-wide, persistent hint to indicate that future visits should include digests. That actually dovetails pretty well with some other ongoing conversations (this kind of requirement comes up often); let me try to write something down.

> - What would be the minimum requirement to revert that response?: A way to switch off digests, so that operators opting for different implementations don't need to pay for it. 

<chair hat on>

OK. For what it's worth, a CfA is asking whether the WG should start work in an area; it's not saying that the exact document is going to turn into the standard. Most drafts we adopt undergo substantial changes in the WG process, sometimes to the point where the original document is unrecognisable. 

As such, the CfA is really looking for a few things: 1) is there enough interest in doing it, and are there potential implementers?  2) Are there any red-flag issues (e.g., pushback from the Security area) that would make starting the work counter-productive?  3) Is the draft mature enough to make WG discussion productive?

Cheers,

---

From: Stefan Eissing <[email-masked]>
Date: Thu, 23 Jun 2016 10:44:04 +0200

FTR: Apache httpd has an implementation of version 0 of the draft, using if from Headers and also to keep internal state of the connection, done by me.

Unsurprisingly, I support adoption of the draft. I had only a glance over of the current changes, there are some interesting additions. Ultimately, feedback from client/intermediaries implementors are necessary here. And I think this WG is a good place to make that happen.

-Stefan

> Am 22.06.2016 um 09:26 schrieb Cory Benfield <[email-masked]>:
> 
>

---

From: Alcides Viamontes E <[email-masked]>
Date: Thu, 23 Jun 2016 17:33:43 +0200

Hello Kazuho,

Thanks! Comments below.

On Wed, Jun 22, 2016 at 11:29 PM, Kazuho Oku <[email-masked]> wrote:

> have

That's really good news! What can be done to make this mechanism less a
matter of browser goodwill?

> permitted size of the digest (e.g. a SETTINGS field as suggested by

After some reflection, I must agree that limiting the size of the digest is
probably a good compromise until we know if scopes are really needed. This
setting could even be used to make the digests opt-in, which is a plus.

./Alcides.

---

From: Alcides Viamontes E <[email-masked]>
Date: Thu, 23 Jun 2016 17:33:10 +0200

Hi,

On Thu, Jun 23, 2016 at 1:32 AM, Mark Nottingham <[email-masked]> wrote:

> > A further concern was scoping: as it stands now, the only scoping
> mechanism for digests is to use different origins.
> >
> It depends on how digests are used. The current idea (reflected in various
> parts of the draft) is that a client will typically send a cache digest at
> the beginning of a connection, right after the first request (e.g., for
> /index.html), but not much afterwards; it's up to the server to track
> things that it's already sent on the current connection (and indeed some
> are already doing that).
>
> If the client caches other responses for the origin during the lifetime of
> the connection (e.g., on another connection), it can inform the server of
> this by sending another digest, which does not replace the current one;
> it's queried alongside any previous digests send on the same connection.
>
> Also, since cache digests are scoped to the lifetime of the connection
> they're sent within, they can't grow indefinitely, in practical terms. And,
> of course, servers can discard digests if they don't want to keep the state.

> Finally, clients aren't required to represent the complete state of their
> cache in a digest; they're allowed to choose what representations to
> include, so (for example) a very large cache can use a heuristic to include
> only the most likely candidates in a digest, to limit its size.
>
>
Ok, sounds sensible.

> One of the reasons we have a fair amount of flexibility in how the format
> works -- and thus so many flags (Hi, Martin), is that it's difficult to
> predict all of the different ways they'll be used, and we want to assure
> that they can address a variety of scenarios. With some implementation
> experience and wider deployment, we can refine what's there, of course.
>
>
> > Another concern we expressed on January was that digests as an HTTP/2
> frame are something that suits well large and well-equipped infrastructure
> Hmm. We'd effectively need a site-wide, persistent hint to indicate that
> future visits should include digests. That actually dovetails pretty well
> with some other ongoing conversations (this kind of requirement comes up
> often); let me try to write something down.
>
>
Thanks!

> OK. For what it's worth, a CfA is asking whether the WG should start work
> in an area; it's not saying that the exact document is going to turn into
> the standard. Most drafts we adopt undergo substantial changes in the WG
> process, sometimes to the point where the original document is
> unrecognisable.
>
> As such, the CfA is really looking for a few things: 1) is there enough
> interest in doing it, and are there potential implementers?  2) Are there
> any red-flag issues (e.g., pushback from the Security area) that would make
> starting the work counter-productive?  3) Is the draft mature enough to
> make WG discussion productive?
>

My bad! Then definitely I caused some confusion. After your clarifications,
I'm not qualified to answer 2 and 3. But for:

1) Yes, my organization is very interested on this, and we will continue
updating our implementation.

Thanks Mark!

./Alcides.

---

From: Natasha Rooney <[email-masked]>
Date: Thu, 23 Jun 2016 17:23:43 +0000

Hi all,

The draft currently has 4 supporters (6 including the authors) and 1 reference to experiment (not sure if this counts as implementation). Please let the list know if you have any strong opinions for / for not adopting the draft. Also let the list know if you have interest in implementing the draft.

Thanks!

Natasha

Natasha Rooney | Technologist, Web and Internet, W3C & IETF | GSMA | [email-masked]<mailto:[email-masked]> | +44 (0) 7730 219 765 | @thisNatasha | Skype: [email-masked]<mailto:[email-masked]>

On Jun 23, 2016, at 4:33 PM, Alcides Viamontes E <[email-masked]<mailto:[email-masked]>> wrote:

Hello Kazuho,

Thanks! Comments below.

On Wed, Jun 22, 2016 at 11:29 PM, Kazuho Oku <[email-masked]<mailto:[email-masked]>> wrote:
Hello,

2016-06-22 20:39 GMT+09:00 Alcides Viamontes E <[email-masked]<mailto:[email-masked]>>:
> On Wed, Jun 22, 2016 at 12:43 PM, Cory Benfield <[email-masked]<mailto:[email-masked]>> wrote:
>>
>>
>> > On 22 Jun 2016, at 09:28, Alcides Viamontes E <[email-masked]<mailto:[email-masked]>>
>> > wrote:
>> >

That's really good news! What can be done to make this mechanism less a matter of browser goodwill?

After some reflection, I must agree that limiting the size of the digest is probably a good compromise until we know if scopes are really needed. This setting could even be used to make the digests opt-in, which is a plus.

./Alcides.

This email and its attachments are intended for the above named only and may be confidential. If they have come to you in error you must take no action based on them, nor must you copy or show them to anyone; please reply to this email or call +44 207 356 0600 and highlight the error.

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 24 Jun 2016 16:19:16 +1200

> We'll wait at least one week before making a decision.

Since discussino has already promoted my fellow coworkers to break the
silence I think it safe to say that the Squid team supports this draft.

While I dont expect to see us developing an implementation for a while
yet. It is great progress on a needed function that we are likely to
make use of eventually.

Amos Jeffries
The Squid Softare Foundation

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Fri, 24 Jun 2016 22:10:35 +0900

> to judge consensus on the CfA, and act as Document Shepherd if we do adopt
​I support this draft, and will implement it in nghttp2.
I've not followed the recent development into the latest draft, but I know
cache digest from its early stage, and like its spirit.  It would make
HTTP/2 push really powerful feature.

For implementation, nghttp2 has an extension framework to add third-party
non-critical frames.  We can use it to implement the draft before its final
release, to offer the decent test bed for interoperability.

Best regards,
Tatsuhiro Tsujikawa​

---

From: Raphaël D <[email-masked]>
Date: Sun, 26 Jun 2016 18:37:14 -0300

On Wed, 22 Jun 2016 13:39:39 +0200 Alcides Viamontes wrote:

> Yes, this is related to caching in general. And it is the reason
> people have to add query strings for doing cache busting. This problem is a
> separate issue, but it interacts with cache digests in that old version
> of assets are kept in the cache and  therefore in the cache digest and the

(email subject renamed to avoid polluting the "Cache Digests for HTTP/2"
 mailing-list thread)

Hi,

I read this incidentally meanwhile trying to understand what alternative
to cache-busting exists.
I'm stretching my head to find which overlooked IETF-http or W3C concept matches.

Nowadays most CMS, javascript/css-frameworks feature cache-busting
(usually using query-string) which is considered as the unavoidable
answer to the "can't forcefully refresh browser cache" issue (and web
traffic pattern).

Among the facts/reasons given are:

1) web-applications support the fact that some webserver set far-future
   Expires times for assets (css, js, fonts)

2) downstream proxies and browser cache are not accessible by the webserver

3) but web application needs to force a HTML page to use freshest version of
   some or all assets even if they were already cached in-browser with a
   far-future Expires time

4) people don't use Last-Modified or ETags because zero request
   always seems better than requesting and waiting for a 304 response.

5) static files are usually not routed through CGI but left to webserver
   configuration rather than web-application itself which is related to
   assumptions like the above n°1

"Is it a mistake to Expires +1 year /js/jquery.js"? seems one of the
underlying meta-questions (like the definitions of "persistence" and "version")

In the event assets are not be cached that long (says only 1 week) and all
(reverse)proxies are correctly configured, the issue still arise under
some circumstance.
For example in case a webapp upgrades assets, it leaves inconsistencies
between the main resource fetched by the user-agent (the HTML page) and
some of its "dependencies". Since old, cached assets that can't be easily
invalidated, new resources are created in caches.
(query-string versioning appears a lot like a N.I.H. ETags mechanism
the difference being that one HTTP response (main webpage) sends the
ETag of the multiple sub-resources it depends upon)

* Should cache-busting be part of HTTP best practices, what references and
  knowledgeable voices have said/to say about it?

* Is there HTTP 1.x or 2.0 alternatives? Is there a need for it ? or
  should the solution be uniquely in the hands of assets
  deployment/distribution/versioning tools and/or web application tweaks
  like query-string suffixes?

* Does the situation implies new precautions when using HTTP Expires header?

thank you

Ref:
¹ https://tools.ietf.org/html/rfc7234
² https://css-tricks.com/strategies-for-cache-busting-css/
³ https://www.shimmercat.com/en/info/articles/caching/#url-query-parameters-are-still-needed-to-update-assets-at-clients-
⁴ https://www.mnot.net/cache_docs/#FAQ ("My images expire a month from now, but I need to change them in the caches now!")
⁵ https://core.trac.wordpress.org/ticket/29201#comment:12

---

From: Alcides Viamontes E <[email-masked]>
Date: Mon, 27 Jun 2016 07:21:56 +0200

Hello Raphaël,

Thanks for taking interest on this. I can only talk on my name, and I do so
from the perspective of both being a web developer and an HTTP/2 edge
server developer.

As a web developer, I can say that cache busting is difficult. In most web
applications out there, the responsibility of creating URLs for static
assets is distributed: front-end stacks create some, and backend
application frameworks like Django also do. Getting coherent URLs for cache
busting across code that runs in different places (browser and server) and
quite often in different programming languages tends to be a draining and
time-consuming effort. It also defeats the purpose of HTTP caching tools
(cache-control, etag, date, etc.) that recognize that representations
change. Finally, cache busting via URLs leaves the old version of the asset
in the cache instead of replacing it immediately, and that means that HTTP
caches have less space available and are therefore less effective of what
they could be.

On the other hand, round-trips and latency are a real issue, and cache
busting through URLs are a practical and available solution.

So I will risk an answer to two of your questions, but note that I don't
consider myself knowledgeable in the topic, and it would be really good if
we can get more feedback from other people here:

* No, cache-busting through URLs should not be part of HTTP best practices.
But it is now. I think that an explicit HTTP mechanism for cache busting
would be highly desirable.

* As far as I know, there is no alternative.

On Sun, Jun 26, 2016 at 11:37 PM, Raphaël D <[email-masked]> wrote:

> ³
> https://www.shimmercat.com/en/info/articles/caching/#url-query-parameters-are-still-needed-to-update-assets-at-clients-
> ⁴ https://www.mnot.net/cache_docs/#FAQ ("My images expire a month from
> now, but I need to change them in the caches now!")

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 27 Jun 2016 14:29:05 +0900

> matter of browser goodwill?

Considering the fact that H2 push is optional (and Cache Digests for
HTTP/2 is an option to H2 push), I am afraid it might not become the
standard way to update a fresh response.

If we are to solve cache busting, it might be better to consider
adding a validator to the link-rel-preload header. For example, we
could add a etag attribute, that presents the etag value that is
_expected_ to be used together with the response that contained the
link header. Upon observing such header, clients can check their
cache, and in case the validators do not match, remove the response
from cache and issue a new request.

In other words,

  link: </style.css>; rel=preload; etag=deadbeef

would instruct the web browser to fetch /style.css if it is not
fresh-cached, or if the etag value of the cached response does not
match the value supplied by the header.

The pros of this approach would be that it could be used over both
HTTP/1 and HTTP/2, and that it does not require the browser to
implement H2 push. The cons would be that it would only be possible to
invalidate responses as part of the preload process.

> setting could even be used to make the digests opt-in, which is a plus.

Thank you for the clarification.

-- 
Kazuho Oku

---

From: Alcides Viamontes E <[email-masked]>
Date: Mon, 27 Jun 2016 09:05:02 +0200

> match the value supplied by the header.

This is a good mechanism! It may require a bit of common sense to avoid
race conditions between PUSH_PROMISEs and STREAM_RESETs but that's not too
much to ask. And as you said, it can even be used with HTTP/1.1

I have opened a ticket in the preload draft and added your text above:
https://github.com/w3c/preload/issues/68 ...

---

From: Alcides Viamontes E <[email-masked]>
Date: Mon, 27 Jun 2016 10:21:55 +0200

I just came gain over these emails and there are a few things which I don't
completely understand.

> of course, servers can discard digests if they don't want to keep the state.

Isn't the cache supposed to contain assets obtained during previous visits?
In that case, wouldn't the cache digest contain assets that the browser
obtained in a previous connection?

> the cookie when it gets too big. Would that be possible  with the current

I went to the draft to understand better the RESET flag. What is the
typical scenario that this flag is intended for? I can see that RESET helps
if something happens to the cache; then the browser can use the flag to
tell the server. But is the browser supposed to clear the cache often in
the middle of a connection? If not, how is this substantially better than
just resetting the connection? .. Both Chrome and Firefox reset connections
when the user clears the cache.

Also, if the server idea of assets held by the browser is bigger than the
assets actually held by the browser, the assets won't be pushed, but the
browser can just request them "normally".

What I was alluding before was close to having the RESET in the other
direction, from server to browser, to ask the browser to clear its cache
for the current origin. This could be good for a number of use cases, not
least of them decreasing the chance of false positives in queries to the
digest. But adding a mechanism for the server to clear the browser's cache
may be a bit out of scope :-( ...

---

From: Amos Jeffries <[email-masked]>
Date: Tue, 28 Jun 2016 00:41:47 +1200

>    Expires times for assets (css, js, fonts)

The existing Best Practice is for redesigns to change the names of their
objects slightly *iff* the object has changed enough to warrant it.
Enough to do cache busting without query-string, but also to retain
backward compatibility with older versions of the application requesting
old assets.

IMO this whole nest of problems listed below is derived solely from
developers not following _that_ best practice. Adding yet another Best
Practice requirement for them to learn, remember and follow is not going
to help the situation.

> 2) downstream proxies and browser cache are not accessible by the webserver

For developers following the #1 best practice this is a net positive.
Those caches aid the backwards compatibility of the service.

>    some or all assets even if they were already cached in-browser with a

For developers following the Best Practice of #1, this does not occur
therefore not a problem. Also assets which have not changed can utilize
teh existing cached data.

>    always seems better than requesting and waiting for a 304 response.

What is this "zero request" you speak of?
 Cache busting is about forcing a full response result. Which is by
definition slower than the revalidation. And also much more detrimental
to other wanted uses of the caching.

As a cache implementer I am seeing a growing trend for use of
anti-busting and de-duplication features being used in the intermediary
web caches. Taken incautiously there is a train wreck ahead for all
service developers, even those services doing the Best Practce from #1.

"Seems" being the operative word of your description. Developer
ignorance is not a good reason for requiring whole new mechanisms which
they would have to learn instead of the existing ones which they still
have not learned to use properly after 15+ years of those mechanisms
existence.

>    assumptions like the above n°1

When using the Best Practice for #1, this is not a problem. Assets can
be located anywhere so long at the referring response referrs to the
correct one for its need. That is a true fact regardless of cache busting.

The use of cache-busting simply means the referring resource needs logic
to track what the latest cache-buster value is. Which is no different
than tracking that its needed resource still exists at the referene URI.

> underlying meta-questions (like the definitions of "persistence" and "version")

No it is not. Expires can be any date at all. The protocol allows for up
to 68 years ahead!

[Putting my web develoepr hat on ...]

Use of the generic "jquery.js" name is suitable only for resource
scripts which use the stable jQuery API features. The ones which are not
going to change within that 1 year Expires period regardless of which
jQuery version sits at that URI, nor whether the URL is fetched 364 days
from now.

The mistake the developer has made is that they are in fact *not* using
stable jQuery API features. So what they need to be referencing is the
versioned jquery-X.Y.Z.js name. The one matching the latest API
version(s) their script is known to work with. The version number itself
does the "cache busting" part cleanly without the query-string being
involved.

[ back to the cache maintainer hat ...]

This mistake, naivety, ignorance is not something we can readily fix at
the protocol or transport level. Though much respect to those who
persist in trying to fix these social problems by technical means.

> (reverse)proxies are correctly configured, the issue still arise under

With those anti- cache-busting features becoming ever more popular these
cases will become more and more problematic. Not just for those using
cache-busting to avoid doing a long standing Best Practice. But also for
everyone mistakenly caught in the de-duplication efforts.

> ETag of the multiple sub-resources it depends upon)

Remember the reason you gave for people *not* using ETags like they
should. Loss of reliability is the direct cost of choosing to throw away
reliability for a "it seems to be faster".

In the same way jumping off a cliff is faster to reach the bottom than
attaching a rope and walking down it.

(A rough analogy there, but far closer than it seems at first.)

>   knowledgeable voices have said/to say about it?

IMO. No.

>   like query-string suffixes?

Correct use of URIs and either renaming asserts when they change, or
correct use of the ETag / Last-Modified equivalent if no rename is done.

Note that uses of query-string to store a deterministic representation
of ETag or Last-Modified header in the URI is not really "cache busting"
as such. Those URI only change when the object does, so are more like
enforcing revalidation when a bad actor in the delivery path does not do
revalidate properly.
 This does run the risk of hitting some middleware admins naive
de-duplication efforts though.

> * Does the situation implies new precautions when using HTTP Expires header?

Not new ones IMHO. The same precautions exist both with and without
cache busting.

If those precautions are correctly taken, then cache-busting becomes
less and less needed up to the point of being useless when all actors in
the environment are behaving themselves.

Sadly we may not quite be there today, but the last 5 or so years have
seen a significant improvement around the web. I am hopeful that we are
collectively getting close to the point of being able to simply tell the
remaining bad actors to do it right instead of catering to the whinging.

Amos

---

From: Amos Jeffries <[email-masked]>
Date: Tue, 28 Jun 2016 01:20:04 +1200

> obtained in a previous connection?

If viewed that way, yes. However the digest itself is not the objects it
lists. It is supposed to be regenerated from the cache state at the
beginning of the newer connection. So objects which have been dropped
out of the cache since that older connection are not even considered for
the new digest.

> when the user clears the cache.

Resetting the whole connection is a huge overhead. Frames with digest
should not really be frequent enough to warrant a whole connnection
close and re-open cycle just to optimize away a single bit per frame.

Having the frame cleared when that huge overhead is being paid anyway
makes sense. But having the dependency the reverse way around would be a
major pain.

> may be a bit out of scope :-( ...

I susect you are mistaking what "cache" is referring to. There are two
caches involved with these drafts.
* a cache of URL + objects. Stored on the browser.
* a cache of digest values. Stored on the server.

The RESET flag on these frame is about sender controlling the cache of
digest values on the recipient. So it makes no sense to send it from the
agent which holds the cache, to the agent which does not.

It can be used to indicate to the server that the browser objects cache
just got wiped for some unspecified and otherwise irrelevant reason.

Amos

---

From: Alcides Viamontes E <[email-masked]>
Date: Mon, 27 Jun 2016 17:02:38 +0200

> backward compatibility with older versions of the application requesting

I do think that having to change the names of modified assets is a bit sad.
Wouldn't it be nice if etag validators were enough? On the other hand, if a
bug needs to be fixed in say "login.js", one would certainly want to
propagate those changes instantaneously. As it is now, I can not have my
pie and eat it :-(

---

From: Matthew Kerwin <[email-masked]>
Date: Tue, 28 Jun 2016 09:39:40 +1000

> propagate those changes instantaneously. As it is now, I can not have my

If you want to avoid round-trips, version the URI
("login-20160628.js") and send Expires. If you want to keep the URI
("index.html"), send Last-Modified and ETag.

I don't think there can ever be a magic bullet (in 1.x, without server
push) to resolve this issue, and it's been known for long enough that
best practices and user expectations should be pretty well established
by now.

Cheers
-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/