Subject: New precondition fields and caching

From: "Martin Thomson" <[email-masked]>
Date: Mon, 07 Sep 2020 15:43:43 +1000

I'm trying to understand whether HTTP can be extended with new preconditions.  I've so many questions.

If-None-Digest can be used for validation, so it currently defines that 304 (Not Modified) applies when a GET or HEAD request fails the precondition.  However, that seems fraught in ways I didn't originally anticipate.

Thinking more generally, the 412 (Precondition Failed) status code can be marked explicitly cacheable, but for this to work the server would need to indicate that all responses Vary based on the preconditions that might fail.  That includes not just the 412 status code, but other status codes.  That requirement to include the new precondition in Vary extends to 2xx responses, which could then vary based on the conditional request header field.  

For example, a request contains a precondition `On: Wednesday` (a bad example perhaps) which ensures that preconditions fail 6 of 7 days of the week.  For this request, both 2xx responses (on Wednesdays) and 412 responses require `Vary: On`.  Without Vary, an intermediary that receives a new precondition `On: Wednesday` could use a cached response and generate a 200 response based on its ignorance of this being a precondition even when the precondition would fail at the origin server.

If the same applied to If-None-Match, a whole lot would break.

Assuming that interpretation is valid, how is this reconciled with existing uses of validation?  If-None-Match and If-Modified-Since specifically, which never in practice get recorded in Vary (at least not as far as I can tell).  Is it that we effectively bless these fields and require caches to understand them?

Separately:

The guidance for new fields suggests that definitions should consider where those fields are applicable.  The current semantics draft just describes each of the existing preconditions as "request header fields", while avoiding being definitional.  There is no text that says addresses the question of where they appear directly, even generic text.  That doesn't seem especially explicit, but is that enough?

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 7 Sep 2020 16:25:41 +1000

> On 7 Sep 2020, at 3:43 pm, Martin Thomson <[email-masked]> wrote:
> 

Yes. You effectively need to introduce a new protocol version to introduce a new precondition, if you want to make its handling mandatory. Designing them so that ignoring the precondition doesn't break anything is the only workaround. They are evaluated completely separately from conneg.

> The guidance for new fields suggests that definitions should consider where those fields are applicable.  The current semantics draft just describes each of the existing preconditions as "request h…

Can you link to where you're talking about?

--
Mark Nottingham   https://www.mnot.net/

---

From: "Martin Thomson" <[email-masked]>
Date: Mon, 07 Sep 2020 16:31:36 +1000

On Mon, Sep 7, 2020, at 16:25, Mark Nottingham wrote:
> Yes. You effectively need to introduce a new protocol version to 
> introduce a new precondition, if you want to make its handling 
> mandatory. Designing them so that ignoring the precondition doesn't 
> break anything is the only workaround. They are evaluated completely 
> separately from conneg.

Shouldn't the specification say that explicitly?

https://httpwg.org/http-core/draft-ietf-httpbis-semantics-latest.html#preconditions says:

> A conditional request is an HTTP request with one or more request header fields that indicate a precondition to be tested before applying the request method to the target resource. 

It says "request header fields" here and in the definition of each, but it doesn't really say that these fields are only valid when in request headers anywhere that I can see.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Sep 2020 07:02:21 +0000

--------
Martin Thomson writes:

> Assuming that interpretation is valid, how is this reconciled with existing
> uses of validation?  If-None-Match and If-Modified-Since specifically, which
> never in practice get recorded in Vary (at least not as far as I can tell).
> Is it that we effectively bless these fields and require caches to understand 
> them?

I think the answer is yes, but with the footnote that Cache-Control
should also be set to mitigate downright disasters, if an intermediary
does not grok If-*

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: "Martin Thomson" <[email-masked]>
Date: Mon, 07 Sep 2020 17:14:27 +1000

> should also be set to mitigate downright disasters, if an intermediary

Are you talking about using Cache-Control for damage mitigation or a genuine backstop?  It seems like you get to choose between bustage and no caching at all if you take that path.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 07 Sep 2020 07:36:58 +0000

--------
Martin Thomson writes:
> Are you talking about using Cache-Control for damage mitigation or a genuine 
> backstop?  It seems like you get to choose between bustage and no caching at 
> all if you take that path. 

Where I have seen people have thought about this, it is clearly
damage mitigation.

---

From: Julian Reschke <[email-masked]>
Date: Mon, 7 Sep 2020 10:44:28 +0200

Am 07.09.2020 um 07:43 schrieb Martin Thomson:
> ...

What spec defines this one?

 > ...

Best regards, Julian

---

From: "Martin Thomson" <[email-masked]>
Date: Tue, 08 Sep 2020 09:10:18 +1000

> What spec defines this one?

It was a proposal I made: https://martinthomson.github.io/if-digest/draft-thomson-http-if-digest.html

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 8 Sep 2020 15:12:20 +1000

> On 7 Sep 2020, at 4:31 pm, Martin Thomson <[email-masked]> wrote:
> 
> 
> 

https://github.com/httpwg/http-core/issues/448

> It says "request header fields" here and in the definition of each, but it doesn't really say that these fields are only valid when in request headers anywhere that I can see.

I think we use 'request header fields', 'response header fields' and other permutations pretty consistently. Would it address your concern if we made <https://httpwg.org/http-core/draft-ietf-httpbis-semantics-latest.html#header.and.trailer.fields> a bit more explicit (i.e., included 'request' and 'response' as limiting modifiers)?

Cheers,

---

From: "Martin Thomson" <[email-masked]>
Date: Tue, 08 Sep 2020 19:31:27 +1000

> https://github.com/httpwg/http-core/issues/448

Thanks. 

> I think we use 'request header fields', 'response header fields' and 
> other permutations pretty consistently. Would it address your concern 
> if we made 
> <https://httpwg.org/http-core/draft-ietf-httpbis-semantics-latest.html#header.and.trailer.fields> a bit more explicit (i.e., included 'request' and 'response' as limiting modifiers)?

That might help. I think that my concern isn't that serious when I look again though. The definition is definitely implicit, in that you say these are header fields in a section called request fields.

You already have "We refer to some named fields specifically as a "header field" when they are only allowed to be sent in the header section."  That is quite clear there, but the use of that is less so with the phrasing chosen. So whatever you think works best.