Subject: #271: SHOULD review in p4

From: Mark Nottingham <[email-masked]>
Date: Fri, 22 Jun 2012 11:44:12 +1000

As per <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/271>, I'm reviewing our use of SHOULD in the documents; here I also pick on a few MAYs. Where I find issues, I've flagged with EDITORIAL or DESIGN as seems appropriate (I won't open issues for the design ones until we discuss; the editorial ones are considered attached to #271).

2.1

"However, if a resource has distinct representations that differ only in their metadata, such as might occur with content negotiation over media types that happen to share the same data format, then a server SHOULD incorporate additional information in the validator to distinguish those representations and avoid confusing cache behavior."

Seems OK, but not really testable.
EDITORIAL - change "a server" to "the origin server"

"A weak entity-tag SHOULD change whenever the origin server considers prior representations to be unacceptable as a substitute for the current representation. In other words, a weak entity-tag SHOULD change whenever the origin server wants caches to invalidate old responses."

First one is OK, but not testable.

EDITORIAL - change first one to "An origin server SHOULD change a weak entity-tag whenever it considers..."
EDITORIAL - the second one should probably change to an "ought to", otherwise we're repeating ourselves.

2.2.1

"Origin servers SHOULD send Last-Modified for any selected representation for which a last modification date can be reasonably and consistently determined, since its use in conditional requests and evaluating cache freshness ([Part6]) results in a substantial reduction of HTTP traffic on the Internet and can be a significant factor in improving service scalability and reliability."

As discussed, this is a social requirement, but justified.

"An origin server SHOULD obtain the Last-Modified value of the representation as close as possible to the time that it generates the Date field-value for its response. This allows a recipient to make an accurate assessment of the representation's modification time, especially if the representation changes near the time that the response is generated."

OK

2.3.1

"Origin servers SHOULD send ETag for any selected representation for which detection of changes can be reasonably and consistently determined, since the entity-tag's use in conditional requests and evaluating cache freshness ([Part6]) can result in a substantial reduction of HTTP network traffic and can be a significant factor in improving service scalability and reliability."

As discussed, this is a social requirement, so OK.

2.4

"""
HTTP/1.1 origin servers:

	• SHOULD send an entity-tag validator unless it is not feasible to generate one.
	• MAY send a weak entity-tag instead of a strong entity-tag, if performance considerations support the use of weak entity-tags, or if it is unfeasible to send a strong entity-tag.
	• SHOULD send a Last-Modified value if it is feasible to send one.
"""

Repeating requirements above, but a nice summary. OK.

"""
HTTP/1.1 clients:

	• MUST use that entity-tag in any cache-conditional request (using If-Match or If-None-Match) if an entity-tag has been provided by the origin server.
	• SHOULD use the Last-Modified value in non-subrange cache-conditional requests (using If-Modified-Since) if only a Last-Modified value has been provided by the origin server.
	• MAY use the Last-Modified value in subrange cache-conditional requests (using If-Unmodified-Since) if only a Last-Modified value has been provided by an HTTP/1.0 origin server. The user agent should provide a way to disable this, in case of difficulty.
	• SHOULD use both validators in cache-conditional requests if both an entity-tag and a Last-Modified value have been provided by the origin server. This allows both HTTP/1.0 and HTTP/1.1 caches to respond appropriately.
"""

OK

"HTTP/1.0 clients and caches might ignore entity-tags. Generally, last-modified values received or used by these systems will support transparent and efficient caching, and so HTTP/1.1 origin servers SHOULD provide Last-Modified values. In those rare cases where the use of a Last-Modified value as a validator by an HTTP/1.0 system could result in a serious problem, then HTTP/1.1 origin servers SHOULD NOT provide one."

EDITORIAL - change SHOULD to "still ought to". (we're repeating ourselves)
DESIGN - What are the "rare cases" being talked about here, and how are servers supposed to detect them?

3.1

"The "If-Match" header field MAY be used to make a request method conditional on the current existence or value of an entity-tag for one or more representations of the target resource."

EDITORIAL - change MAY to "can"

3.2

"The "If-None-Match" header field MAY be used to make a request method conditional on not matching any of the current entity-tag values for representations of the target resource."

EDITORIAL - change MAY to "can"

"If-None-Match MAY also be used with a value of "*" to prevent an unsafe request method (e.g., PUT) from inadvertently modifying an existing representation of the target resource when the client believes that the resource does not have a current representation."

EDITORIAL - change MAY to "can"

"Instead, if the request method was GET or HEAD, the server SHOULD respond with a 304 (Not Modified) status code, including the cache-related header fields (particularly ETag) of the selected representation that has a matching entity-tag."

OK

3.3

"The "If-Modified-Since" header field MAY be used to make a request method conditional by modification date: if the selected representation has not been modified since the time specified in this field, then do not perform the request method; instead, respond as detailed below."

EDITORIAL - change MAY to "can"

"If the selected representation has not been modified since a valid If-Modified-Since date, the server SHOULD return a 304 (Not Modified) response."

OK

3.4

"The "If-Unmodified-Since" header field MAY be used to make a request method conditional by modification date: if the selected representation has been modified since the time specified in this field, then the server must not perform the requested operation and must instead respond with the 412 (Precondition Failed) status code."

EDITORIAL - change MAY to "can"

"If the selected representation has not been modified since the time specified in this field, the server SHOULD perform the request method as if the If-Unmodified-Since header field were not present."

OK (but why not MUST here?)

"If the request normally (i.e., without the If-Unmodified-Since header field) would result in anything other than a 2xx or 412 status code, the If-Unmodified-Since header field SHOULD be ignored."

OK (but why not MUST here?)

EDITORIAL: "If the request.." -> "If a request..."
EDITORIAL: the language around ignoring headers / "were not present" here is needlessly divergent.

4.1

"Since the goal of a 304 response is to minimize information transfer when the recipient already has one or more cached representations, the response SHOULD NOT include representation metadata other than the above listed fields unless said metadata exists for the purpose of guiding cache updates (e.g., future HTTP extensions)."

Good.

--
Mark Nottingham   http://www.mnot.net/

---

From: Julian Reschke <[email-masked]>
Date: Sat, 23 Jun 2012 13:28:35 +0200

> EDITORIAL - change "a server" to "the origin server"

OK.

> EDITORIAL - the second one should probably change to an "ought to", otherwise we're repeating ourselves.

OK.

> ...

Ack (note this was a lowercase should).

> DESIGN - What are the "rare cases" being talked about here, and how are servers supposed to detect them?

Good question. We may want to drop this.

> EDITORIAL - change MAY to "can"

Absolutely.

> EDITORIAL - change MAY to "can"

Absolutely.

> EDITORIAL - change MAY to "can"

Absolutely.

> ...

Absolutely.

> ...

Absolutely.

> OK (but why not MUST here?)

Dunno, it's a lowercase "should" in RFC 2068. 
(<http://tools.ietf.org/html/rfc2068#section-14.28>)

> OK (but why not MUST here?)

Good question.

> EDITORIAL: the language around ignoring headers / "were not present" here is needlessly divergent.

I changed the second instance to:

    If a request normally (i.e., in absence of the If-Unmodified-Since
    header field) would result in anything other than a 2xx or 412 status
    code, the If-Unmodified-Since header field SHOULD be ignored.

> ...

Applied with <http://trac.tools.ietf.org/wg/httpbis/trac/changeset/1691> 
(except for the open issues).

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Sun, 24 Jun 2012 10:22:55 +1000

Now <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/363>.

---

From: John Sullivan <[email-masked]>
Date: Tue, 26 Jun 2012 16:22:59 +0100

>> Good question. We may want to drop this.

1) Algorithmically generated output where ETag can represent the
current control or seed values directly, but generating Last-Modified
would require extra persistent storage.

1b) Resources gatewayed from systems that have no concept of
last-modification time, but ETag along with the URL can again
represent parameters that are guaranteed to retrieve the same
entity or detect a change/error.

2) Rapidly changing, but cacheable, resources, where the origin knows
that Last-Modified could never become strong by S13.3.3. Normally
harmless but pointless. If it suspects that a downstream would
wrongly use the weak Last-Modified where it shouldn't, then it might
want to avoid that dangerous possibility altogether.

John
--

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 27 Jun 2012 09:22:00 +1000

> want to avoid that dangerous possibility altogether.

Those are reasons not to generate a LM, but they aren't specific to HTTP/1.0 -- agreed?

I can see dropping the last sentence and qualifying the remaining SHOULD with these sorts of reasons, but not retaining it.

Cheers,

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 27 Jun 2012 14:32:03 +1200

On 27.06.2012 11:22, Mark Nottingham wrote:
>>>>> "HTTP/1.0 clients and caches might ignore entity-tags. Generally, 
>>>>> last-modified values received or used by these systems will support 
>>>>> transparent and efficient caching, and so HTTP/1.1 origin servers 
>>>>> SHOULD provide Last-Modified values. In those rare cases where the 
>>>>> use of a Last-Modified value as a validator by an HTTP/1.0 system 
>>>>> could result in a serious problem, then HTTP/1.1 origin servers 
>>>>> SHOULD NOT provide one."
>>>>>
>> harmless but pointless.

This bit ...

>> If it suspects that a downstream would

... seems to be repeating the "rare cases" clause. Again with no 
supporting evidence for its existence, or identifiable way the origin 
might know about *future* downstream caching decisions.

To me the clause looks like dependence on the "evil bit" or equivalent 
being sent.

Although if the request was sent with a Date: which was noticeably out 
of sync with the servers own clock or some IMS timestamp in the future. 
Those could be a sign that the downstream is not able to depend on 
timestamps like LM properly. But I wouldn't bet on that either with the 
clockless algorithms.

Drawing at straws here it may be referring to the (hopefully rare) case 
of a server being required to add a Date: to a response with 
Last-Modified already in place? (ie CGI script created hedes being 
processed by an origin).
  In that case would it be better to add the Date and drop the 
Last-Modified? or simply not add the Date? (violating part 2 section 
10.2)

AYJ

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 27 Jun 2012 14:01:56 +1000

On 27/06/2012, at 12:32 PM, Amos Jeffries wrote:

> ... seems to be repeating the "rare cases" clause. Again with no supporting evidence for its existence, or identifiable way the origin might know about *future* downstream caching decisions.
> 
> 
> To me the clause looks like dependence on the "evil bit" or equivalent being sent.
> 
> 
> Although if the request was sent with a Date: which was noticeably out of sync with the servers own clock or some IMS timestamp in the future. Those could be a sign that the downstream is not able to depend on timestamps like LM properly. But I wouldn't bet on that either with the clockless algorithms.
> 
> 
> Drawing at straws here it may be referring to the (hopefully rare) case of a server being required to add a Date: to a response with Last-Modified already in place? (ie CGI script created hedes being processed by an origin).
> In that case would it be better to add the Date and drop the Last-Modified? or simply not add the Date? (violating part 2 section 10.2)

I think this is getting into error-handling territory (inside the same implementation!).

I think we should just remove the last sentence. We already contextualise it:

"""
Origin servers SHOULD send Last-Modified for any selected representation for which a last modification date can be reasonably and consistently determined...
"""

Cheers,

---

From: John Sullivan <[email-masked]>
Date: Wed, 27 Jun 2012 13:00:00 +0100

> Those are reasons not to generate a LM, but they aren't specific to HTTP/1.0 -- agreed?

Sure.

> I can see dropping the last sentence and qualifying the remaining SHOULD with these sorts of reasons, but not retaining it.

Absolutely - explain, not ignore or gloss over.

There's actually another bit which does much the same thing  that
I've got on my list of quibbles:

RFC 2616 S13.10

   In order to prevent denial of service attacks, an invalidation based
   on the URI in a Location or Content-Location header MUST only be
   performed if the host part is the same as in the Request-URI.

httpbis-p6-cache S2.6

   However, a cache MUST NOT invalidate a URI from a Location or
   Content-Location response header field if the host part of that URI
   differs from the host part in the effective request URI (Section 5.5
   of [Part1]).  This helps prevent denial of service attacks.

It's never fully explained how such a denial of service attack would
manifest, nor how serious it could be. The immediate possibility is
being able to effectively clear a cache causing more load on the
origin that usual, or more transit from the cache than usual, but I
have difficulty seeing how that is a serious concern:

This basically requires a malicious origin to work. If the cache is
inside the control of the origin, then presumably the origin isn't
malicious.

If the cache is outside the control of the origin, then if the target
is a different origin it presumably has to be able to deal with
a reasonable level of uncached requests hitting it anyway. If the
target is the cache itself then (without the vhost check) we can
clear at most 2 unrelated URLs per incoming client request. But a
client could clear those URLs anyway by sending max-age=0 and dropping
the connection after the first body byte. This will cause many
implementations to drop the previously cached response and cease
storing the new one too.

So I don't see how malicious use of Location/Content-Location is
much worse than other techniques, and it's arguably harder to pull
off than a purely client driven attack.

I'm not suggesting dropping the vhost check here, it seems like a
reasonable idea, but the justification seems overblown.

John
--