Subject: Re: ETags and concurrency control

From: "Mark Baker" <[email-masked]>
Date: Thu, 1 May 2008 23:30:53 -0400

On Mon, Apr 28, 2008 at 3:27 PM, Henrik Nordstrom
<[email-masked]> wrote:
>
>  On mån, 2008-04-28 at 13:44 +0200, Robert Siemer wrote:
>
>  > That raises three issues:
>  >
>  > 1) It's not in RFC2616 (weak comparison for non-GET) and so it's not on
>  >    RFC2616bis charta.
>
>  Not convinced. The current limitations on weak etags is just silly with
>  the exception of If-Range..
>
>  In my view it's a specification error that validators based on
>  Last-Modified is allowed in more places than weak etag based ones.

Well said.  The meaning of any non-range conditional request message
using a weak validator is unambiguous.

So would this be a "clarify conformance criteria" fix per the charter?
 Those MUST NOTs seem to make no sense AFAICT, unless there's
implementation issues I'm not aware of.

Mark.
-- 
Mark Baker. Ottawa, Ontario, CANADA. http://www.markbaker.ca
Coactus; Web-inspired integration strategies http://www.coactus.com

---

From: Julian Reschke <[email-masked]>
Date: Fri, 02 May 2008 16:59:21 +0200

Mark Baker wrote:
> ...
> ...

+1.

In particular, it would be sufficient to *allow* servers to support weak 
entity tags in these cases. This change wouldn't break (IMHO) any 
existing compliant HTTP/1.1 client/server.

BR, Julian

---

From: Pablo Castro <[email-masked]>
Date: Fri, 2 May 2008 08:53:38 -0700

+1

That would solve the scenarios we've been struggling with lately. In the context of Astoria we'd produce week ETags if we can't guarantee full coverage of the values that form a resource.

Two "style" questions about how to proceed:
-Does it seem reasonable to go ahead and include this in the implementation at this point? In the end, somebody has to go first :)
-About adding the clarification itself, how does that happen? We've been trying hard to avoid being "creating" about interpreting specs, so if we'll introduce this change in the implementation it would be great to know the path from now until the clarification makes it to the right place.

-pablo

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]]
Sent: Friday, May 02, 2008 7:59 AM
To: Mark Baker
Cc: Henrik Nordstrom; Robert Siemer; Brian Smith; Pablo Castro; [email-masked]; HTTP Working Group
Subject: Re: ETags and concurrency control

+1.

BR, Julian

---

From: Henrik Nordstrom <[email-masked]>
Date: Fri, 02 May 2008 19:21:13 +0200

On ons, 2008-04-30 at 21:46 -0700, Brian Smith wrote:

> AFAICT, The issue with i101 is that some servers do not general
> *useful* weak ETags; at least as Apache seems to just generate weak
> ETags that it will never match (which is inefficient but not totally
> broken).

This is a bug in Apache, not the specifications. There is an open bug
report on this, and has been acked by several Apache developers. But
it's a very minor one, based on the simple fact that Apache when using
the default filesystem based store can not guarantee there is no other
process messing with the filesystem content at the same sub-second.
Additionally current releases allow the administrator to tune this,
enabling the server to generate strong ETags immediately (must only be
enabled if one knows there is no other processes modifying the files in
"bad ways")

Other Apache backends using weak etags can handle them fine.

> To me, "semantically equivalent" is something that is definitely
> vague. However, the use of strong ETags for range requests makes
> things clearer: If you can guarantee that your server will always
> generate a byte-for-byte identical representation (usable for range
> requests) for a given ETag, use a strong one; otherwise, if you think 
> that generating an ETag makes sense at all, use a weak one.

Exactly.

And if you can guarantee (within reasonable doubt) thatno two
"significantly different" objects share the same etag always send an
etag. You should only skip ETag if you strongly suspect that you may
assign the same etag to two versions which is significantly different.

The typical case for strong/weak etags is 

Strong:
- Guaranteed octet equality.

Weak:

- Dynamically generated content froma versioned / controlled or static
source, where some parameters such as the generation timestamp, or the
exact formatting of the response may differ from request to request
based on parameters which for some reason can not be included in the
ETag generation algorithm used by the server.

- When updates isn't critical and it doesn't matter at all if someone
uses a version which is somewhat dated. I.e. A pageview counter image.

> For example, mod_deflate should never return a strong ETag because the
> entity it generates is dependent on the system configuration. With a
> strong ETag, mod_deflate needs to ensure that the ETag changes
> whenever the mod_deflate configuration changes and whenever the
> system's zlib changes; with a weak ETag, it could continue to ignore
> these little details (like it does now when generating strong ETags.)

With this reasoning pretty much nothing except for a static file store
can return strong ETags. I disagree. Servers like mod_deflate SHOULD use
the system configuration parameters as input to their ETag generation
algorithm if these are likely to change during the lifetime of an
object.

> In fact, I would say that weak ETags should be the default choice, and
> strong ETags should only be used when the application has specifically
> ensured that there is a one-to-one correspondence between the ETag and
> the byte stream that comprises the entity--in other words, only use
> strong ETags when you could support range requests (whether you
> support range requests or not). The restriction against using weak
> ETags in PUT and DELETE requests forces applications to use strong
> ETags in situations where they are not guaranteeing this one-to-one
> correspondence.

To this I can agree.

> The alternative would be to deprecate weak ETags and discourage their
> use, weaken the definition of strong ETag to match what weak ETags
> were original for, and then say that strong ETags are only guaranteed
> to have a one-to-one correspondence with entities if the server
> supports range requests for that resource. That seems to be the
> effective result of the proposed i101 resolution.

This won't fly as the server has no control over who makes ranges of the
response. It may well be an intermediary proxy server between the client
and server.

Regards
Henrik

---

From: Julian Reschke <[email-masked]>
Date: Fri, 02 May 2008 20:36:27 +0200

> -Does it seem reasonable to go ahead and include this in the implementation at this point? In the end, somebody has to go first :)

I don't think that would be a problem.

If you fear doing something RFC2616 says you shouldn't, there's a 
workaround you may want to consider: instead of RFC2616's "If-*" 
headers, use the "If" header defined in WebDAV 
(<http://greenbytes.de/tech/webdav/rfc4918.html#rfc.section.10.4>).

This one lets you do the checks you need, even with weak entity tags.

> -About adding the clarification itself, how does that happen? We've been trying hard to avoid being "creating" about interpreting specs, so if we'll introduce this change in the implementation it w…

In general, I see it like this:

a) raise the issue over here on the mailing list (done)

b) get rough consensus that we should do something (partly done)

c) get it onto the issues list (I think it's part of issue 101, 
<http://www3.tools.ietf.org/wg/httpbis/trac/ticket/101>)

d) achieve rough consensus for new text

e) editors apply change, new draft is published some time later

...

x) IESG Last Call

y) IESG approval

z) RFC publication

(I omitted a few steps :-)

BR, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 9 May 2008 15:46:16 +1000

I think this is a somewhat different issue, but it is certainly related.

See:
   http://www3.tools.ietf.org/wg/httpbis/trac/ticket/116

The relevant text seems to be:

p4 section 5:
> Clients MAY issue simple (non-subrange) GET requests with either  
> weak validators or strong validators. Clients MUST NOT use weak  
> validators in other forms of request.

Proposal:

Clients MUST NOT use weak validators in range requests [ref to p5].

> A cache or origin server receiving a conditional request, other than  
> a full-body GET request, MUST use the strong comparison function to  
> evaluate the condition.
Proposal:

A cache or origin server receiving a conditional range request [ref to  
p5] MUST use the strong comparison function to evaluate the condition.

p4 section 7.4:
> See Section 5 for rules on how to determine if two entities tags  
> match. The weak comparison function can only be used with GET or  
> HEAD requests.

Proposal:

See Section 5 for rules on how to determine if two entity tags match.

On 03/05/2008, at 4:36 AM, Julian Reschke wrote:

>
> Pablo Castro wrote:
>> +1
>> That would solve the scenarios we've been struggling with lately.  
>> In the context of Astoria we'd produce week ETags if we can't  
>> guarantee full coverage of the values that form a resource.
>> -Does it seem reasonable to go ahead and include this in the  
>> implementation at this point? In the end, somebody has to go first :)
>
> headers, use the "If" header defined in WebDAV (<http://greenbytes.de/tech/webdav/rfc4918.html#rfc.section.10.4 
> >).
>
>> -About adding the clarification itself, how does that happen? We've  
>> been trying hard to avoid being "creating" about interpreting  
>> specs, so if we'll introduce this change in the implementation it  
>> would be great to know the path from now until the clarification  
>> makes it to the right place.
>
> c) get it onto the issues list (I think it's part of issue 101, <http://www3.tools.ietf.org/wg/httpbis/trac/ticket/101 
> >)
>

--
Mark Nottingham     http://www.mnot.net/

---

From: Julian Reschke <[email-masked]>
Date: Fri, 09 May 2008 09:44:56 +0200

>   http://www3.tools.ietf.org/wg/httpbis/trac/ticket/116

Thanks for making that a separate issue.

>> Clients MAY issue simple (non-subrange) GET requests with either weak 
>> validators or strong validators. Clients MUST NOT use weak validators 
>> in other forms of request.
> 
> Proposal:
> 

+1.

>> A cache or origin server receiving a conditional request, other than a 
>> full-body GET request, MUST use the strong comparison function to 
>> evaluate the condition.
> Proposal:
> 

+1.

>> match. The weak comparison function can only be used with GET or HEAD 
>> requests.
> 
> Proposal:
> 

+1.

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Fri, 09 May 2008 16:37:30 +0200

I'd also mention it in Changes from RFC2616:

"Appendix A.  Compatibility with Previous Versions

A.1.  Changes from RFC 2616

    Allow weak entity tags in all requests except range requests
    (Sections 5 and 7.4)."

Full diffs for the proposed change at 
<http://tools.ietf.org/wg/httpbis/trac/attachment/ticket/116/i116.diff>...

BR, Julian

---

From: Henrik Nordstrom <[email-masked]>
Date: Sun, 11 May 2008 17:44:31 +0200

+1

+1

+1

Regards
Henrik

---

From: Julian Reschke <[email-masked]>
Date: Wed, 14 May 2008 17:45:27 +0200

OK.

It seems this one is ready for closure.

If there are no objections, I'll apply the change proposed in 
<http://tools.ietf.org/wg/httpbis/trac/attachment/ticket/116/i116.diff> 
tomorrow.

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 15 May 2008 13:46:05 +0200

> <http://tools.ietf.org/wg/httpbis/trac/attachment/ticket/116/i116.diff> 

Done with <http://www3.tools.ietf.org/wg/httpbis/trac/changeset/245>.

BR, Julian

---

From: Pablo Castro <[email-masked]>
Date: Sat, 17 May 2008 19:19:02 -0700

This is now implemented in Astoria, so I guess we already have 1 working implementation of this nit in the spec :)

More seriously, the effective/fast discussion and turn-around of this clarification was great. appreciated.

-pablo

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]]
Sent: Thursday, May 15, 2008 4:46 AM
To: Henrik Nordstrom
Cc: Mark Nottingham; Pablo Castro; Mark Baker; Robert Siemer; Brian Smith; [email-masked]; HTTP Working Group
Subject: Re: [#116] ETags and concurrency control

BR, Julian