Subject: Re: Etag-on-write, 2nd attempt (== IETF draft 01)

From: Lisa Dusseault <[email-masked]>
Date: Wed, 6 Sep 2006 14:21:51 -0700

This draft has discussion of how subsequent rewrites might affect a  
decision on how to return ETags in PUT responses when the content  
might have been rewritten.  That's great, but there's another kind of  
consideration entirely, and that's how a client that maintains an  
offline synchronized store, or cache of resources, will be affected.

A likely real-world example is a CalDAV server that can augment the  
information in a VEVENT: filling in attendee status, adding alternate  
attendee addresses, or filling in location information, or adding a  
URL where the VEVENT can be seen as a HTML Web page (think of  
calendar sites like meetup and upcoming.org).  While a subsequent  
edit of the same event might or might not result in the same outcome  
whether or not the client downloaded the intermediate version,  
there's still a problem if the cached/offline copy isn't as rich as  
the server copy.  A user who sees the discrepancy will assume bugginess.

It would sure be great if existing clients that already do offline  
synch, and might already encounter this case, would work  
appropriately with a new mechanism improving the functionality of  
ETags on PUTs.

Lisa

On Aug 25, 2006, at 2:03 PM, Julian Reschke wrote:

>
> Hi,
>
> this new draft attempts to resolve issues raised over here two  
> weeks ago, mainly in the extensibility model (removed for now to  
> avoid bikeshed-type discussions), and with respect to backwards  
> compatibility with components that treat newly introduced headers  
> as entity headers (I'm trying to get away by making the header an  
> entity header, eagerly awaiting new feedback).
>
> I hope that this addresses the feedback I received so far (thanks a  
> lot!).
>
> Best regards, Julian
>
>
> [email-masked] wrote:
>> A New Internet-Draft is available from the on-line Internet-Drafts  
>> directories.
>> 	Title		: The Hypertext Transfer Protocol (HTTP) Entity Tag  
>> ("ETag") Response Header in Write Operations
>> 	Author(s)	: J. Reschke
>> 	Filename	: draft-reschke-http-etag-on-write-01.txt
>> 	Pages		: 20
>> 	Date		: 2006-8-25
>> 	
>> The Hypertext Transfer Protocol (HTTP) specifies a state identifier,
>>    called "Entity Tag", to be returned in the "ETag" response header.
>>    However, the description of this header for write operations  
>> such as
>>    PUT is incomplete, and has caused confusion among developers and
>>    protocol designers, and potentially interoperability problems.
>>    This document explains the problem in detail and suggests both a
>>    clarification for a revision to the HTTP/1.1 specification  
>> (RFC2616)
>>    and a new header for use in responses, making HTTP entity tags  
>> more
>>    useful for user agents that want to avoid round-trips to the  
>> server
>>    after modifying a resource.
>> A URL for this Internet-Draft is:
>> http://www.ietf.org/internet-drafts/draft-reschke-http-etag-on- 
>> write-01.txt
>> To remove yourself from the I-D Announcement list, send a message  
>> to [email-masked] with the word unsubscribe in the  
>> body of the message. You can also visit https://www1.ietf.org/ 
>> mailman/listinfo/I-D-announce to change your subscription settings.
>> Internet-Drafts are also available by anonymous FTP. Login with  
>> the username "anonymous" and a password of your e-mail address.  
>> After logging in, type "cd internet-drafts" and then "get draft- 
>> reschke-http-etag-on-write-01.txt".
>> A list of Internet-Drafts directories can be found in
>> http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow- 
>> sites.txt
>> Internet-Drafts can also be obtained by e-mail.
>> Send a message to:
>> 	[email-masked].
>> In the body type:
>> 	"FILE /internet-drafts/draft-reschke-http-etag-on-write-01.txt".
>> 	
>> NOTE:	The mail server at ietf.org can return the document in
>> 	MIME-encoded form by using the "mpack" utility.  To use this
>> 	feature, insert the command "ENCODING mime" before the "FILE"
>> 	command.  To decode the response(s), you will need "munpack" or
>> 	a MIME-compliant mail reader.  Different MIME-compliant mail readers
>> 	exhibit different behavior, especially when dealing with
>> 	"multipart" MIME messages (i.e. documents which have been split
>> 	up into multiple messages), so check your local documentation on
>> 	how to manipulate these messages.
>> Below is the data which will enable a MIME compliant mail reader
>> implementation to automatically retrieve the ASCII version of the
>> Internet-Draft.
>> --------------------------------------------------------------------- 
>> ---
>> _______________________________________________
>> I-D-Announce mailing list
>> [email-masked]
>> https://www1.ietf.org/mailman/listinfo/i-d-announce
>
>

---

From: "Joe Gregorio" <[email-masked]>
Date: Fri, 8 Sep 2006 16:55:03 -0400

Editorial nits:

1. " programmers asked servers to always return "ETag" headers upon
   PUT, never ever to change t"

   I can't follow that sentence. Sorry, no construtive suggestions on my part.

2. Section 1.2

    "reasons for HTTP's success, allowing to be used for a wide range of"

     allowing IT to be

General:

 How is "Entity-Transform: identity" any different than returning
a strong ETag as opposed to a weak ETag?

   -joe

On 8/25/06, Julian Reschke <[email-masked]> wrote:
>
> Hi,
>
> this new draft attempts to resolve issues raised over here two weeks
> ago, mainly in the extensibility model (removed for now to avoid
> bikeshed-type discussions), and with respect to backwards compatibility
> with components that treat newly introduced headers as entity headers
> (I'm trying to get away by making the header an entity header, eagerly
> awaiting new feedback).
>
> I hope that this addresses the feedback I received so far (thanks a lot!).
>
> Best regards, Julian
>
>
> >       Title           : The Hypertext Transfer Protocol (HTTP) Entity Tag ("ETag") Response Header in Write Operations
> >       Author(s)       : J. Reschke
> >    However, the description of this header for write operations such as
> >    clarification for a revision to the HTTP/1.1 specification (RFC2616)
> >    and a new header for use in responses, making HTTP entity tags more
> >    useful for user agents that want to avoid round-trips to the server
> > http://www.ietf.org/internet-drafts/draft-reschke-http-etag-on-write-01.txt
> >
> > To remove yourself from the I-D Announcement list, send a message to
> > [email-masked] with the word unsubscribe in the body of
> > the message.
> > You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce
> > to change your subscription settings.
> >
> > Internet-Drafts are also available by anonymous FTP. Login with the
> > username "anonymous" and a password of your e-mail address. After
> > logging in, type "cd internet-drafts" and then
> > "get draft-reschke-http-etag-on-write-01.txt".
> >
> > http://www.ietf.org/shadow.html
> > or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
> >
> > ------------------------------------------------------------------------
> >

-- 
Joe Gregorio        http://bitworking.org

---

From: Julian Reschke <[email-masked]>
Date: Tue, 12 Sep 2006 20:10:52 +0200

>   I can't follow that sentence. Sorry, no construtive suggestions on my 

Any kind of feedback is useful :-) In this case, it's a typo, it's 
supposed to say:

"...client programmers asked server programmers..."

>    "reasons for HTTP's success, allowing to be used for a wide range of"

Yep.

> a strong ETag as opposed to a weak ETag?

The issue is that without further information, a client can not assume 
that upon PUT, the entity was stored octet-by-octet, thus it needs to 
refetch the content if it needs that kind of equivalence. This may not 
be obvious from RFC2616, but it is certainly the conclusion we arrived 
at over here on the HTTP mailing list something like 10 months ago. Thus 
the proposed clarifications in Section 3, and the "Entity-Transform" 
extension header definition.

As this is the key point of the whole exercise, I would really 
appreciate if you could re-read the draft with an eye on this, and let 
me know whether you agree with the analysis of the problem in the first 
place :-)

Thanks a lot,

Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 12 Sep 2006 20:01:17 +0200

Lisa Dusseault schrieb:
> decision on how to return ETags in PUT responses when the content might 
> have been rewritten.  That's great, but there's another kind of 
> attendee addresses, or filling in location information, or adding a URL 
> where the VEVENT can be seen as a HTML Web page (think of calendar sites 
> like meetup and upcoming.org).  While a subsequent edit of the same 
> event might or might not result in the same outcome whether or not the 
> client downloaded the intermediate version, there's still a problem if 
> the cached/offline copy isn't as rich as the server copy.  A user who 
> sees the discrepancy will assume bugginess.
> 
> synch, and might already encounter this case, would work appropriately 
> with a new mechanism improving the functionality of ETags on PUTs.

I do agree that this is an issue for which a new HTTP feature may be 
desirable, similar things apply to AtomPub clients using "foreign 
markup" (which servers may or may not preserve).

That being said, defining HTTP extensions is hard enough. Thus, I prefer 
to identify a single problem, and to propose a minimal solution to that. 
If enough people are interested to work on a broader issue instead, 
that's fine with me. However I feel that extending the scope is probably 
the best way not to get anywhere at all (but I'd be happy to find out 
otherwise).

Best regards, Julian

---

From: Lisa Dusseault <[email-masked]>
Date: Tue, 12 Sep 2006 11:46:50 -0700

Actually, I'm proposing to narrow the scope by adding the requirement  
of working with existing deployed offline-cache clients, which will  
limit the number of solutions that will meet the overall set of  
requirements.  This doesn't require a new feature.

Lisa

On Sep 12, 2006, at 11:01 AM, Julian Reschke wrote:

> Lisa Dusseault schrieb:
>> This draft has discussion of how subsequent rewrites might affect  
>> a decision on how to return ETags in PUT responses when the  
>> content might have been rewritten.  That's great, but there's  
>> another kind of consideration entirely, and that's how a client  
>> that maintains an offline synchronized store, or cache of  
>> resources, will be affected.
>> A likely real-world example is a CalDAV server that can augment  
>> the information in a VEVENT: filling in attendee status, adding  
>> alternate attendee addresses, or filling in location information,  
>> or adding a URL where the VEVENT can be seen as a HTML Web page  
>> (think of calendar sites like meetup and upcoming.org).  While a  
>> subsequent edit of the same event might or might not result in the  
>> same outcome whether or not the client downloaded the intermediate  
>> version, there's still a problem if the cached/offline copy isn't  
>> as rich as the server copy.  A user who sees the discrepancy will  
>> assume bugginess.
> I do agree that this is an issue for which a new HTTP feature may  
> be desirable, similar things apply to AtomPub clients using  
> "foreign markup" (which servers may or may not preserve).
>
> That being said, defining HTTP extensions is hard enough. Thus, I  
> prefer to identify a single problem, and to propose a minimal  
> solution to that. If enough people are interested to work on a  
> broader issue instead, that's fine with me. However I feel that  
> extending the scope is probably the best way not to get anywhere at  
> all (but I'd be happy to find out otherwise).
>
> Best regards, Julian
>

---

From: Julian Reschke <[email-masked]>
Date: Wed, 13 Sep 2006 00:49:50 +0200

Lisa Dusseault schrieb:
> 
> Actually, I'm proposing to narrow the scope by adding the requirement of 
> working with existing deployed offline-cache clients, which will limit 
> the number of solutions that will meet the overall set of requirements.  
> This doesn't require a new feature.
> 
> Lisa

I'm really not sure what you're talking about. The draft spends a lot of 
time explaining that ETags returned upon PUT do /not/ indicate the state 
of the entity sent by the client, but the state of the entity as stored 
by the server. As servers are allowed to (and indeed do) rewrite 
content, there's no generic way in HTTP to avoid refetching the content, 
if octet-by-octet identity is required (which I don't think is in many 
cases).

That being said, the draft makes a proposal for a very small extension 
which is backwards compatible with HTTP, and which allows clients to 
detect when a server indeed did not rewrite the content.

That's all.

If you don't agree with the base analysis (1st paragraph over here, and 
Section 2 in the draft), please be more specific.

Best regards, Julian

---

From: Lisa Dusseault <[email-masked]>
Date: Tue, 12 Sep 2006 15:55:09 -0700

On Sep 12, 2006, at 3:49 PM, Julian Reschke wrote:

> Lisa Dusseault schrieb:
>> Actually, I'm proposing to narrow the scope by adding the  
>> requirement of working with existing deployed offline-cache  
>> clients, which will limit the number of solutions that will meet  
>> the overall set of requirements.  This doesn't require a new feature.
>> Lisa
>
> I'm really not sure what you're talking about. The draft spends a  
> lot of time explaining that ETags returned upon PUT do /not/  
> indicate the state of the entity sent by the client, but the state  
> of the entity as stored by the server. As servers are allowed to  
> (and indeed do) rewrite content, there's no generic way in HTTP to  
> avoid refetching the content, if octet-by-octet identity is  
> required (which I don't think is in many cases).

It's technically correct to say that servers are allowed to rewrite  
content.  However, existing deployed offline-cache clients assume  
that they do not.  Thus, it would be a seriously beneficial feature  
and possibly a requirement of new work to have backward-compatibility  
with that assumption, whether it was a wise assumption or not -- it's  
a deployed assumption.

Lisa

---

From: Helge Hess <[email-masked]>
Date: Wed, 13 Sep 2006 01:33:40 +0200

On Sep 13, 2006, at 24:55, Lisa Dusseault wrote:
> that they do not.

To the client it doesn't(/shouldn't) matter whether the server  
rewrote the content or whether another client did? They need to deal  
with the situation anyway? It changed and they need to update the  
data. The earlier they get changes by "others" the better, ideally in  
the PUT response (yes, it doesn't even need to be the case that the  
server modified the resource if the etag changes, it could have been  
a different client).

Existing deployed servers assume that they do that. I don't think  
that one or the other has an "advantage" here :-)

Greets,
   Helge
-- 
Helge Hess
http://docs.opengroupware.org/Members/helge/

---

From: Julian Reschke <[email-masked]>
Date: Wed, 13 Sep 2006 01:48:41 +0200

Lisa Dusseault schrieb:
> content.  However, existing deployed offline-cache clients assume that 
> they do not.  Thus, it would be a seriously beneficial feature and 
> possibly a requirement of new work to have backward-compatibility with 
> that assumption, whether it was a wise assumption or not -- it's a 
> deployed assumption.

Lisa,

a) if deployed clients make that assumption, and HTTP says that's an 
incorrect assumption, this problem needs to be resolved, unless you can 
demonstrate that these clients indeed work with all deployed servers 
(which they wouldn't).

b) as far as I can recall we had the very same discussion and you failed 
to come up with a deployed client making that assumption (see 
<http://lists.w3.org/Archives/Public/ietf-http-wg/2006AprJun/0094.html>).

Best regards, Julian

---

From: "Robert Sayre" <[email-masked]>
Date: Tue, 12 Sep 2006 20:03:36 -0400

On 9/12/06, Julian Reschke <[email-masked]> wrote:
>
> a) if deployed clients make that assumption,

Are they making the assumption that the response will be
octet-for-octet identical or semantically identical?

Atom clients wouldn't understand the difference between an entry
submitted in UTF-8 and one returned in UTF-16.

-- 

Robert Sayre

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 13 Sep 2006 01:05:41 +0100

> server modified the resource if the etag changes, it could have been  

Isn't this the problem scenario:

    1. An existing deployed client (or cache, it doesn't matter) sends PUT.

    2. The server sends 200 OK with Etag back in response.

    3. Later, same client sends GET with If-None-Match and the Etag it
       received in step 2.

    4. The server sends 304 Not Modified, because the Etag matches
       what it sent before.

    5. The existing deployed client erroneously* uses the entity it
       sent in step 1.  (If it's a proxy cache, it serves that entity
       to its clients).

* Erroneous only if the server is allowed to modify the entity it
receives in PUT _and_ respond with an Etag which matches that modified
entity.

To avoid such errors _in new deployments_ which are intended to
interoperate with existing ones, one (or both) of the following are
suitable:

   1. If a server sends a strong Etag in a PUT response, future
      conditional requests using that Etag will only match if the
      entity has not changed at all.  (If a server sends a weak Etag
      in a PUT response, future conditional requests using that Etag
      will only match if the entity has not changed according to the
      meaning of "weak".)

   2. If a client receives a strong Etag in a PUT response, it should
      ignore it.

If there are already deployed clients which don't satisfy requirement
2, then it would be appropriate to insist on requirement 1 in future
specifications, so that future deployments interoperate with existing
ones.

If there are already deployed servers which don't satisfy requirement
1, then it would be appropriate to insist on requirement 2 in future
specifications, so that future deployments interoperate with existing
ones.

But if there are _both_ deployed clients and servers which don't
satisfy those requirements out there, those will already fail to
interoperate with each other.  There is no specification which can fix
this.  However, the choice of 1, 2 or both will affect
interoperability of future deployments with the existing onesa.

-- Jamie

---

From: Lisa Dusseault <[email-masked]>
Date: Tue, 12 Sep 2006 19:00:39 -0700

> specifications, so that future deployments interoperate with existing

Yes, that's exactly it -- Xythos WebFile Client behaved this way, and  
based on their user-facing behavior I suspect many others do too  
although I can't test them right now.  I'm thinking particularly  
synch tools like 'sitecopy', but possibly also networked share  
viewers like WebDAV-FS on Mac and the Windows clients.

Throughout 1999 to 2005, the WebDAV and HTTP community tended to  
encourage this kind of implementation. This W3C Note certainly does:  
http://www.w3.org/1999/04/Editing/

> specifications, so that future deployments interoperate with existing

We are starting to envision such servers, but I don't know that  
they're deployed yet, and certainly not for the general case.  It's  
particularly for the case of servers specialized to store certain  
formats, like calendar data or XML data, that this type of behavior  
may in the future become implemented/deployed.

The two special cases I'm aware of are CalDAV and XCAP: both store a  
specialized format and a certain kind of data, understand the data at  
a semantic level, and a server could conceivably be "extra helpful"  
and add data to those resources.

  * For CalDAV, we proposed that if a server adds information to a  
resource upon write, it not return a strong ETag, and thus work with  
existing clients.

  * XCAP says in 7.11:
    When a client uses conditional PUT and DELETE operations, it can
    apply those changes to its local cached copy, and update the  
value of
    the entity tag for the locally cached copy based on the Etag header
    field returned in the response.  As long as no other clients try to
    modify the document, the client will be able to perform conditional
    operations on the document without ever having to perform separate
    GET operations to synchronize the document and it's entity tags with
    the server.

Note that XCAP also says in 8.5 that the server MUST return the ETag  
header in all 200 or 201 responses to PUT.  To me, combining these  
statements means that there's only one way a server could make  
modifications to a resource on write:
	Start with a resource with ETag value E1.
	Accept the PUT request and return a strong ETag as required.  It  
must be a new value, lets say E2.
	Perform its modification and change the resource ETag to another new  
value, say E3.
	
With this approach, an XCAP server could literally meet the  
requirement to return the ETag header, yet also force clients that  
are written to work as described in 7.11 to get the correct resource  
version.

JDR may be able to clarify what the common assumption is for XCAP --  
perhaps it's that servers *not* modify resources.

Anyway, leaving the special cases again, I am not aware of any  
general WebDAV servers -- servers which accept general content just  
like a file server would -- which make modifications before returning  
a strong ETag.

> interoperability of future deployments with the existing one.
>

Yes, this is a good analysis.  I know there are significant client  
deployments based on assuming no server-rewriting.  I am not aware of  
server deployments that do rewriting and return strong ETags.  I'm  
thus quite confident about where the interop advantage lies.

Lisa

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 12 Sep 2006 19:52:08 -0700

On Sep 12, 2006, at 7:00 PM, Lisa Dusseault wrote:
> deployments based on assuming no server-rewriting.  I am not aware  
> of server deployments that do rewriting and return strong ETags.   
> I'm thus quite confident about where the interop advantage lies.

I agree.  As I said last time, the easier solution is to simply  
require that
ETag in a response to PUT means that the client can use that entity  
tag in
future conditional requests (that includes IMS, If-Match, and Range- 
based
conditional requests).  There is absolutely no reason for the server  
to send
an ETag on a PUT response if the server does not intend to support that
functionality, and I am quite sure that none of the server vendors will
care provided that the requirement is properly stated as an interface
constraint and not an implementation constraint.

OTOH, the mechanism for how the server "stores" the representation,  
whether
or not it is stored byte-for-byte, and how a server might otherwise  
manage
to accomplish the feat of handling a conditional request without  
preserving
byte-level equality is none of our business and should never appear  
in an
IETF specification regarding HTTP.

....Roy

---

From: Yves Lafon <[email-masked]>
Date: Wed, 13 Sep 2006 14:46:41 +0200 (MEST)

>>> content.  However, existing deployed offline-cache clients assume

Well, I have a server that does exactly that, as CVS is expanding keywords 
like $Id$ on commit.

>    2. The server sends 200 OK with Etag back in response.

Would it be OK for a server to return a 205 Reset Content with the Etag ?
That way the client would know that a refetch is in order.

> receives in PUT _and_ respond with an Etag which matches that modified

What is the scope of the "modification" here? 13.3.3 says that the concept 
of a modification is up to the server.

-- 
Yves Lafon - W3C
"Baroula que barouleras, au tiéu toujou t'entourneras."

---

From: Julian Reschke <[email-masked]>
Date: Wed, 13 Sep 2006 17:55:36 +0200

Jamie Lokier schrieb:
>> ...

That wouldn't always be a problem. There are lot's of scenarios where it 
would be ok to do that, as outlined in 
<http://greenbytes.de/tech/webdav/draft-reschke-http-etag-on-write-01.html#rfc.section.A.1>.

>       will only match if the entity has not changed according to the

That would defeat some uses cases which (IMHO) are legal as per RFC2616 
(where octet-by-octet identity isn't required).

> specifications, so that future deployments interoperate with existing

I know these servers exists and are deployed, because I wrote one of 
them :-).

> interoperability of future deployments with the existing onesa.

As far as I can tell, the interop problem already is there (that's why I 
wrote the draft suggesting an approach that I *think* is faithful to 
RFC2616 and resolves the problem for new clients and servers, as 
discussed in XCAP and CalDAV).

Best regards, Julian

---

From: Lisa Dusseault <[email-masked]>
Date: Wed, 13 Sep 2006 09:06:46 -0700

On Sep 13, 2006, at 5:46 AM, Yves Lafon wrote:

> Well, I have a server that does exactly that, as CVS is expanding  
> keywords like $Id$ on commit.

And also returns a strong ETag on PUT when making these changes?   
Even if so, that sounds like one of those cases which, if the client  
caches the resource entity they PUT, whether that's for viewing now  
or modification later, will work reasonably well.  Not perfectly --  
clients won't have the replacement value for those keywords, which  
might be nice for the user viewing the resource.

So just how do your clients handle this?  do they have offline caches?

> That way the client would know that a refetch is in order.

That seems good to me so far.

thx,
Lisa

---

From: Julian Reschke <[email-masked]>
Date: Wed, 13 Sep 2006 18:09:16 +0200

Lisa Dusseault schrieb:
> although I can't test them right now.  I'm thinking particularly synch 
> tools like 'sitecopy', but possibly also networked share viewers like 
> WebDAV-FS on Mac and the Windows clients.  

That's incorrect, at least as the Xythos client is concerned (see 
<http://lists.w3.org/Archives/Public/ietf-http-wg/2006AprJun/0090.html>).

> encourage this kind of implementation. This W3C Note certainly 
> does: http://www.w3.org/1999/04/Editing/

I'm really not sure how this is relevant for the discussion. It's 
perfectly OK for a client to use continue editing using an ETag obtained 
from PUT, even if the server rewrote the content. Saving the modified 
content again will just cause the content to be rewritten again. This is 
not a case of a "lost update" problem.

> We are starting to envision such servers, but I don't know that they're 
> deployed yet, and certainly not for the general case.  It's particularly 

They are, and have been around for quite some time (SAP's KM allows 
content rewriting upon PUT, in which case it will still return strong 
ETags).

> for the case of servers specialized to store certain formats, like 
> calendar data or XML data, that this type of behavior may in the future 
> become implemented/deployed.   

Other use cases are filtering and keyword expansion.

> specialized format and a certain kind of data, understand the data at a 
> semantic level, and a server could conceivably be "extra helpful" and 
> add data to those resources.  

AtomPub as well.

> resource upon write, it not return a strong ETag, and thus work with 

It would be nice if you could provide evidence of clients (written prior 
to the text in the CalDAV spec) that make this assumption.

>    apply those changes to its local cached copy, and update the value of

Yes. Note that this works even though XCAP servers usually will reformat 
the XML, as long as no byte-range requests are made.

> Accept the PUT request and return a strong ETag as required.  It must be 
> a new value, lets say E2. 
> With this approach, an XCAP server could literally meet the requirement 
> to return the ETag header, yet also force clients that are written to 
> work as described in 7.11 to get the correct resource version.  

I'm not sure why a third ETag is needed. Could you please clarify that 
point?

> Anyway, leaving the special cases again, I am not aware of any general 
> WebDAV servers -- servers which accept general content just like a file 
> server would -- which make modifications before returning a strong ETag.

Well, I am (see previous mail for details).

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 13 Sep 2006 18:12:54 +0200

> That way the client would know that a refetch is in order.

That probably would have been a good approach, but as RFC2616 doesn't 
require servers to do this, clients can not rely on it.

My draft tries to give the client the information it needs without 
breaking interoperability with existing servers.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 13 Sep 2006 18:15:08 +0200

> octet-for-octet identical or semantically identical?

I guess the former. Let's stay away from the weak etag/semantic 
equivalence thing for now :-).

> submitted in UTF-8 and one returned in UTF-16.

Right, and that's not a problem at all in practice.

Best regards, Julian

---

From: Lisa Dusseault <[email-masked]>
Date: Wed, 13 Sep 2006 09:46:54 -0700

On Sep 13, 2006, at 9:09 AM, Julian Reschke wrote:

> Lisa Dusseault schrieb:
>> Yes, that's exactly it -- Xythos WebFile Client behaved this way,  
>> and based on their user-facing behavior I suspect many others do  
>> too although I can't test them right now.  I'm thinking  
>> particularly synch tools like 'sitecopy', but possibly also  
>> networked share viewers like WebDAV-FS on Mac and the Windows  
>> clients.
>

I can't really see how we disagree here.  If the server returns a  
strong ETag, the Xythos client assumes that the content was written  
as sent.  Later, when the client attempts to refresh its cache, if  
the ETag is still the same, the client happily continues using the  
entity that it PUT.   Thus, a user can refresh and refresh and  
refresh, and still not see quite what the server has stored and other  
clients/users see.

With some kinds of modifications that may be important, in some cases  
it may be annoying, in some cases it might be all-but-indetectable.   
But I worry about an approach that handles that whole scale of  
modifications the same way and leaves existing deployed clients  
ignorant of all such changes.

I guess it might be reasonable to slice and dice.
  - For changes that really have no semantic effect  (like XML re- 
formatting) it would be OK for a server to behave as described in  
your draft.
  - For changes that have even small semantic effects (like variable  
replacement), it's very useful to notify existing clients of the  
change in such a way that the cached content eventually gets updated  
and the user sees what other users see.
  - For changes that have larger semantic effect or aren't  
repeatable, we do move to the "lost-update" problem and I believe we  
agree on the best course there.

> perfectly OK for a client to use continue editing using an ETag  
> obtained from PUT, even if the server rewrote the content. Saving  
> the modified content again will just cause the content to be  
> rewritten again. This is not a case of a "lost update" problem.

I agree, or to say again, I can't really see that we disagree here.   
If the server is making modifications that can be repeated  
successfully, there isn't a lost-update problem.  There may be an  
annoyed user problem.

>> they're deployed yet, and certainly not for the general case.   
>> It's particularly
>
> content rewriting upon PUT, in which case it will still return  
> strong ETags).

What kind of content-rewriting?  On what kind of resources?  Would a  
GIF be allowed and possibly modified?  A Word document?  Does the  
product ship this way or is it that customer modifications could  
result in a service instance that did content rewriting?

>
>> Note that XCAP also says in 8.5 that the server MUST return the  
>> ETag header in all 200 or 201 responses to PUT.  To me, combining  
>> these statements means that there's only one way a server could  
>> make modifications to a resource on write:
>> must be a new value, lets say E2. Perform its modification and  
>> change the resource ETag to another new value, say E3.
>> are written to work as described in 7.11 to get the correct  
>> resource version.
>
> I'm not sure why a third ETag is needed. Could you please clarify  
> that point?

The 3rd ETag is needed if the kind of change the server made might  
possibly be of any use to the user.  The 3rd Etag means that if the  
client refreshes its offline cache, the server change is downloaded.   
For simple XML spacing changes, that might not be any use.  For any  
semantic change, even if it's the kind that can be repeated for any  
future update (similar to CVS variable replacement), the 3rd ETag  
means that the user will see the content as its seen by others.

thanks,
Lisa

---

From: Julian Reschke <[email-masked]>
Date: Wed, 13 Sep 2006 18:58:03 +0200

Lisa Dusseault schrieb:
> I can't really see how we disagree here.  If the server returns a strong 
> ETag, the Xythos client assumes that the content was written as sent.  
> Later, when the client attempts to refresh its cache, if the ETag is 
> still the same, the client happily continues using the entity that it 
> PUT.   Thus, a user can refresh and refresh and refresh, and still not 
> see quite what the server has stored and other clients/users see.

Yes, but if the server *doesn't* return an ETag (as mandated by CalDAV), 
it simply uses the Last-Modified time stamp again. That is, it doesn't 
work at all with servers rewriting the content upon PUT, no matter 
whether they return an ETag or not.

> With some kinds of modifications that may be important, in some cases it 
> may be annoying, in some cases it might be all-but-indetectable.  But I 
> worry about an approach that handles that whole scale of modifications 
> the same way and leaves existing deployed clients ignorant of all such 
> changes.

Well, I was trying to avoid a situation where the extensions tries to be 
all for everybody, resulting in something nobody agrees to and 
implements. The draft as current (01) just provides a flag (rewritten or 
not), but of course it could be extended with other information.

>  - For changes that really have no semantic effect  (like XML 
> re-formatting) it would be OK for a server to behave as described in 
> your draft.  
> replacement), it's very useful to notify existing clients of the change 
> in such a way that the cached content eventually gets updated and the 
> user sees what other users see.  
>  - For changes that have larger semantic effect or aren't repeatable, we 
> do move to the "lost-update" problem and I believe we agree on the best 
> course there.

The problem here is the definition of "semantic effect". It all depends 
on the client. An XCAP client will not consider XML reformatting to be 
important, as long as the Infoset is preserved. A 
WebDAV-to-filesystem-mapper may be concerned by any kind of changes.

>>> particularly 
> What kind of content-rewriting?  On what kind of resources?  Would a GIF 
> be allowed and possibly modified?  A Word document?  Does the product 
> ship this way or is it that customer modifications could result in a 
> service instance that did content rewriting? 

Any kind of content rewriting is possible. It's hard to predict what 
customers actually use, but I know that at least a virus checker ships 
with the product.

>>> Accept the PUT request and return a strong ETag as required.  It must 
>>> be a new value, lets say E2. Perform its modification and change the 
>>> resource ETag to another new value, say E3.
>>>
> future update (similar to CVS variable replacement), the 3rd ETag means 
> that the user will see the content as its seen by others.

That's certainly one way to do that, but wouldn't issueing the second 
etag (E2) be pointless in this case?

Best regards, Julian

---

From: Lisa Dusseault <[email-masked]>
Date: Wed, 13 Sep 2006 10:28:01 -0700

On Sep 13, 2006, at 9:58 AM, Julian Reschke wrote:

>> strong ETag, the Xythos client assumes that the content was  
>> written as sent.  Later, when the client attempts to refresh its  
>> cache, if the ETag is still the same, the client happily continues  
>> using the entity that it PUT.   Thus, a user can refresh and  
>> refresh and refresh, and still not see quite what the server has  
>> stored and other clients/users see.
>
> Yes, but if the server *doesn't* return an ETag (as mandated by  
> CalDAV), it simply uses the Last-Modified time stamp again. That  
> is, it doesn't work at all with servers rewriting the content upon  
> PUT, no matter whether they return an ETag or not.

True.  We agree on the facts :)  To me, the implication is that a  
server that rewrote content would behave best with the Xythos client  
if it issued one ETag on PUT, then another ETag on server-rewrite.   
Then the server-rewrite behaves just like a client-rewrite and the  
Xythos client will eventually download the new content.

>>>> ETag header in all 200 or 201 responses to PUT.  To me,  
>>>> combining these statements means that there's only one way a  
>>>> server could make modifications to a resource on write:
>>>>
>>>> requirement to return the ETag header, yet also force clients  
>>>> that are written to work as described in 7.11 to get the correct  
>>>> resource version.
>>>
>> possibly be of any use to the user.  The 3rd Etag means that if  
>> the client refreshes its offline cache, the server change is  
>> downloaded.  For simple XML spacing changes, that might not be any  
>> use.  For any semantic change, even if it's the kind that can be  
>> repeated for any future update (similar to CVS variable  
>> replacement), the 3rd ETag means that the user will see the  
>> content as its seen by others.
>
> That's certainly one way to do that, but wouldn't issueing the  
> second etag (E2) be pointless in this case?

E2 must be issued because the server can't reuse E1 when it responds  
with 200 OK.  E2 must be different from E1 to comply with spec  
requirements; E3 must be different than E2 (and than E1) so that the  
client synchronizes the rewrite changes.

For a non-XCAP server, E2 isn't strictly necessary (though last- 
modified tricks might be necessary if the server returns no ETag).   
But for an XCAP server requiring a Etag on a 200 OK response to a  
PUT, E2 is necessary to comply with the spec requirements.

Lisa

---

From: Yves Lafon <[email-masked]>
Date: Wed, 13 Sep 2006 22:08:21 +0200 (MEST)

On Wed, 13 Sep 2006, Lisa Dusseault wrote:

> And also returns a strong ETag on PUT when making these changes?  Even if so,

Yes, it returns a strong ETag.

> that sounds like one of those cases which, if the client caches the resource 
> entity they PUT, whether that's for viewing now or modification later, will 
> work reasonably well.  Not perfectly -- clients won't have the replacement 
> value for those keywords, which might be nice for the user viewing the 
> resource.
>

Clients have local caches, but it doesn't really matter, as having the 
modified copy or the original one won't trigger an extra rewrite (ie: the 
cvs version is not changed) in case of a subsequent PUT without any other 
modifications, so per rfc2616,13.3.3, it is indeed a strong validator.

> That seems good to me so far.

Of course the issue is already deployed software, but every new way of 
handling this issue won't solve the way they behave.
It would be interesting to get the number of clients implementing 205.

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 14 Sep 2006 10:16:54 +0100

> That way the client would know that a refetch is in order.

No, because 205 Reset Content doesn't mean that.

Firstly, what's the point in sending the Etag in that case?  The
client can't do anything useful with it.

Secondly, 205 Reset Content has a different meaning which won't be
appropriate in many cases:

   the user agent SHOULD reset the document view which caused the
   request to be sent. This response is primarily intended to allow
   input for actions to take place via user input, followed by a
   clearing of the form in which the input is given so that the user
   can easily initiate another input action

Thirdly, 205 Reset Content does not mean "reset your cache entry to
force a refetch", and existing clients and proxies won't do that.

To cause a refresh, "Cache-control: no-cache" in the PUT response
_might_ work.  I'm not sure.

> of a modification is up to the server.

13.3.3 describes strong and weak validators, and explains that any
change to the entity, including a semantically insignificant one,
should change a strong Etag but doesn't have to change a weak Etag.

So the server is free to make modifications, but if it does then it
shouldn't send a strong Etag corresponding to the modified entity in
the PUT response.  It's ok to send a weak Etag corresponding to the
modified entity, provided the modification is semantically
insignificant - and that _is_ up to the server.  It's also ok to send
a strong Etag corresponding to the entity prior to modification.
Useless (except in pathological cases), but ok.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 14 Sep 2006 11:13:27 +0100

> <http://greenbytes.de/tech/webdav/draft-reschke-http-etag-on-write-01.html#rfc.section.A.1>.

The usage in that article should use a weak Etag, not a strong one.
>From the article:

    Note that the only problem here is that the client doesn't have an
    exact copy of the entity it's editing. However, from the server's
    point of view this is entirely irrelevant, because the "Revision"
    keyword will be automatically updated upon every write anyway.

Think about what happens if the client were to do a byte-range GET to
get part of the entity at this point (e.g. because it's editing a
large entity, and decides to discard _part_ of it from its cache).
Because of the substitution at the server, the byte ranges won't match
and the file being edited will get really messed up.

Also, think about what happens if the user _wants_ to view the actual
content at the server including substitutions made by the sever.
Either they won't be able to, as intermediate proxy caches will
continue serving the entity sent by PUT, no matter what client they're
using (or even multiple clients).  Or they can use a client which
disables proxy caching entirely even when that is unnecessary.

This is why the usage in
    <http://greenbytes.de/tech/webdav/draft-reschke-http-etag-on-write-01.html#rfc.section.A.1>
should use a weak Etag.

> (where octet-by-octet identity isn't required).

Where octet-by-octet identity isn't required, that's what _weak_ Etags
(beginning with "W/") are for.  It's ok to send those in a PUT
response without confusing clients.  Or at least, any clients which
treat weak Etags as though they were strong deserve to be confused, as
it's been in the spec forever.

> them :-)

Tsk! :)

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Thu, 14 Sep 2006 12:27:31 +0200

> client can't do anything useful with it.

Whether a refetch is in order isn't always clear. For instance, if you 
look at the keyword expansion example, a refetch may not be necessary, 
although the content was rewritten.

> force a refetch", and existing clients and proxies won't do that.

Correct. Also, PUT says servers SHOULD return 200 or 204 on a successful 
PUT.

> the PUT response.  It's ok to send a weak Etag corresponding to the

I'm not sure how you come to that conclusion. As pointed out in 
<http://greenbytes.de/tech/webdav/draft-reschke-http-etag-on-write-01.html#rfc.section.1.3>, 
"ETag" is a response header, and response headers per definition apply 
to the entity on the server, not the one in the request:

"The response-header fields allow the server to pass additional 
information about the response which cannot be placed in the 
Status-Line. These header fields give information about the server and 
about further access to the resource identified by the Request-URI." 
(<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.6.2>).

> Useless (except in pathological cases), but ok.

We discussed this last year over here, and the consensus was that if an 
ETag is returned in PUT (or another write method such as PROPPATCH), it 
applies to what the server has, not what the client sent. I was 
surprised about that as well, and I really think a clarification of 
RFC2616 is required here. See also 
<http://greenbytes.de/tech/webdav/draft-reschke-http-etag-on-write-01.html#rfc.section.3.p.7>.

Best regards, Julian

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 14 Sep 2006 12:41:54 +0200

> The usage in that article should use a weak Etag, not a strong one.

<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.13.3.3> 
disallows usage of weak validators for anything except GET without 
byte-range, so it can't be used with PUT:

"Clients MAY issue simple (non-subrange) GET requests with either weak 
validators or strong validators. Clients MUST NOT use weak validators in 
other forms of request."

See also <http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.14.24>:

"A server MUST use the strong comparison function (see Section 13.3.3) 
to compare the entity tags in If-Match."

> and the file being edited will get really messed up.

Well, the draft also says that clients MUST NOT assume octet-by-octet 
identity unless the server indicates this using the "Entity-Transform" 
header. As far as I can tell, this is backwards compatible with RFC2616.

> disables proxy caching entirely even when that is unnecessary.

My interpretation is that a cache updates it's cache entry with the 
content sent with PUT would be buggy. See 
<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.9.6>:

"If the request passes through a cache and the Request-URI identifies 
one or more currently cached entities, those entries SHOULD be treated 
as stale. Responses to this method are not cacheable."

>     <http://greenbytes.de/tech/webdav/draft-reschke-http-etag-on-write-01.html#rfc.section.A.1>

Nope :-)

> it's been in the spec forever.

See above. RFC2616 doesn't allow weak etags for cache validation in 
write operations.

Best regards, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 14 Sep 2006 12:11:33 +0100

> whether they return an ETag or not.

But why not a weak Etag, along with "Cache-Control:
max-age=0,must-revalidate"?  (Or "proxy-revalidate" if you don't mind
_that particular_ client reusing the sent entity).

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 14 Sep 2006 12:26:36 +0100

> modifications, so per rfc2616,13.3.3, it is indeed a strong validator.

Firstly, what about _other_ clients using the same intermediate proxy
cache.  They'll see the PUT entity without modifications too.
Which may be fine, or it may be an unintended side effect.

Secondly, byte-range requests to refresh partially cached entities.
Those are allowed with a strong Etag (not a weak one), and they'll do
the wrong thing unless the server simply refuses them.

So a server which behaves this way should refuse byte range requests
when sent an "If-Match" with an Etag corresponding to an entity that
it rewrote, at least if the rewriting would clash with the byte ranges.

> It would be interesting to get the number of clients implementing 205.

In RFC 2616, 10.2.6 205 Reset Content, it says nothing about telling
the client to refetch the content.

Only things about document views and forms.

So even if there are clients which implement 205, there's no reason to
believe it would have the effect desired here.

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Thu, 14 Sep 2006 13:31:27 +0200

> _that particular_ client reusing the sent entity).

RFC2616 says that weak etags can't be used with PUT/If-Match. So of what 
use would that be?

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 14 Sep 2006 13:37:27 +0200

> Which may be fine, or it may be an unintended side effect.

As far as I can tell, proxies are not allowed to use the entity sent 
with PUT for updating the cache:

"If the request passes through a cache and the Request-URI identifies 
one or more currently cached entities, those entries SHOULD be treated 
as stale. Responses to this method are not cacheable." 
(<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.9.6.p.2>).

> the wrong thing unless the server simply refuses them.

No, they will do exactly the right thing, as long as the client complies 
to RFC2616 (remember that it says that the ETag is for the stored 
entity, not the submitted one).

> it rewrote, at least if the rewriting would clash with the byte ranges.

It could do that, but I don't think it's necessary for correctly working 
clients.

> believe it would have the effect desired here.

Correct. And again; returning 205 upon PUT violates a SHOULD-level 
requirement:

"If an existing resource is modified, either the 200 (OK) or 204 (No 
Content) response codes SHOULD be sent to indicate successful completion 
of the request." 
(<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.9.6.p.1>).

Best regards, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 14 Sep 2006 12:20:06 +0100

> Xythos client will eventually download the new content.

Or (near-)equivalently, the PUT response could send the Etag
"this-never-matches".

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 14 Sep 2006 12:17:21 +0100

Julian Reschke wrote:
> important, as long as the Infoset is preserved.

Even if there is a proxy cache in the network between the XCAP client
and the server, and the proxy cache does partial discard of large
entities, and byte-range requests (with If-Match) to refresh them?

I think even an XCAP client would be lost in that case.

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Thu, 14 Sep 2006 13:56:00 +0200

Roy T. Fielding schrieb:
> I agree.  As I said last time, the easier solution is to simply require 
> that
> ETag in a response to PUT means that the client can use that entity tag in
> future conditional requests (that includes IMS, If-Match, and Range-based
> conditional requests).  There is absolutely no reason for the server to 
> send

Let's be clear about that this breaks compatibility with RFC2616, and it 
also defeats real-world editing scenarios like the ones described by 
Yves Lafon and myself.

I prefer my own proposal, as it IMHO doesn't break any existing 
functionality, but only adds additional information for clients that care.

In any case, whatever the solution to this problem is should be 
described in a document that updates RFC2616, otherwise it doesn't 
really address the current problem I'm trying to solve (meaning that 
protocols using HTTP make conflicting requirements, as summarized in 
<http://greenbytes.de/tech/webdav/draft-reschke-http-etag-on-write-01.html#rfc.section.1.4>).

> OTOH, the mechanism for how the server "stores" the representation, whether
> or not it is stored byte-for-byte, and how a server might otherwise manage
> to accomplish the feat of handling a conditional request without preserving
> byte-level equality is none of our business and should never appear in an

Here I fully agree. What I'd like to see as a result of the current 
discussion is a clarification on what clients can rely on, and - 
potentially and depending on that clarification - an extension that 
gives them the additional information they want.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 14 Sep 2006 14:41:46 +0200

> I think even an XCAP client would be lost in that case.

Jamie,

again: my understanding is that proxies are not allowed to cache the 
response from a PUT, thus this isn't a problem.

Best regards, Julian

---

From: Cyrus Daboo <[email-masked]>
Date: Thu, 14 Sep 2006 09:45:10 -0400

Hi Jamie,

> client can't do anything useful with it.

Not true.

Here is my take on what information is useful to a client:

1) Server does not re-write the content at all - client should be given the 
ETag of the stored entity so that it does not have to download it in future 
if it does not get re-written by someone else.

2) Server does a non-semantic change (e.g. adds unimportant whitespace, 
re-formats, uses different XML namespace prefixes etc) - client should be 
given the ETag of the changed entity plus an indication that there was a 
non-semantic change. If the client cares about octet-equality it can then 
download the modified resource, if not it can use the ETag as above.

3) Server does significant change to the resource - client should be given 
the ETag of the new entity plus an indication that the resource was 
changed. Why is that useful? Well if the client does decide to fetch the 
re-written resource it will know, by comparing the ETag it got with what is 
actually on the resource, whether any other entity has in the meantime 
itself over-written the resource. The issue here is being able to 
distinguish between a re-write done to MY resource (the one I put) as 
opposed to a re-write done as a result of someone else's put or some other 
action. That may or may not be useful information but I see no reason not 
to expose it.

CalDAV chose to solve the case of discriminating between (1) and (2)/(3) 
together. So if a strong ETag is returned the client knows that (1) is 
true, otherwise either (2) or (3) may be the case, but there is no way to 
tell between them.

So, for Julian's current proposal, I would like to see an additional 
entity-transform-keyword to help discriminate between (2) and (3). Of 
course what constitutes a 'non-semantic' change is not at all clear, and 
will likely depend on the application and data type. So perhaps 
entity-transform-keyword just needs to be extensible, and other specs can 
extend it - the important point being that if the client sees a keyword it 
does not know about it treats it as 'unspecified'. So CalDAV could declare 
a 'CalDAV-unchanged' keyword that serves could use to indicate case (2). 
AtomPub, XCAP etc could define there own. Each spec would have to be very 
explicit as to what 'non-semantic' changes are allowed to the data.

-- 
Cyrus Daboo

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 14 Sep 2006 14:57:33 +0100

Julian Reschke wrote:
> "ETag" is a response header, and response headers per definition apply to 
> the entity on the server, not the one in the request:

> (<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.6.2>).

It says "about further access to the resource identified by the
Request-URI".  It does not say the Etag in a 200 response corresponds
to the entity which will be returned by subsequent GETs.  It says
"information [...] about further access", on which I think we all agree :)

> applies to what the server has, not what the client sent.

What do you mean by "has"? :)

I.e. is that what the server "has" before it modifies the entity, or after?

Conceptually it's fine to say the 200's Etag matches the response
before the modification, and the modification is _as if_ another
client changed the entity.

More importantly, that's what fits the caching model, and fits the
octet-equivalence that is generally implied by a strong Etag (as
described for byte-range requests).

Also, you can imagine a server which doesn't actually do the
modification at the time of the PUT.  It might do it at the time of
the next GET, instead.  The difference ought to be transparent; it's a
server implementation detail, and potentially a performance
enhancement to delay the modification and thus calculation of Etag.

After all this, though, regardless of the wording in existing specs,
practical correctness must have precedence.  There's these reasons why
PUT's 200 should not send a strong Etag corresponding to a
server-modified value:

    1. It'll break existing caches which use byte-range GET requests
       to the origin server, to refresh part of an entity when they
       kept another part.  Strong Etags imply octet equivalence for
       validating byte-range GET operations.

    2. Alternatively, the server can simply refuse byte-range requests
       in these cases, but that just means wasting bandwidth.  And
       this is not likely to be implemented uniformly.

    3. It'll also break existing proxy caches which _respond_ to
       byte-range GET requests (this is different to 1).  This
       includes "reverse-proxies" which are used to accelerate server
       clusters.

    4. It'll break clients who, for whatever reason, want to see the
       exact entity that clients using other network paths are seeing.
       (Other network paths meaning via different proxy caches).  It
       shouldn't be necessary to use "Cache-control: no-cache" for
       that; that introduces unnecessary overhead when the value could
       be cached safely.

    4. To avoid such breakage, generic data clients and proxy caches
       will end up being programmed to ignore the Etag in PUT
       responses.  This would be an unfortunate bandwidth overhead.
       (Or an unfortunate proliferation of "don't cache if the Server
       header is equal to..." heuristics).

And these lesser reasons:

    5. If you send a weak Etag with PUT responses, then all the
       caching and lost-update avoidance you wanted to do in specific
       XCAP / CalDAV clients etc. is still possible.  Those clients
       can be taught that a weak Etag means things like whitespace
       formatting (in XML) and $Id$ keywords (in revision control) may
       have changed but it doesn't matter.  And that's exactly what
       weak Etags are for!

    6. If you really want the behaviour where the PUTting client will
       continue to use the unmodified entity, without teaching the
       client about weak Etag, for a specific server application, then
       you can do this instead of sending the "after" Etag in the PUT
       response:

       Send the "before" Etag in response to PUT.  Subsequent GETs (to
       all clients) will have the "after" Etag.  Program the server to
       match _both_ the "before" and "after" Etags in
       If-Match/If-None-Match.  That gives you the same client cache
       behaviour as sending the "after" Etag in response to PUT, and
       the same updating behaviour for version control etc., but
       doesn't break the caching model for other applications (as
       described in points 1-4.

Because of points 5 and 6, there is no advantage to sending the
"after" Etag in PUT responses: you can get the same intended effects
in other ways, with no overhead.

Because of points 1-4, there are several disadvantages.

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Thu, 14 Sep 2006 15:55:39 +0200

Cyrus Daboo schrieb:
> entity-transform-keyword just needs to be extensible, and other specs 
> can extend it - the important point being that if the client sees a 
> keyword it does not know about it treats it as 'unspecified'. So CalDAV 
> could declare a 'CalDAV-unchanged' keyword that serves could use to 
> indicate case (2). AtomPub, XCAP etc could define there own. Each spec 
> would have to be very explicit as to what 'non-semantic' changes are 
> allowed to the data.

Right. That was the intent with the extensibility model defined in draft 
00 
(<http://greenbytes.de/tech/webdav/draft-reschke-http-etag-on-write-00.html#rfc.section.4>). 
I removed it in 01, because - as always - the extension point distracted 
people from the base problem. It could be simply re-added once there's 
support for the remainder of the document.

Best regards, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 14 Sep 2006 15:17:58 +0100

> to compare the entity tags in If-Match."

For If-Match in GET, that makes sense.

> as stale. Responses to this method are not cacheable."

I see!

This model is inconsistent, unfortunately.  While the above statement
applies to proxy caches, it's deliberately not being applied to the
"cached data model" inside an editing client application, for example.
While they are different things, they are conceptually very similar.

The conceptual similarity is most apparent if your editor is a web or
WebDAV browser, and you click "refresh" and "view source" hoping to
see what other people see... and find that it's different.  The
browser's internal cache may cache what it PUT, if the assumptions
being discussed in this thread about Etag were (incorrectly) assumed.

It's also quite unfortunate that PUT entities cannot be cached by
proxies (and other transparent caches), for bandwidth reasons.  If, in
the next specification, we were to say that PUT response Etags allow
the PUT entity to be stored in the cache (otherwise, the default is
that it's not allowed), that would be nice.

> See above. RFC2616 doesn't allow weak etags for cache validation in 

That's not cache validation.  It's update validation: a different
thing, whose semantics you are presumably free to define at this
point?

> to compare the entity tags in If-Match."

It does seem as though this condition on If-Match ought to be relaxed
for PUT operations to server applications which can define "weak" to
mean "sufficiently good for safe updates".

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Thu, 14 Sep 2006 16:21:05 +0200

> "information [...] about further access", on which I think we all agree :)

Well, we had that discussion last year, and back then, consensus seemed 
to be that it's for the entity that will be returned upon GET (see 
thread around 
<http://lists.w3.org/Archives/Public/ietf-http-wg/2005OctDec/0017.html>). 
Note also:

"The ETag response-header field provides the current value of the entity 
tag for the requested variant." 
(<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.14.19>)

...so the real question here, WTF does "requested variant" mean in this 
context (this is why I have a reminder in the spec to get that clarified).

> What do you mean by "has"? :)

...what it will return on GET...

> I.e. is that what the server "has" before it modifies the entity, or after?

That's not really meaningful, because the server may be rewriting 
content upon write and read... Let's stick with the language "upon GET".

> client changed the entity.

I thought so as well until last year, but was convinced otherwise (see 
above link).

> described for byte-range requests).

You keep mentioning caching, although RFC2616 is very clear about 
caching of PUT responses. Could you please clarify?

> enhancement to delay the modification and thus calculation of Etag.

Correct :-). The time of rewrite is an implementation detail.

>        validating byte-range GET operations.

No, it won't, unless these caches are non-compliant in the first place.

>        this is not likely to be implemented uniformly.

A server could do that, but I don't think it's needed.

>        includes "reverse-proxies" which are used to accelerate server

I think we won't make any progress with this issue until you can give a 
concrete example of a RFC2616-compliant intermediate that would be 
broken in this case.

>        header is equal to..." heuristics).

RFC2616 already says that the response for PUT isn't cacheable. And as 
there never has been a guarantee that there is octet-by-octet storage 
upon PUT, I still don't see the problem.

>        have changed but it doesn't matter.  And that's exactly what

Again: RFC2616 says that weak etags can't be used for authoring:

"A server MUST use the strong comparison function (see Section 13.3.3) 
to compare the entity tags in If-Match." 
(<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.14.24>).

>        doesn't break the caching model for other applications (as

That's an interesting idea on it's own; I just don't see where the 
caching model is broken at all. As far as I understand, my draft 
currently describes what RFC2616 says (as discussed last year over 
here). The fact that we seem to disagree about what RFC2616 says clearly 
indicates to me that it really needs to be clarified (one way or the other).

Best regards, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 14 Sep 2006 15:29:52 +0100

> RFC2616 says that weak etags can't be used with PUT/If-Match. So of what 

Point, I hadn't noticed.

If any part of the spec would be changed, I'd suggest that :)
Plus allowing PUT entities to be cached by proxies, iff the PUT
response has an Etag.

And I concede all your points about the meaning of the existing spec.,
and see that all clients and servers which exactly implement the
existing spec are fine, with whichever Etag may be sent in PUT
responses (before or after server modification of the entity).

However, given that the choice seems to be open still, I side with
those who reckon sending the Etag corresponding to the entity _before_
modification by the server, just because it seems more consistent with
the meaning of Etags.

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Thu, 14 Sep 2006 16:32:53 +0200

> For If-Match in GET, that makes sense.

Yes. And I agree that it may be written only with GET in mind. The 
problem here is that we don't know, and that it seems to be extremely 
hard to get consensus for a change. Thus, my attempt to (a) do apply 
minimal fixes to RFC2616 (to be acceptable by "most"), and (b) add an 
extension that is a simple as possible (to prevent bikeshed discussions).

> being discussed in this thread about Etag were (incorrectly) assumed.

Clients that cache the entity that was PUT today already will have 
potential interop problems with servers that rewrite content. There's no 
way to undo that damage. My draft just proposes a hopefully clean 
solution for future servers/client without breaking anything that was 
compliant before.

> that it's not allowed), that would be nice.

My impression is that caching in HTTP is optimized for read access. 
Anyway, the proposed header would enable that kind of cache optimization 
as well.

> thing, whose semantics you are presumably free to define at this

I think it is cache validation. This is the terminology used in RFC2616, 
whether the cache is in the client or in an intermediary.

> mean "sufficiently good for safe updates".

I absolutely agree that there are other ways to achieve the stated goal. 
The problem is that the more changes you need to make to RFC2616, the 
less likely it is to get consensus. At least I think so.

Best regards, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 14 Sep 2006 15:46:29 +0100

> response from a PUT, thus this isn't a problem.

I agree; you've quoted the relevant part of RFC2616.

But of course it should be possible, somehow.  Not least because we're
talking about applications that do cache what they PUT; the semantics
for proxies should be similar.

I can't help thinking that allowing a proxy to cache PUT when the
response contains Etag would make sense.  But that's at odds with your
proposal :)

But using another response header to tell a proxy it can cache the PUT
response would be just as good.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 14 Sep 2006 16:02:05 +0100

> explicit as to what 'non-semantic' changes are allowed to the data.

I like this.

Promising generic keywords suggest themselves: encoding (for text/*),
XML-whitespace, XML-syntax (whitespace, encoding, namespace prefix
changes, collapse empty tags, reorder attributes, generic "&#..;",
etc.), content-encoding ((re-)compression).

-- Jamie