Subject: Re: I-D ACTION:draft-reschke-http-etag-on-write-00.txt

From: Julian Reschke <[email-masked]>
Date: Wed, 09 Aug 2006 22:32:20 +0200

Hi,

we've been discussing the interpretation of ETags returned upon an HTTP 
PUT request for some time now. Jim Whitehead started work on an Internet 
Draft discussing this topic in February (see 
<http://greenbytes.de/tech/webdav/draft-whitehead-http-etag-00.html>), 
but unfortunately we didn't make any progress since.

Personally, I think that we really need a very minor clarification, plus 
a simple new feature to help clients that want to avoid a re-fetch after 
sending the content. I therefore decided to write up my own draft. It 
summarizes the situation (as RFC2616 is concerned), proposes one 
clarification to RFC2616 (as mentioned in 
<http://lists.w3.org/Archives/Public/ietf-http-wg/2006JanMar/0003.html>), 
and also proposes that minor new feature (a new response header).

Feedback appreciated. I mean it. We really should resolve this, as two 
drafts in front of the IESG already make contradicting requirements.

HTML at 
<http://greenbytes.de/tech/webdav/draft-reschke-http-etag-on-write-00.html>.

Best regards, Julian

[email-masked] schrieb:
> A New Internet-Draft is available from the on-line Internet-Drafts directories.
> 
> 
> 	Title		: The Hypertext Transfer Protocol (HTTP) Entity Tag (^ETag^) Response Header in Write Operations
> 	Author(s)	: J. Reschke
> 	Filename	: draft-reschke-http-etag-on-write-00.txt
> 	Pages		: 19
> 	Date		: 2006-8-9
> 	
>    The Hypertext Transfer Protocol (HTTP) specifies a state identifier,
>    called "Entity Tag", to be returned in the "ETag" response header.
>    However, the description of this header for write operations such as
>    PUT is incomplete, and has caused confusion among developers and
>    protocol designers, and potentially interoperability problems.
> 
>    This document explains the problem in detail and suggests both a
>    clarification for a revision to the HTTP/1.1 specification (RFC2616)
>    and a new response header, making HTTP entity tags more useful for
>    user agents that want to avoid round-trips to the server after
>    modifying a resource.
> 
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-reschke-http-etag-on-write-00.txt
> 
> To remove yourself from the I-D Announcement list, send a message to 
> [email-masked] with the word unsubscribe in the body of the message.  
> You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
> to change your subscription settings.
> 
> 
> Internet-Drafts are also available by anonymous FTP. Login with the username
> "anonymous" and a password of your e-mail address. After logging in,
> type "cd internet-drafts" and then
> 	"get draft-reschke-http-etag-on-write-00.txt".
> 
> A list of Internet-Drafts directories can be found in
> http://www.ietf.org/shadow.html 
> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
> 
> 
> Internet-Drafts can also be obtained by e-mail.
> 
> Send a message to:
> 	[email-masked].
> In the body type:
> 	"FILE /internet-drafts/draft-reschke-http-etag-on-write-00.txt".
> 	
> NOTE:	The mail server at ietf.org can return the document in
> 	MIME-encoded form by using the "mpack" utility.  To use this
> 	feature, insert the command "ENCODING mime" before the "FILE"
> 	command.  To decode the response(s), you will need "munpack" or
> 	a MIME-compliant mail reader.  Different MIME-compliant mail readers
> 	exhibit different behavior, especially when dealing with
> 	"multipart" MIME messages (i.e. documents which have been split
> 	up into multiple messages), so check your local documentation on
> 	how to manipulate these messages.
> 		
> 		
> Below is the data which will enable a MIME compliant mail reader
> implementation to automatically retrieve the ASCII version of the
> Internet-Draft.
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> I-D-Announce mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/i-d-announce

---

From: Sylvain Hellegouarch <[email-masked]>
Date: Wed, 09 Aug 2006 22:35:28 +0100

Hi Julian,

Julian Reschke a écrit :
>
> Hi,
>
> we've been discussing the interpretation of ETags returned upon an 
> HTTP PUT request for some time now. Jim Whitehead started work on an 
> Internet Draft discussing this topic in February (see 
> Personally, I think that we really need a very minor clarification, 
> plus a simple new feature to help clients that want to avoid a 
> re-fetch after sending the content. I therefore decided to write up my 
> own draft. It summarizes the situation (as RFC2616 is concerned), 
> proposes one clarification to RFC2616 (as mentioned in 
I'm not sure the Entity-Transform header is needed since you might as 
well achieve the same result by using both the status code and the 
content type of the returned entity.

Server does not modify the resource => Sends a 204 No Content
Server does modify the resource => Sends a 200 and the newly created 
entity with its new media-type.

That way the user agent knows if the resource has been modified and how.

- Sylvain

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 09 Aug 2006 16:12:08 -0700

Hi Sylvain,

> That way the user agent knows if the resource has been modified and how.

Not all clients will want to get the updated resource back from the server, 
so having the server always return it is not ideal, and in some 
environments (mobile devices) extremely bad!

-- 
Cyrus Daboo

---

From: Julian Reschke <[email-masked]>
Date: Thu, 10 Aug 2006 08:24:24 +0200

> That way the user agent knows if the resource has been modified and how.

Specifying that behavior would be incompatible with RFC2616, which 
already allows a server to rewrite the content without returning it in 
the response body.

And, as Cyrus pointed out, returning the new body is expensive, and most 
of the times completely useless (which I tried to demonstrate in 
<http://greenbytes.de/tech/webdav/draft-reschke-http-etag-on-write-00.html#simple.authoring>). 
To optimize the case where the client indeed *wants* the new content, we 
would need a new way to signal that, such as a new request header 
(feedback appreciated).

Best regards, Julian

---

From: "Sylvain Hellegouarch" <[email-masked]>
Date: Thu, 10 Aug 2006 08:09:30 +0100 (BST)

> would need a new way to signal that, such as a new request header

Both are fair points. I am still not convinced by the way Entity-Tranform
is specified though. It seems over complicated. Why not defining the
Entity-Transform header has follow:

Entity-Transform = "Entity-Transform" ":" media-type

Thus taking benefit from the existing IANA registration for what you call
token in your proposal of the header. At least in that case the user-agent
would know precisely how the server has transformed the request entity and
the impact of that extension would be minimum.
If the header was not present in the response it would mean the server did
not transform the entity.

- Sylvain

---

From: Julian Reschke <[email-masked]>
Date: Thu, 10 Aug 2006 09:21:06 +0200

> is specified though. It seems over complicated. Why not defining the

Well, that's why I was submitting a draft -- feedback is needed. I 
realize that Entity-Transform currently may violate the KISS principle, 
maybe I should go back to it's minimal form of...:

Entity-Transform    = "Entity-Transform" ":" 1#transform-info
transform-info      = "identity" | "unspecified"

...and leave everything else to future specs.

> the impact of that extension would be minimum.

Reusing media types sounds like a good idea, but I'm not sure how this 
is going to work. Could you give examples for some of the use cases 
mentioned in the document (XML Infoset preserved, HTML filtering, SVN 
keyword substitution, AtomPub...)?

> not transform the entity.

We can't do that, because it's incompatible to RFC2616 (as reasoned in 
<http://greenbytes.de/tech/webdav/draft-reschke-http-etag-on-write-00.html#rfc.section.1.3> 
-- if people feel I got this one wrong by all means speak up :-))

Best regards, Julian

---

From: "Sylvain Hellegouarch" <[email-masked]>
Date: Thu, 10 Aug 2006 08:24:15 +0100 (BST)

> transform-info      = "identity" | "unspecified"

+1

> keyword substitution, AtomPub...)?

Fair point. I had not thought this through. In that case I think your
above proposal is a good one.

- Sylvain

---

From: Julian Reschke <[email-masked]>
Date: Thu, 10 Aug 2006 09:30:04 +0200

>> transform-info      = "identity" | "unspecified"

Hmm, I think this would still need to be

transform-info      = "identity" | "unspecified" | extension
extension           = token

...so it *can* be extended later.

More feedback appreciated... Do people think it's worthwhile for a 
server to be able to specify the kind of transform?

 > ...

Best regards, Julian

---

From: Jeffrey Mogul <[email-masked]>
Date: Thu, 10 Aug 2006 10:26:42 -0700

Julian Reschke writes:

    

Yes.

I think the biggest mistake that protocol designers make is
to force endpoints to make inferences, in the name of "keeping
the protocol simple."  In fact, if you oversimplify the protocol
design, the implementations become both complex and unpredictable,
because different implementors tend to make different inferences.

If you believe there is ANY good reason why the client might want to
know what the transform is (and I think you have made that case,
in previous email) then I would strongly support a design that
allows (even "encourages") the server to provide specific information.

I'd also suggest creating the appropriate IANA registry in the
same document, rather than doing this in two I-Ds, since it
makes no sense to have an extension mechanism like this without
some plan about how to use it to maintain interoperability.

You might also want to define some of the initial registrations,
if you know what they are, rather than waiting until people
have adopted conflicting meanings for extension tokens.

-Jeff

---

From: Jeffrey Mogul <[email-masked]>
Date: Thu, 10 Aug 2006 15:34:57 -0700

I thought about this a little more.  There's a potential problem
with simply specifying the grammar as

The problem is that this (1) allows multiple transforms, and
(2) doesn't explicitly specify what order they were applied in.

But one could easily imagine that, if a server does apply
multiple transforms, they are not commutative.  For example
(not that you would ever expect this combination), "uuencode"
followed by "remove-nasty-words" is not the same as
"remove-nasty-words" followed by "uuencode".

See the language in RFC2616 for Content-Encoding as an example
for how to solve this problem:

   If multiple encodings have been applied to an entity, the content
   codings MUST be listed in the order in which they were applied.

Without such an explicit requirement, one could imagine
implementations that use other orders (such as: alphabetical,
or reverse-order).

This is another case of where an incautious over-application
of the KISS principle could lead to ambiguity and hence
complexity.

-Jeff

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 10 Aug 2006 19:20:41 -0700

On Aug 9, 2006, at 1:32 PM, Julian Reschke wrote:

> plus a simple new feature to help clients that want to avoid a re- 
> fetch after sending the content. I therefore decided to write up my  
> proposes one clarification to RFC2616 (as mentioned in <http:// 
> lists.w3.org/Archives/Public/ietf-http-wg/2006JanMar/0003.html>),  

No thanks.  A new response header will just be interpreted as an entity
header.  The easier solution is to simply require that ETag in a  
response
to PUT means that the client can use that entity tag in future  
conditional
requests.  How the server manages to accomplish that feat if it isn't
storing things octet-for-octet is none of your business.

Note that this will solve your problem without requiring that the server
become a filesystem, and is also consistent with XCAP (even though XCAP
is a really bad use of HTTP).  In spite of the overspecification in  
CALDAV,
the implementation will work fine as well for the above case.

....Roy

---

From: Julian Reschke <[email-masked]>
Date: Fri, 11 Aug 2006 08:29:56 +0200

> of the KISS principle could lead to ambiguity and hence

Jeff,

I think this is indeed an example where the KISS example should apply, 
at least for now. The purpose of the proposal is to resolve the conflict 
between various proposals for applications of HTTP, currently under IESG 
evaluation.

The nature of the problem is how the ETag can be used, not whether or 
not a server can indicate what kinds of transforms have been applied. 
Thus, the discussion about the actual contents of the Entity-Transform 
header for now is distracting us from the base problem it intends to 
solve. Thus I would prefer to discuss the format only after the base the 
document (the discussion of the problem, and that path how to resolve 
it) has some kind of consensus.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Fri, 11 Aug 2006 09:16:27 +0200

Roy T. Fielding wrote:
> header...

That's correct, although I'm not sure how bad that is in practice.

First of all, if we restrict this for usage in PUT, intermediates 
shouldn't be a problem because 
<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.9.6.p.2> says 
that responses to PUT aren't cacheable.

As far as the user agent itself is concerned, we can make sure that it 
knows about that header by making this an opt-in behavior. I thought we 
could avoid having a new request header as well, but if this is really 
required, we could do that.

> ... The easier solution is to simply require that ETag in a response
> to PUT means that the client can use that entity tag in future conditional
> requests...

The basic question I'd like to see answered is whether clients - as per 
RFC2616 - can rely on an ETag being usable in conditional byte range 
requests. I think they can't (as the ETag is for what was stored, not 
what was sent), thus the attempt to make that piece of information 
available to the client.

> ...  How the server manages to accomplish that feat if it isn't

As demonstrated in the draft, in general conditional requests will work 
just fine. Problems only can occur when the client assumes it knows the 
exact content, for instance for a PUT with byte range.

OK, here's a thought: When a server wants to return an entity tag upon 
PUT (or related messages), and a transform did occur, it can return an 
ETag that is marked as "transformed content". That ETag will be usable 
for any operation that doesn't require octet-by-octet identity. For 
instance:

C1: PUT /x HTTP/1.1
     Host: example.com
     Content-Type: text/plain

     TestOne

S1: HTTP/1.1 201 Created
     ETag: "01234-temp"

(temporary etag assigned)

C2: PUT /x HTTP/1.1
     Host: example.com
     If-Match: "01234-temp"
     Content-Type: text/plain

     TestTwo

S2: HTTP/1.1 200 OK
     ETag: "01235-temp"

(validation ok, new temporary etag assigned)

C3: PUT /x HTTP/1.1
     Host: example.com
     If-Match: "01235-temp"
     Content-Type: text/plain
     Range: bytes=5,

     Three

S3: HTTP/1.1 412 Precondition Failed
     ETag: "01235"

(validation fails because octet-by-octet equality is required, so server 
says 412 and returns the final Etag).

Would that work? And even if it does, are user agents allowed to assume 
that every server works that way (I think no, they can't, but that's the 
issue I'd like to see resolved).

(And yes, I'm aware of weak etags, but 
<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.14.24> 
requires the strong comparison for If-Match, so they don't seem to be 
usable here).

> is a really bad use of HTTP).  In spite of the overspecification in CALDAV,

Right now (and in the case of servers rewriting the content upon PUT), 
CALDAV disallows returning a strong ETag, while XCAP requires it. I 
really think that's a problem, so I think at least one of the two needs 
to be changed before it can be published on the standards track. I'm 
note sure we are in agreement here. Are we?

Best regards, Julian