Subject: Re: I-D ACTION:draft-whitehead-http-etag-00.txt

From: Julian Reschke <[email-masked]>
Date: Thu, 02 Mar 2006 09:12:03 +0100

Hi,

while working on the revision of RFC2518 (WebDAV), the WebDAV working 
group came to the conclusion that what HTTP says about returning ETag 
headers in write operations (such as PUT) is insufficient.

Jim's draft summarizes the various issues and also contains some 
thoughts about what to do, from which we'd like to start a discussion on 
  the HTTP mailing list (<mailto:[email-masked]>).

Feedback appreciated,

Julian

[email-masked] wrote:
> A New Internet-Draft is available from the on-line Internet-Drafts directories.
> 
> 
> 	Title		: Design Considerations for State Identifiers in HTTP and WebDAV
> 	Author(s)	: J. Whitehead
> 	Filename	: draft-whitehead-http-etag-00.txt
> 	Pages		: 13
> 	Date		: 2006-3-1
> 	
>    This document discusses design considerations for state identifiers
>    in the Hypertext Transfer Protocol (HTTP) and related protocols such
>    as WebDAV.
> 
> 
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-whitehead-http-etag-00.txt
> 
> To remove yourself from the I-D Announcement list, send a message to 
> [email-masked] with the word unsubscribe in the body of the message.  
> You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
> to change your subscription settings.
> 
> 
> Internet-Drafts are also available by anonymous FTP. Login with the username
> "anonymous" and a password of your e-mail address. After logging in,
> type "cd internet-drafts" and then
> 	"get draft-whitehead-http-etag-00.txt".
> 
> A list of Internet-Drafts directories can be found in
> http://www.ietf.org/shadow.html 
> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
> 
> 
> Internet-Drafts can also be obtained by e-mail.
> 
> Send a message to:
> 	[email-masked].
> In the body type:
> 	"FILE /internet-drafts/draft-whitehead-http-etag-00.txt".
> 	
> NOTE:	The mail server at ietf.org can return the document in
> 	MIME-encoded form by using the "mpack" utility.  To use this
> 	feature, insert the command "ENCODING mime" before the "FILE"
> 	command.  To decode the response(s), you will need "munpack" or
> 	a MIME-compliant mail reader.  Different MIME-compliant mail readers
> 	exhibit different behavior, especially when dealing with
> 	"multipart" MIME messages (i.e. documents which have been split
> 	up into multiple messages), so check your local documentation on
> 	how to manipulate these messages.
> 		
> 		
> Below is the data which will enable a MIME compliant mail reader
> implementation to automatically retrieve the ASCII version of the
> Internet-Draft.
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> I-D-Announce mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/i-d-announce

---

From: Larry Masinter <[email-masked]>
Date: Thu, 02 Mar 2006 15:18:20 -0800

> Jim's draft summarizes the various issues 

Looking through this for the issues, this is what I come up with:

It looks like the HTTP spec doesn't say enough about
ETag headers in 200 and 204 responses to PUT. 

And there is a question, when a HTTP server accepts a PUT
but will modify the octet stream before a subsequent GET
of whether it can return a strong ETag (presumably for
the data it has, not for what was sent).

But doing so wouldn’t be useful -- the client stored
something, but gets back a strong etag for data that it
doesn't have.

So, I'd suggest a couple of things:

(a) any server response for a successful PUT may contain
 an ETag header (200 and 204 as well as 201).
(b) If a strong ETag is returned, then the client can 
   assume that the data was stored exactly as sent.
(c) If the server modifies the data before storing it
  in a way that it cannot guarantee a byte-for-byte
  copy in a subsequent GET, it shouldn't use strong eTags.

Larry
-- 
http://larry.masinter.net

---

From: Martin Duerst <[email-masked]>
Date: Fri, 03 Mar 2006 11:19:24 +0900

A very tiny adjustment below. But since HTTP (RFC 2616) is still
at Draft Standard, couldn't this kind of clarification be added
in when going to Standard? Or is that the plan?

 >   assume that the data was stored exactly as sent.

I think that should read "If a strong ETag is returned, then the client can
assume that the data will subsequently be served exactly as sent."

How the data is stored is purely the server's business
(and (c) uses the right language).

Regards,    Martin.

---

From: Julian Reschke <[email-masked]>
Date: Fri, 03 Mar 2006 16:58:55 +0100

> in when going to Standard? Or is that the plan?

As far as I can tell, there doesn't seem to be any kind of activity of 
updating RFC2616 (sad enough). But yes, should it ever be updated, I 
would expect it to incorporate whatever this document says in the end.

> ...

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Fri, 03 Mar 2006 17:03:23 +0100

> something, but gets back a strong etag for data that it

It still could be useful. For instance, a SCM server that does keyword 
expansion upon PUT could return strong ETags, and if the client would 
use that ETag in subsequent PUT (with If-Match request header), there 
wouldn't be a problem at all.

I think we should not rule out use cases like this one.

>   copy in a subsequent GET, it shouldn't use strong eTags.

That's a simple solution, but it doesn't cover many use cases where 
servers DO rewrite content, and clients still want ETags for authoring.

Best regards, Julian

---

From: Wilfredo Sánchez Vega <[email-masked]>
Date: Fri, 3 Mar 2006 12:25:30 -0800

In section 2, under "Authoring Requirements":

   pp.2: I think we want to add to the query: "... and if so, does  
this change result in a semantically equivalent document?"

   pp.4: I'm not sure I agree that the client needs to be able to  
tell the server that it's not allowed to modify the document.  We  
need a specific use case.  Presumably, if the server is modifying the  
document, it should know something about how to safely do so.  Even  
binary formats may be reasonably editable by servers.  For example, a  
server might add JPEG metadata or ID3 tags into an MP3 file.  My  
concern would be that clients might just ask for this to play it  
safe, and thereby eliminate the flexibility this affords servers.   
For example, I bet that WebDAVfs authors would be very tempted to use  
this all the time, and then not be able to talk to servers that they  
should be able to.

	-wsv

---

From: Wilfredo Sánchez Vega <[email-masked]>
Date: Fri, 3 Mar 2006 12:29:26 -0800

And on section 5, I'm liking Jim's proposal.

   I'd rather say that a server may chose not to sent the *-State  
headers if they aren't specifically requested, but that they may  
chose to include them anyway if they wish to.  (I think Jim's trying  
to avoid forcing the server to compute something that may be  
expensive, not to restrict what a server can send to clients.)

   I think Content-Handling needs to be a bit simpler than an  
arbitrary list of possible strings.  In particular, there are three  
primary options of interest:

     1- Stored entity is octet-equivalent to the sent entity.
     2- Stored entity is semantically equivalent to the sent entity.
     3- Stored entity is not the sent entity.

   And that #2 may have some sub-states.  So perhaps values like:

     none
     equivalent
     equivalent:xml
     equivalent:encoding
     equivalent:keyword
     equivalent:encoding,keyword
     modified

   I'm not sure what the use cases are for the sub-states, though,  
but this allows a client to treat all of the equivalence cases in the  
same manner, which I think is a likely case, and is suitable for  
caching.

   Also, I think we might want to note that there may be server  
implementations where both state IDs are in lock-step with each  
other.  For example, if property storage tied to the entity storage  
(eg. my DAV server uses file metadata for property storage), then an  
update to one looks like an update to the other (they share a time  
stamp).  This is potentially less efficient, but should still work.

   Are we sure that Resource-State can't be folded into ETag and  
needs to be it's own header?

   It may be useful for the server to be able to specify that a state  
ID/ETag is temporary and expires in some number of seconds.  This may  
be a good solution to the Apache weird ETag problem.  Rather than  
issuing a weak ETag, Apache would ideally issue a temporary ETag.   
For example, "T:1/computed-etag-here" would indicate that the ETag is  
temporary and expires in 1 second.  A client could then wait one  
second and GET the latest version of the resource with a permanent ETag.

	-wsv

---

From: Wilfredo Sánchez Vega <[email-masked]>
Date: Fri, 3 Mar 2006 12:30:33 -0800

Some text for section 3, the Apache situation, is below.  Let me  
know if more info is needed here.

	-wsv

   Some HTTP servers, specifically the Apache web server, return  
ETags derived from data that includes the time stamp of the file.   
Usually the computed ETag for a given resource will be a strong  
ETag.  However, because the granularity of time stamps on files is  
discrete (in the worst cases, it is one second), if the resource was  
very recently created, this scheme produces an unreliable ETag; the  
resource may change several times within this initial time window,  
but the computed ETag would not, which violates the required  
semantics of ETags.  Therefore, during the initial time window, a  
weak ETag is computed instead.

   The effect of this is that when a resource is created, it starts  
with a weak ETag (eg. "W/Fri Feb 24 18:53:48 GMT 2006").  After the  
system clock proceeds past the initial time window, the resource has  
a strong ETag (eg. "Fri Feb 24 18:53:48 GMT 2006").

   Note that the weak ETag semantics are being abused to some degree  
here.  The server is deciding that any changes to the resource during  
the initial time window are semantically equivalent.  While the  
criteria for semantic equivalence are not defined by HTTP, this seems  
like a bit of a stretch beyond what most people would consider  
appropriate.  This does, however, have the desired effect for caching.

   Note also that after a PUT request, the ETag will almost always  
start in the weak state.  This makes the utility of the ETag returned  
on PUT rather marginal, if not useless, for authoring because it will  
almost certainly change the next time the client checks the ETag.

---

From: Larry Masinter <[email-masked]>
Date: Sun, 05 Mar 2006 21:15:46 -0800

At one time (perhaps 4 years ago?) Jim Gettys offered to spin
a version of the HTTP spec with the Errata included. I
don't know how big a job it would be.

If the server is doing rewrites of the data, the server
could return a 'weak' ETag. It's up to the server to
decide what is equivalent.

But it would be unreasonable to return a strong ETag
for a body that had never before 'crossed the wire',
and certainly an incompatible change.

Larry

---

From: Jim Gettys <[email-masked]>
Date: Mon, 06 Mar 2006 07:09:00 -0500

I did prepare a draft for full standard with all errata of the time
included, *and* submitted it, IIRC.

There were no comments at the time, and there were open dependencies, so
it could not progress.

I will not bother to prepare another until someone can tell me with a
straight face that 1) all the dependencies have made it to full
standard, and therefore it won't be a waste of time, 2) at least 2
people commit to me personally they will bother to check it for
editorial errors.

Even then, I don't know when I'd have time to prepare another draft.
			Regards,
				- Jim

---

From: Julian Reschke <[email-masked]>
Date: Mon, 06 Mar 2006 14:07:35 +0100

> standard, and therefore it won't be a waste of time, 2) at least 2

I just checked, and the various MIME RFCs are still at "draft", so I 
don't think that HTTP can progress.

On the other hand, I think a revised version would be usefuk, even if it 
needs to stay at the "draft" level.

Best regards, Julian

---

From: Jim Gettys <[email-masked]>
Date: Mon, 06 Mar 2006 09:56:50 -0500

> needs to stay at the "draft" level.

I don't know if the draft for full standard I did prepare is up on the
web anywhere or not.  It is long since expired.  The dates on some
copies I have kicking around on my laptop is December, 2003, so it was
actually a bit over 2 years ago.  It is a sad statement that the
dependencies are *still* not satisfied.

If it would be useful to people to have it on line (and it doesn't
happen to already be on line), I can dig the draft version submitted out
of my files, but as far as I know, no one has vetted it for errors, so I
don't know if it is better to have it available, or not.  

If the mailing list would like me to do so, I can dig around for it, I
can resubmit it, and if multiple people then *read and verify the
changes*, and think it is ready to recycle at draft without having to
produce another draft of the document (any nits are small enough that
the RFC editor can deal with all the changes required), then you could
get a new draft standard document.  I don't think digging up the draft
is worthwhile unless people promise to read it this time.

I personally won't submit a document like this to the IESG for action
for action *unless and until* I believe other eyes than mine have
actually read the thing. I do not take silence as assent in a case like
this, but follow the M.D. koan: "First, do no harm". 

IMHO, the feedback from several people needs to be: "I really read the
thing, compared with the errata list and mailing list mail, and found
the following typo's" from several people, verifying that I didn't screw
up.  Last time, it was deafening silence, so I did not request the IESG
to recycle the document.

Realistically, paging the right information back in to my head and
resurrecting how to produce the draft, reviewing mailing list traffic,
and doing the leg work to identify all the references that must change,
reviewing IETF process changes and their impact on the document
approaches a week's work (that's what the previous draft cost in time,
as I remember).  It is much more overhead than turning a draft when you
are actively progressing a series of drafts toward a standard.

Fundamentally, the question of me producing another draft other than the
one I prepared in 2003 is moot at the moment, and we found people to
verify it, I could not spend the time to prepare it given my work
schedule; realistically, I can't forsee having time for 18 months.  And
so long as the dependencies have not moved, I won't even consider seeing
if I could squeeze the time somehow.  

If there is a volunteer, someone else could generate a final draft if
need be. Be warned: it is of order a week's work (maybe significantly
less for someone who is actively producing drafts who is up on process
and up on the state of related documents; such a person, familiar with
HTTP, might be able to turn a draft in a day or two's effort).

It would also be good to get it converted from .doc form to XML; when I
looked last, the tools were not up to the job without a lot of hand work
that I didn't have time for then.

	Your testy editor, who likes his draft to be read in
	return for the effort of preparing it,
			Sincerely,
				- Jim Gettys

---

From: Julian Reschke <[email-masked]>
Date: Mon, 06 Mar 2006 16:07:40 +0100

> that I didn't have time for then.

An example version of RFC2616 has been available for several years now 
(<http://greenbytes.de/tech/webdav/#rfc2616>), and I would think that 
probably would be a good starting point if the modest goal is to apply 
the collected errata, and to update references.

If what's needed here is mainly a document *editor*, and there is indeed 
  consensus over here that we want that revision, then consider me a 
volunteer.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 06 Mar 2006 16:08:20 +0100

> An example version of RFC2616 has been available for several years now 

s/example/XML/

Best regards, Julian

---

From: Jim Gettys <[email-masked]>
Date: Mon, 06 Mar 2006 10:53:45 -0500

> > the collected errata, and to update references.

Said XML crashes Firefox 1.0.7.  Not a good sign.

> >  consensus over here that we want that revision, then consider me a 

I think much harder to get commitment that people will actually read and
verify a draft.  *THAT* is what failed last time, and the number of
errata since 2616 were enough that you can't trust one person to get it
right.

IIRC, the draft I did covered all errata through December 2003.  I don't
know how much has transpired since then; I've not paid attention.  There
may not be a need for another draft.

Remember, if the changes are small (e.g. could be fit on a page or two
of directions to the RFC editor, particularly if they are mostly
boilerplate process changes), another draft would not need preparation,
just resubmission of the draft I prepared as an internet draft.

Whether the draft I prepared meets current process requirements for
Internet Drafts, I also don't know; any submission has to meet those
requirements.  That is part of the overhead I was referring to, that
people who are currently producing documents know and I don't, along
with going through the mail archives mail message at a time.

The RFC editor also deals with edits for references to other documents
in the usual case, so if changes since that draft are mostly in the
references, they can be done by the RFC editor.

The fundamental process here is to: 
	o submit a draft to the Internet Drafts repository, 
	o get multiple people to *READ IT*, 
	o those people state that they actually read and verified it
	  against the eratta and mailing list *in public*, 
	o discuss any remaining issues and reach rough consensus on
	  any RFC editor instructions needed, 
	o and send mail to the area director asking for action (with
	 said directions to the RFC editor attached, for any nits
	 discovered in the reads that are to be included in a final
	 document). 
If it is more than simple instructions to the RFC editor, then you get
to prepare another draft, and repeat the cycle. Wash, rinse cycle,
repeat until all the soap is out...

I believe (under process of several years ago) there is a month minimum
for individual submissions before they can be considered for action; 2
weeks for WG submissions, but we're not a WG at this date, and I'd be
amazed if you can get multiple people to read it in a month anyway, so
the minimum length of time to the point of it being in the IESG's hands
is that period from submission to when one has multiple reads completed
by credible people.

So the first issue is finding readers, in my opinion, *THEN* you can
worry about whether another draft (and editor) needs to be found.  I
don't think the editor counts as a reader, in my book anyway.
			Regards,
				Your testy editor,
				- Jim Gettys

---

From: Julian Reschke <[email-masked]>
Date: Mon, 06 Mar 2006 17:20:31 +0100

> Said XML crashes Firefox 1.0.7.  Not a good sign.

If it crashes, submit a bug report. If it just produces warning messages 
regarding missing XSLT extensions, that's a known shortcoming of Firefox 
(see 
<http://greenbytes.de/tech/webdav/rfc2629xslt/rfc2629xslt.html#rfc.section.5.2>).

> errata since 2616 were enough that you can't trust one person to get it

I would suspect that people would prefer just to review the changes 
instead of re-reading everything, but that works best if changes are 
kept to a minimum.

To add something constructive to the discussion, I just put the draft in 
question to our site 
(<http://greenbytes.de/tech/webdav//draft-gettys-http-v11-spec-rev-00.txt>), 
and produced drafts relative to RFC2616 
(<http://greenbytes.de/tech/webdav/draft-gettys-http-v11-spec-rev-00-from-rfc2616.diff.html>).

> references, they can be done by the RFC editor.

That may be true for some changes, but certainly not for all. For 
instance, the revision of RFC2396 changes both grammar productions and 
terminology, so I wouldn't expect that just replacing the reference 
would be good enough.

Best regards, Julian

---

From: "max clark" <[email-masked]>
Date: Mon, 6 Mar 2006 09:21:30 -0800

I wrote to the moderator about removal.  "it sent a MESSAGE"  THAT THE 
EMAILS DO NOT ORRIGINATE FROM THE GROUP.
NOW I MUST FILE COMPLAINTS.
AS FAR AS I KNOW YOU ARE NOT IN MY BROADCAST LIST.  I AM UNABLE TO
REMOVE THIS FROM MY SYSTEM
CAN SOMEONE PLEASE HELP ME HERE?

THANKS,
MAX CLARK
PRINTING EQUIPMENT/INDUSTRIAL BROKER.
----- Original Message ----- 
From: "Julian Reschke" <[email-masked]>
To: <[email-masked]>
Cc: "Larry Masinter" <[email-masked]>; "'HTTP Working Group'" 
<[email-masked]>
Sent: Monday, March 06, 2006 5:07 AM
Subject: Re: I-D ACTION:draft-whitehead-http-etag-00.txt

> I just checked, and the various MIME RFCs are still at "draft", so I don't 
> think that HTTP can progress.
>
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.1.375 / Virus Database: 268.1.2/274 - Release Date: 3/3/2006
>
>

---

From: Larry Masinter <[email-masked]>
Date: Mon, 06 Mar 2006 09:37:19 -0800

It's been a while since there's been traffic, and people might
not remember that they were subscribed (or email addresses
may have been recycled).

To remove yourself from this , send a message to 

  "[email-masked]" with the word "unsubscribe"
as the subject line.

mailto:[email-masked]?subject=unsubscribe

If you look at the full email headers of a message sent
to the list, you may be able to see what email address
is actually subscribed; sometimes people have mail forwarding
services. The email must originate with a "From:" address
matching the one subscribed on the mailing list.

Larry

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 6 Mar 2006 18:07:59 +0000

On a closely related note...

When I joined this list a couple of years ago, I had a couple of
questions that were unsatisfactorily answered in RFC2616, and after
discussing them here, proposed clearer wording.

One, which is a mild kind of security hole because of differing
implementations, is whether whitespace is allowed before the colon
following a header name.  The text seems to suggest yes and no
simultaneously depending how you parse it, so I proposed something
clearer.  Apache's implementation was, after many years, changed from
"no" to "yes" ostensibly to fix a security hole, yet it's questionable
if that's better or worse because implementations aren't consistent
about their interpretation of the space, and a secure proxy (for
example) should block rather than allow it, because of that inconsistency.

Another, which I questioned and so did someone else a few months
later, was about pipelining and Expect: 100-continue.  The text on
that is a bit unclear in parts, although by deduction there's only one
valid behaviour.  I had to have it explained to me, so it seemed like
a good idea to clarify the text.

I joined the list to ask those questions and hoping to clarify the
text, if nothing else - if I was confused, you could bet I wasn't the
only confused implementor.

But how do such changes get to the errata list?  I got the impression
that the HTTP errata list was no longer accepting additions, and ran
out of time and motivation then.

-- Jamie

---

From: Larry Masinter <[email-masked]>
Date: Mon, 06 Mar 2006 10:09:19 -0800

Maybe we could evaluate whether the downlevel references from
HTTP might qualify under the guidelines of BCP 97, RFC 3967
("Clarifying when Standards Track Documents may Refer Normatively
  to Documents at a Lower Level"), since that seemed to be
the sticking point last time we looked at this (which was before
RFC 3967 was around).

Larry

---

From: Jim Gettys <[email-masked]>
Date: Mon, 06 Mar 2006 13:39:11 -0500

That's a good idea.

Still doesn't solve the problem of getting people to *read* and *verify*
the results, which is the fundamental sticking point, IMHO.  So far, no
promises from anyone I've heard.
			- Jim

---

From: "William A. Rowe, Jr." <[email-masked]>
Date: Mon, 06 Mar 2006 12:53:56 -0600

Jim,

If this can be presented as redline markup then, yes I'd be happy to, count
me in to help vet and review proposed changes.  I'd even be willing, if we
chop up the list by months, to advocate a specific span of time, say postings
across some specific six month period, to discover which have value and
summarize those posts with references.

Spending hours looking at diffs, however, is not something I have cycles for.
I'll do that in one stab at the very end of the markup review process, as it's
still a necessary evil, but not one that works well during the editing process
for the reviewers.

Yours,

Bill
Apache httpd Server PMC member

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 6 Mar 2006 11:34:03 -0800

> promises from anyone I've heard.

It isn't worth the effort.  The minimum tasks for an update include
re-specifying the grammar using the now-standard IETF BNF notation and
the RFC 3986 rules for URIs, and then split the specification into
readable drafts on message syntax, http/https URIs, caching, and
content negotiation.  I don't see that happening any time soon, since
it makes more sense to let MIME be updated first.

....Roy

---

From: Larry Masinter <[email-masked]>
Date: Tue, 07 Mar 2006 17:25:31 -0800

Things get onto the errata list when someone writes up an errata
and we get 'rough consensus' on this list. 

     http://purl.org/NET/http-errata

is currently maintained by Scott Lawrence. Scott?

I think it needs someone to write up what they think the errata
should say, and for us to get some agreement about it.

Including the text on eTag, if we're looking at clarification
of original intent rather than new protocol capabilities.

Larry

---

From: Scott Lawrence <[email-masked]>
Date: Tue, 07 Mar 2006 20:52:55 -0500

> is currently maintained by Scott Lawrence. Scott?

Yup - I'm still here :-)

-- 
Scott Lawrence
http://skrb.org/scott/