Subject: Re: [Ietf-caldav] [Fwd: draft-reschke-http-addmember-00]

From: Julian Reschke <[email-masked]>
Date: Tue, 15 Feb 2005 17:07:38 +0100

Cyrus Daboo wrote:

(following up on [email-masked])

> Do we anticipate that MKCOL will also have the same problem as PUT? i.e. 
> do we have to have an 'ADDCOLLECTION' method to take care of that case?

That's a good question that I thought about as well (but forgot to add a 
reminder for). In this version, I tried to stick with 
terminology/features of base HTTP (where MKCOL doesn't exist).

> How is a client supposed to know to use ADDMEMBER as opposed to PUT? 
> Does it have to check OPTIONS each time before it creates a resource in 
> a new part of the URI hierarchy?

If it does have a name it can try the PUT which will fail; we could 
define a simple way to find out that it's supposed to use ADDMEMBER 
instead (such as a specific precondition name for DAV:error).

If it doesn't have a name it can try ADDMEMBER right away (which will 
fail if the server can't do it).

> Is there any reason why we can't simply define a new status code for a 
> PUT (or MKCOL) that results in the resource being renamed by the server? 
> e.g.:
> 
> PUT /calendar/evt1.ics HTTP/1.1
> Host: localhost
> ...
> 
> HTTP/1.1 2xx Created and Moved
> Location: /calendar/xyz123.ics

As far as I can tell, a server doing that wouldn't implement the 
specified PUT semantics anymore.

> Yes, this would probably break existing http clients doing a PUT, but 
> those clients are going to have problems anyway since the server will 
> rename the resource from under them anyway. What it does do is provide a 
> clear indication for clients that can understand this behaviour. What is 
> more this response code can be used on any method that creates resources 
> without the need to have separate new methods for this. e.g. there are 
> now a whole host of MKxxxx methods that may need the same treatment as 
> PUT. The same probably applies to MOVE and COPY too.

I think it needs to be discussed whether you need a 
"server-defines-the-URL" variant for all these methods. If that is 
indeed the case, I'd propose to use a optional extension (RFC2774) 
through which a client could explicitly state that that behaviour is 
acceptable.

Julian

-- 
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760

---

From: Julian Reschke <[email-masked]>
Date: Tue, 15 Feb 2005 17:51:29 +0100

Cyrus Daboo wrote:
> But the PUT does not fail. It succeeds, however the server immediately 
> 'moves' the resource to a new location and the client does not know 
> about that. Alternatively the server would return a new '4xx Try 
> ADDMEMBER' response code - but I would argue my proposal for a '2xx 
> Created and Moved' is a better solution as it only needs one roundtrip.

The whole point of the proposal is to offer an alternative to what 
CalDav proposes, because (IMHO) that violates PUT semantics.

> There are actually two problems here:
> 
> 1) Servers that cannot accept arbitrary client-generated URIs on new 
> resources and need to 'rename' those to something that is internally 
> consistent with its own policies.
> 
> 2) Clients that 'dont care' about the name of the resource being created 
> - they just want it to be unique and leave it up to the server to 
> generate the name.
> 
> I thought we were only trying to solve (1), but your ADDMEMBER proposal 
> also solves (2). However, (2) is also relevant for MKxxx, MOVE and COPY 
> and we don't have those solved.

It doesn't attempt to solve those for anything other than plain 
resources, and not for namespace operations. That's why it's so simple.

> One could argue that the types of server we are talking about don't 
> implement PUT semantics anyway, by virtue of the fact that they move the 
> resource as soon as it is created without telling the client. So '2xx 
> Created and Moved' is no worse than the current state for existing 
> clients - but it is better for clients that are 'aware' of this behaviour.

Yes, but protocols should not mandate a behaviour that directly 
contradicts it's definition in the base protocol -- *even* if it applies 
only to special clients.

Best regards, Julian

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 15 Feb 2005 16:25:19 +0000

Hi Julian,

> terminology/features of base HTTP (where MKCOL doesn't exist).

I think this server behaviour will impact WebDAV too (at least in the 
CalDAV case).

But the PUT does not fail. It succeeds, however the server immediately 
'moves' the resource to a new location and the client does not know about 
that. Alternatively the server would return a new '4xx Try ADDMEMBER' 
response code - but I would argue my proposal for a '2xx Created and Moved' 
is a better solution as it only needs one roundtrip.

2) Clients that 'dont care' about the name of the resource being created - 
they just want it to be unique and leave it up to the server to generate 
the name.

I thought we were only trying to solve (1), but your ADDMEMBER proposal 
also solves (2). However, (2) is also relevant for MKxxx, MOVE and COPY and 
we don't have those solved.

One could argue that the types of server we are talking about don't 
implement PUT semantics anyway, by virtue of the fact that they move the 
resource as soon as it is created without telling the client. So '2xx 
Created and Moved' is no worse than the current state for existing clients 
- but it is better for clients that are 'aware' of this behaviour.

> "server-defines-the-URL" variant for all these methods. If that is indeed
> the case, I'd propose to use a optional extension (RFC2774) through which
> a client could explicitly state that that behaviour is acceptable.

I think it would be reasonable for a client request to contain a header 
that indicates it knows how to handle '2xx Created and Moved' which will 
then allow the server to generate that response code if it needs to.

-- 
Cyrus Daboo

---

From: Julian Reschke <[email-masked]>
Date: Tue, 15 Feb 2005 18:50:34 +0100

Cyrus Daboo wrote:
> Hi Julian,
> 
> --On February 15, 2005 5:51:29 PM +0100 Julian Reschke 
> Well, I don't see this as 'contradicting' the base protocol, rather its 
> 'extending' it to clarify behaviour that some servers (perfectly 
> legitimately) may do.

OK, I'll have to update the draft with a problem description :-).

HTTP 2616 says in Section 9.6 
(<http://greenbytes.de/tech/webdav/rfc2616.html#PUT>):

"The PUT method requests that the enclosed entity be stored under the 
supplied Request-URI. If the Request-URI refers to an already existing 
resource, the enclosed entity SHOULD be considered as a modified version 
of the one residing on the origin server. If the Request-URI does not 
point to an existing resource, and that URI is capable of being defined 
as a new resource by the requesting user agent, the origin server can 
create the resource with that URI."

That is, a server that stores the entity under a different URI than the 
one provided by the client isn't doing what the client asked for.

Therefore I propose that the client should explicitly state what it 
wants. One approach is a new method, other approaches would be to extend 
the PUT *request*.

Best regards, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 15 Feb 2005 18:31:53 +0000

> wants. One approach is a new method, other approaches would be to extend 

These clients creating entities under different URIs -- what URI do
they use in the request?

They must use a URI which identifies what the server should do with
the request, rather than identifying the URI of the new entity.

(For example, a real implementation might use the URI identifying a
particular mail folder or calendar in the request, and the request is
to create a new message "inside" the identified folder or calendar,
with a server-selected URI).

When the request URI identifies an object which has internal logic and
processes the request body -- isn't this quite well suited to POST?

And, to get the URI of the created entity -- isn't this quite well
suited to a 303 response from POST?

(For example, POST to a mail folder or calender URI could reasonably
create a new message "inside" the identified folder or calendar, if
the request entity has an appropriate MIME type).

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Tue, 15 Feb 2005 20:01:55 +0100

> the request entity has an appropriate MIME type).

All of this is correct. However, how is a generic client going to know 
that sending a POST request to that container will indeed have the 
desired semantics of "storing the entity at a server-defined URI and 
returning the URI"?

If a protocol would use POST for this, it needs an out-of-band 
communication to ensure that the server indeed will process the POST as 
desired (for instance, the Atom publishing protocol does this through 
advertising serviceURLs). The proposed new method has fixed semantics 
and thus doesn't require out-of-band information.

Best regards, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 15 Feb 2005 19:51:06 +0000

> desired semantics of "storing the entity at a server-defined URI and 

Can you give an example of such a generic client being used?

I'm finding it hard to imagine a scenario where a generic client would
be asked to do this, where the caller of the generic client wouldn't
already know what to expect.

> and thus doesn't require out-of-band information.

It does require out-of-band information if the proposal for using
OPTIONS is needed: OPTIONS is the out-of-band information.

Otherwise the client is back to making assumptions about the server's
behaviour.

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Tue, 15 Feb 2005 21:04:02 +0100

> Can you give an example of such a generic client being used?

Well, not yet. The whole point of this is that potentially 
Groupdav/Caldav/Atompub clients can go ahead and use ADDMEMBER instead 
of having to invent their own solution to the problem (which is a matter 
of fact at the time of this writing is an extended PUT for *dav and a 
atom-specific POST for atompub).

> Otherwise the client is back to making assumptions about the server's

OK, "out-of-band" probably was a bad choice of words.

Using OPTIONS and the "Allow" response header is entirely covered by 
RFC2616. Thus, if a server says "I can apply ADDMEMBER to this 
resource", and ADDMEMBER is published as standards-track IETF document, 
clients can be reasonably sure that sending an ADDMEMBER request will 
have the desired effect.

Best regards, Julian

---

From: Helge Hess <[email-masked]>
Date: Tue, 15 Feb 2005 22:43:15 +0100

On 15. Feb 2005, at 18:50 Uhr, Julian Reschke wrote:
> resource, the enclosed entity SHOULD be considered as a modified 
> version of the one residing on the origin server. If the Request-URI 
> does not point to an existing resource, and that URI is capable of 
> being defined as a new resource by the requesting user agent, the 
> origin server can create the resource with that URI."
>
> That is, a server that stores the entity under a different URI than 
> the one provided by the client isn't doing what the client asked for.
>
> wants. One approach is a new method, other approaches would be to 
> extend the PUT *request*.

I can't resist but to post it once again, sorry about that ;-):

http://greenbytes.de/tech/webdav/rfc2616.html#status.201
---snip---
10.2.2 201 Created

The request has been fulfilled and resulted in a new resource being 
created. The newly created resource can be referenced by the URI(s) 
returned in the entity of the response, with the most specific URI for 
the resource given by a Location header field.
...
---snap---

I still have the opinion that PUT+201+location+if-none-match as a way 
to create new items is in the core HTTP spec and that we don't need a 
new method nor a PUT extension for that.

IMHO the only thing which is really missing is a review on which WebDAV 
clients (few other support PUT?) have issues with a location header (or 
not).

best regards,
   Helge
-- 
http://docs.opengroupware.org/Members/helge/
OpenGroupware.org

---

From: Roy T. Fielding <[email-masked]>
Date: Tue, 15 Feb 2005 15:02:46 -0800

This feature of HTTP is already defined as

   POST + media-type ==> 201 + location

There is no need for any new methods, header fields, or any of
the other assorted proposals to add more crap to HTTP.  The client
knows what it is doing and can tell from the server's response
whether or not it succeeded.  The media type is more than
sufficient to distinguish this action from any other type of
misdirected POST, and thus there is no need to check ahead of
time whether or not the feature is supported.

This has been the original definition of POST going back to 1992.
It is based on NNTP's post command and has the property of adding
resources to a collection under a name chosen by the receiving
server.

Cheers,

Roy T. Fielding                            <http://roy.gbiv.com/>
Chief Scientist, Day Software              <http://www.day.com/>

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 15 Feb 2005 23:30:56 +0000

> time whether or not the feature is supported.

I agree 100%.  Especially about the media type.

As if anyone needs a bit more clarity, RFC2616 says, under POST:

   If a resource has been created on the origin server, the response
   SHOULD be 201 (Created) and contain an entity which describes the
   status of the request and refers to the new resource, and a Location
   header (see section 14.30).

> resources to a collection under a name chosen by the receiving

Absolutely.  Thank you for stating it clearly.

-- Jamie

---

From: Mark Baker <[email-masked]>
Date: Wed, 16 Feb 2005 07:53:42 -0500

>   POST + media-type ==> 201 + location

I was thinking the same thing Roy, but a new method would have the
advantage of improved visibility; an intermediary observing a POST/201
interaction wouldn't see anything in the POST request which licensed it
to interpret that request as an attempt to store state, whereas a new
method would provide exactly that license.

Had HTTP a means for extending a method to declare this additional
expectation (as described in the draft's A.3 using RFC 2774), I agree
that POST + extension would be appropriate.

> The media type is more than
> misdirected POST,

How so?  AFAICT, the media type could be anything.  Did I miss something
in the I-D?  But even if there was such a limitation, that seems a bit
kludgy to me; could such an assumption be expected to hold for even the
foreseeable future?  I wouldn't have thought so.

Cheers,

Mark.
-- 
Mark Baker.   Ottawa, Ontario, CANADA.        http://www.markbaker.ca

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 16 Feb 2005 14:53:25 +0000

> method would provide exactly that license.

What kind of POST does -not- involve an attempt to store state?
RFC 2616:

   POST is designed to allow a uniform method to cover the following
   functions:

      - Annotation of existing resources;
      - Posting a message to a bulletin board, newsgroup, mailing list,
        or similar group of articles;
      - Providing a block of data, such as the result of submitting a
        form, to a data-handling process;
      - Extending a database through an append operation.

I'm trying to imagine what you think an intermediary (or generic
client) would do with the proposed method differently from what it
would do with POST.  They are both an attempt to modify state on the
server, and both potentially result in a created resource with a new URI.

I don't see how the intermediary is supposed to treat them
differently, so can you give an example?

-- Jamie

---

From: Roy T. Fielding <[email-masked]>
Date: Wed, 16 Feb 2005 17:45:46 -0800

> method would provide exactly that license.

POST is always an attempt to store state.  It is the response that
tells you whether the state has been stored.  No POST can be
interpreted by intermediaries, just as no "ADDMEMBER" can either.
The system gains nothing from adding a new method.  It might have
gained something by not allowing HTML forms to use POST for safe
actions, but that debate ended years ago.  Creating new unsafe
actions that do the same as what POST has already been defined
to do is absurd.

> that POST + extension would be appropriate.

It doesn't because it never needed it.

> How so?

Because the target URI knows what type of collection it is and
how to process POST requests.

>   AFAICT, the media type could be anything.  Did I miss something
> in the I-D?

The I-D is irrelevant -- the feature already exists in RFC 2616
and all predecessors.

> But even if there was such a limitation, that seems a bit

The media type is not limited in HTTP and it is not equivalent
to data format.  The media type tells the recipient how to process
the message given the method semantics.  That is why sending the
same data as "text/html" is functionally different from sending it
as "text/plain" -- the two messages have different semantics.

That will hold for anything compliant with HTTP/1.1.

....Roy

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 14:54:38 +0100

> actions that do the same as what POST has already been defined

Well, ADDMEMBER does not do the same thing. It's specced to do *one* 
very specific thing, while POST basically allows the server to do 
anything it likes.

> It doesn't because it never needed it.

OK, let's step back and look at the situation that caused me to make 
this proposal.

Two separate communities (CalDAV/GroupDav and Atompub) encountered the 
same issue: what's the best way to create a new resource on an HTTP 
server when I can't use PUT as the URI space may be entirely 
server-controlled?

1) CalDav's approach is: use PUT to an arbitrary member URI of the 
container; then let the server automagically move it somewhere else, and 
report that in the Location response header 
(<http://greenbytes.de/tech/webdav/draft-reschke-http-addmember-00.html#rfc.section.A.2>).

2) Atom's approach is to use service URIs to which the client can POST, 
and to discover these service URIs by parting entity (feed) bodies.

Somehow I have the (perhaps naive) wish that identical requirements 
should lead to common protocol constructs.

1) Seems to break PUT semantics;

2) Is specific to Atom and not applicable to other protocols.

> That will hold for anything compliant with HTTP/1.1.

So what would be your recommendation to the authors of CalDav and the 
Atom Publishing Protocol? What's the best way to add a new member to a 
container (storing the entity just like PUT, but letting the server 
decide on the URI)?

Best regards, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 17 Feb 2005 14:36:06 +0000

...
> 2) Is specific to Atom and not applicable to other protocols.

ADDMEMBER is no different from POST in this respect:

You still have to "discover the service URI" to use with ADDMEMBER.

-- JAmie

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 15:45:34 +0100

> You still have to "discover the service URI" to use with ADDMEMBER.

No, it would be the container resource itself. For CalDav, the calendar 
collection; for Atompub, the feed resource itself.

Best regards, Julian

---

From: Joe Gregorio <[email-masked]>
Date: Thu, 17 Feb 2005 15:27:27 +0000

> and to discover these service URIs by parting entity (feed) bodies.

To be clear, Atom uses POST + media-type ==> 201 + location exactly
as Roy outlined. 

As you point out, there is an issue with the discovery of the URI to POST
to, but I'll get back to that below.

> 2) Is specific to Atom and not applicable to other protocols.

None of what Atom does is specific to Atom, unless you count
the media-type of "application/atom+xml".

The real source of the problem/disconnect here seems to be
not over the semantics of PUT or POST, but goes back a level
further into the differences betweeen Atom and WebDAV and how 
they use URIs, which in turn may foster different solutions
to the problem of resource creation. 

To make this clear, let's define two terms:

URI Construction:
   An algorithm for pasting together some information into 
   URI path and query parameters to form a URI.

   Two examples:
    
   1. WebDAV's use of PUT. To create a new resource append 
       to the path your are currently using and PUT a representation 
       to that URI.
   2. Del.icio.us API (http://del.icio.us/doc/api) Ignoring for now
the non-idenpotent
       uses of GET, the API is just a recipe for how to construct URIs.

Hypertext Navigation:
   Following opaque URIs in a hypertext document.

   Example:
     The <a/> element in HTML.

Now we can categorize:

   Atom:        All URIs are opaque and it uses Hypertext Navigation
to find URIs.
   WebDAV:  In many instances uses URI Construction.
   HTML:       Uses a blend of both. For example <a/> elements are pure 
                   Hypertext Navigation, while HTML Forms that 
                   use GET use a combination, creating a URI from data in 
                   the Hypertext to construct a URI.

As a side note, what does seem to be missing is a generic mechanism
for URI construction using data culled from hypertext that can be used
for web services. That is, while HTML can construct Form GET requests
using query parameters there is no analogue for constructing path and
query components of a URI from XML documents.

    -joe

-- 
Joe Gregorio        http://bitworking.org

---

From: Cyrus Daboo <[email-masked]>
Date: Thu, 17 Feb 2005 16:05:26 +0000

Hi Julian,

> (<http://greenbytes.de/tech/webdav/draft-reschke-http-addmember-00.html#r

I want to add that in the CalDAV case this URI naming restriction will also 
apply to COPY and MOVE methods too, not just PUT. So we do need to come up 
with a solution that at least works with those methods too. Of course it is 
possible to emulate COPY and MOVE using PUT (client downloads original 
data, creates it in new location) and DELETE (remove old data in the case 
of MOVE) - but we should really have the proper atomic operations for this.

-- 
Cyrus Daboo

---

From: Stefan Eissing <[email-masked]>
Date: Thu, 17 Feb 2005 17:34:02 +0100

Am 17.02.2005 um 17:05 schrieb Cyrus Daboo:
>> (<http://greenbytes.de/tech/webdav/draft-reschke-http-addmember 
>> -00.html#r
>> fc.section.A.2>).
>
> I want to add that in the CalDAV case this URI naming restriction will  
> also apply to COPY and MOVE methods too, not just PUT. So we do need  
> to come up with a solution that at least works with those methods too.  
> Of course it is possible to emulate COPY and MOVE using PUT (client  
> downloads original data, creates it in new location) and DELETE  
> (remove old data in the case of MOVE) - but we should really have the  
> proper atomic operations for this.

It seems to me that however you solve the PUT/ADDMEMBER/POST problem,  
it can be used to solve the MOVE and COPY scenarios as well, at least  
for single, plain resources:

PUT/ADDMEMBER/POST a new, empty resource -> 201 with Location
COPY/MOVE to that new Location

Would this work?

//Stefan

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 17 Feb 2005 16:41:42 +0000

> collection; for Atompub, the feed resource itself.

Why can't you POST to the container resource?

That's what POST is for, after all.

-- Jamie

---

From: Cyrus Daboo <[email-masked]>
Date: Thu, 17 Feb 2005 16:49:30 +0000

Hi Stefan,

--On February 17, 2005 5:34:02 PM +0100 Stefan Eissing 
<[email-masked]> wrote:

> It seems to me that however you solve the PUT/ADDMEMBER/POST problem,  it
> can be used to solve the MOVE and COPY scenarios as well, at least  for
> single, plain resources:
>

Well, it may be that the server requires some form of valid data to be 
stored in some location. i.e. a calendar server may refuse to create an 
empty resource as there has to be some valid calendar data there. That 
means having to create a 'dummy' resource containing valid but not 'useful' 
data, then overwrite that with COPY/MOVE. Of course that is ugly, but then 
we are dealing with an ugly problem anyway, so maybe that is the best we 
can do.

-- 
Cyrus Daboo

---

From: Cyrus Daboo <[email-masked]>
Date: Thu, 17 Feb 2005 17:03:53 +0000

Hi Jamie,

> That's what POST is for, after all.

The WebDAV rfc has the following statement in it in Section 5.3 as a 
justification for creating a new method (MKCOL in this case) rather than 
using a special POST operation:

>    While the POST method is sufficiently open-ended that a "create a
>    collection" POST command could be constructed, this is undesirable
>    because it would be difficult to separate access control for
>    collection creation from other uses of POST.

Wouldn't the same issue be relevant here?

Interestingly the current WebDAV ACL document does not appear to mention 
POST at all - even in the 'Normative' Method Privilege Table in Appendix B. 
Is there a reason for that?

Whatever solution we come up with it ideally needs to work seamlessly with 
WebDAV ACL.

-- 
Cyrus Daboo

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 18:18:25 +0100

> That's what POST is for, after all.

Because in general, I will have no idea what the server will do with the 
POST.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 18:19:35 +0100

Cyrus Daboo wrote:
> Hi Jamie,
> 
> --On February 17, 2005 4:41:42 PM +0000 Jamie Lokier 
> <[email-masked]> wrote:
> 
> POST at all - even in the 'Normative' Method Privilege Table in Appendix 
> B. Is there a reason for that?

Yes. As you can do anything with POST, it's impossible to state what 
privileges you'll need.

> Whatever solution we come up with it ideally needs to work seamlessly 
> with WebDAV ACL.

That would be preferrable.

Best regards, Julian

---

From: Jan Algermissen <[email-masked]>
Date: Thu, 17 Feb 2005 19:06:06 +0100

Julian Reschke wrote

> Because in general, I will have no idea what the server will do with 
> the POST.

Hmm...the meaning of POST is that the server will process the message. 
If you receive a 201,
the server has created a resource.

What is the use case that does not work with these sementics of POST but 
that would work
with ADDMEMBER?

I recall that there was a similar discussion on rest-discuss regarding 
POST vs QUERY (query
having the semantics of 'process query and send 201 to refer to the 
resource that is the query result).

Mark and Roy had a similar exchange like in this thread. I am somewhere 
in between; I see Mark's point
but I lack the in-depth understanding (especially I have no practical 
experience with the difference).

Jan

-- 
Jan Algermissen
Consultant & Programmer
http://jalgermissen.com

---

From: Scott Lawrence <[email-masked]>
Date: Thu, 17 Feb 2005 13:07:14 -0500

> >>    collection creation from other uses of POST.

That statement misses the point - it may be true that it's difficult to
express the access control based just on the method, but that doesn't
mean that it's difficult to implement appropriate access control in
either the client or the server.  The method alone does not specify the
operation - indeed, in the case of POST the full specification of the
operation is deliberately expanded to include the body mime type and the
body content.

I don't think you've shown how what you're trying to do is any different
from what POST has always done.

-- 
Scott Lawrence <[email-masked]>
http://skrb.org/scott/

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 17 Feb 2005 18:25:05 +0000

> Because in general, I will have no idea what the server will do with the 

In general, you don't know what it will do with ADDMEMBER either.

This is better illustrated with PUT.  Clearly, PUT is intended for
creating or modifying an entity at a client-specific URI.

I have the impression you expect this is literally what PUT should do:
that afterwards, GET from that URI, if permitted, will return the last
entity that was put.

Plenty of HTTP servers which don't work like that.  Often, you can PUT
to a URI ending in ".cgi" or ".php" for example, and (possibly
depending on the put'd content type, possibly not) GET will not return
the same entity; instead it will initiate processing using the
resource now present at that URI.

You can PUT to a URI ending in ".html" and with content-type
"text/html" even, but with some servers, GET from the same URI will
initiate content negotiation and may return that entity, a different
one, a transcoded one, a tidied one, etc., according to server-defined
rules.  (Section 5.4 of WebDAV, RFC 2518 has more to say on this).

Given that PUT has somewhat open-ended semantics - won't ADDMEMBER
have similarly open-ended semantics?

How is an ADDMEMBER whose effect is as open-ended as PUT, but which
creates a resource at a server-selected URI, any different from POST -
even for access control?

It seems to me that if you're considering making access control
decisions on the difference between ADDMEMBER and POST, then you
should really be making those decisions on the known behaviour of the
request URI.  If you don't recognise the request URI, then you don't
really know what ADDMEMMER _or_ POST will do, in general.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 17 Feb 2005 18:38:09 +0000

Julian Reschke wrote:
> But *what* does it contain? In general, I can not assume it will store 
> the entity as sent (like PUT).
                      ^^^^^^^^

You cannot assume PUT will store the entity as sent either.

Read section 5.4 "Source Resources and Output Resources" in RFC 2518, WebDAV.
(That part's not specific to WebDAV; it's a just a good explanation).

>From the client's point of view, once you have PUT an entity, in
general you have no way to retrieve that exact entity.  Many servers
will process the resource you created to produce the response to GET,
instead of merely returning the entity you sent earlier.

Because you can't necessarily retrieve the entity sent, you can't make
assumptions about what the server stores.  It might store only part of
what you sent, or a transformed version.  What it actually stores --
and the side effects of doing so -- are quite open.

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 19:21:06 +0100

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 19:39:12 +0100

> from what POST has always done.

It's a aubset with well-defined semantics. I consider this a feature.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 19:42:56 +0100

> In general, you don't know what it will do with ADDMEMBER either.

I do, because I'll make the assumption that a server that advertises 
support for ADDMEMBER indeed implements what the spec says. Are you 
saying that I can't make that assumption?

> that afterwards, GET from that URI, if permitted, will return the last

Yes.

> resource now present at that URI.

I'd consider that a bug.

> rules.  (Section 5.4 of WebDAV, RFC 2518 has more to say on this).

In which case the server shouldn't have supported PUT on that URI in the 
first place.

> have similarly open-ended semantics?

I don't agree that it has open-ended semantics. But it's good that you 
raise this issue, because it's essential to understand the rational for 
ADDMEMBER.

> really know what ADDMEMMER _or_ POST will do, in general.

See above.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 19:47:53 +0100

> (That part's not specific to WebDAV; it's a just a good explanation).

In fact, it says:

"However, if remote editing of the source resource(s) is desired, the 
source resource(s) should be given a location in the URI namespace. This 
source location should not be one of the locations at which the 
generated output is retrievable, since in general it is impossible for 
the server to differentiate requests for source resources from requests 
for process output resources."

> and the side effects of doing so -- are quite open.

You keep saying this, but it would be nice if you could point out where 
that kind of behaviour is described. RFC2616 says:

supplied Request-URI."

And that's certainly what today's WebDAV servers implement and WebDAV 
clients do expect.

Best regards, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 17 Feb 2005 19:51:13 +0000

> saying that I can't make that assumption?

You can define the spec that way of course.  It might even be a good idea. :)

But then I'd argue it is not the same as "PUT except that the resource
is created as a server-selected URI, using the request URI to indicate
which container".

> >that afterwards, GET from that URI, if permitted, will return the last

To double check, I just tried it on my Apache 2 server with CGI
execution enabled.  PUT to a .cgi URI overwrites a CGI script; GET
runs it and returns the parsed output.

> >resource now present at that URI.

Maybe it is a bug (but RFC 2616 doesn't have a SHOULD or MUST, it just
says "the PUT method requests...").

But it exists in practice, so it would be foolish for an intermediary
doing access control to blindly _assume_ the behaviour you expect
based on the method name alone.

And if the intermediary has to look at other things like the server
name and URI, then POST is adequately specified for this because you
can simply define POST to a _known_ URI has having that behaviour.

> In which case the server shouldn't have supported PUT on that URI in the 

Apache 2 does (assuming authorization); it's what I expected, and
it's occasionally even useful.

> raise this issue, because it's essential to understand the rational for 

Yes.  It seems you want an ADDMEMBER which is more tightly specified
than just "PUT with server-selected URI".

In which case, maybe what you really want is a more tightly specified
alternative to PUT with an option for server-selected URI.

-- Jamie

---

From: Scott Lawrence <[email-masked]>
Date: Thu, 17 Feb 2005 15:00:49 -0500

> It's a aubset with well-defined semantics. I consider this a feature.

But you could just as easily and precisely define those semantics by
using POST and defining the mime type and operations it supports.

You won't get caught be firewalls and proxy servers that think they know
better about what methods are legitimate (which you most assuredly will
if you create a new method - ask the WebDav implementors), and you won't
have changed the semantics of the method at all.

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 17 Feb 2005 20:10:29 +0000

> for process output resources."

Indeed, it's not a requirement though, and people do actually use PUT
to update source resources using the output URI space.

> that kind of behaviour is described.

I don't know if it is explicitly described.  Section 5.4 of WebDAV
seems to come closest.

> RFC2616 says:

Yes, it's a request, not a SHOULD or MUST.  It also doesn't say what
the server will actually do with the entity when it's stored URI.

As a test, I just tried it on my Apache 2 server with CGI execution
enabled.  PUT to a .cgi URI overwrites a CGI script; GET runs it and
returns the parsed output.

That's a fairly standard configuration, and I think that mode of
operation is quite widely used too (updating a web site with scripts
using PUT).

> And that's certainly what today's WebDAV servers implement and WebDAV 

Sure, but forget about WebDAV for a moment, I only brought up that
document because of the nice text on source and output resources.

Real servers do sometimes store the sent entity using PUT, but return
a processed output from GET at the same URI -- and people do actually
use that feature sometimes.

And so you can't have an intermediary _assume_ PUT will stick to the
behaviour you think it should - even if we had a consensus that it
should.

And so, for ADDMEMBER to be useful, you need to specify it more
tightly than "PUT except the created resource URI is selected
differently".

If you do that, it makes sense.  If you don't, then ADDMEMBER is no
different from POST.

That said, even if you do specify ADDMEMBER more tightly, I still
don't see how the difference would benefit you.  You say it could be
useful to an intermediary, but can you give a plausible example?

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 21:07:15 +0100

> using POST and defining the mime type and operations it supports.

In which case I couldn't use the content-type of my actual request body 
for the Content-Type request header, right?

> have changed the semantics of the method at all.

I am one of these WebDAV implementors, thanks. I haven't had any issues 
with issues for a long time.

Best regards, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 17 Feb 2005 20:12:10 +0000

> have changed the semantics of the method at all.

Side note: yes, I have to use port 81 to update my web server over
WebDAV because the transparent proxy in the way on port 80 blocks DAV
methods...

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Thu, 17 Feb 2005 21:48:59 +0100

> If you do that, it makes sense.  If you don't, then ADDMEMBER is no

OK, to summarize: your point is that PUT doesn't have the strict 
semantics (that I claim it has). Let's just agree that we disagree here.

> useful to an intermediary, but can you give a plausible example?

Did I?

Best regards, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 17 Feb 2005 22:49:39 +0000

> semantics (that I claim it has). Let's just agree that we disagree here.

Actually I'm saying it isn't implemented with strict semantics in practice.
I hope we don't disagree about that ;)

Which means there is no point pretending it does have strict semantics
for the purpose of access control to an unknown server.

> >useful to an intermediary, but can you give a plausible example?

Ah.  It wasn't you who mentioned intermediaries needing to know what
the method will do.

> Because in general, I will have no idea what the server will do with the 

which I took to mean you wanted to be sure of what would happen,
perhaps with a view to a security policy or intermediary behaviour.
My mistake.

So I'll humbly rephrase my question:

Even if you do specify ADDMEMBER more tightly (than POST), I still
don't see how the difference would benefit you.  You say it is useful
to have an idea what the server will do with a POST, in general.  In
general means an arbitrary URI with unknown properties, because if
it's a URI with known properties, you know exactly what POST will do.

How does specifying ADDMEMBER to have tighter semantics than PUT and
POST benefit you when sending the request to an arbitrary URI with
unknown properties?  Can you give an example?

Thanks,
-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Fri, 18 Feb 2005 00:05:56 +0100

> unknown properties?  Can you give an example?

The same way as COPY, LOCK, MOVE, VERSION-CONTROL, 
<insert-your-favorite-method-here>...

I don't think I understand your point. Any operation *can* be marshalled 
through POST, but that doesn't mean it's the best way to do it.

Best regards, Julian

---

From: Roy T. Fielding <[email-masked]>
Date: Thu, 17 Feb 2005 15:50:39 -0800

On Feb 17, 2005, at 5:54 AM, Julian Reschke wrote:
> container; then let the server automagically move it somewhere else,  
> and report that in the Location response header  
> -00.html#rfc.section.A.2>).

That is not allowed in HTTP.

> 2) Atom's approach is to use service URIs to which the client can  
> POST, and to discover these service URIs by parting entity (feed)  
> bodies.

That is allowed.

> 2) Is specific to Atom and not applicable to other protocols.

No, it isn't. The client must "discover" the server's URI somehow,
whether that be via hypertext, save under dialogs, or an atom link.
They are all equivalent discovery mechanisms.  When Atom does a POST
it will send a representation of an entry (which happens to be just
another resource).  When a generic HTTP client does a POST it will
send a representation of a resource.

The server receives a POST message targeting a URI and containing
a representation.  It can do one of three things: 1) deny the request
with method not allowed (or any other error); 2) treat all POSTs
as the same; or 3) decide, based on the URI alone or in combination
with the representation, how to process the request.

In all cases, the client can assume the addmember semantics of POST
without any interoperability issues.  It is the server's responsibility
to provide resource references to the client that match the actions
called for by whatever it is the client is doing, and even if the
client picks some random URI the server is more than capable of
providing an appropriate response.

Geoff asked what happens if the resource already supports POST
for some other purpose.  The answer is one of

    1) if the implementation supports the addmember purpose
       in addition to its other duties, it will return 201 Created;

    2) if the implementation does not allow subsidiary resources
       to be created, then

       a) if the implementation is written according to HTTP,
          it will return 415 Unsupported Media Type; or

       b) if the implementation is clueless, it will attempt to
          parse the media type that it expects and will fail with
          some form of 4xx response. [Note that this is the case
          whether the method is POST or ADDMEMBER -- clueless
          implementations do not check the method either.]

       c) if the implementation is totally ignorant of all things
          HTTP, it will respond with 200 and a friendly error message
          about how this site only works with MSIE 5 (again,
          regardless of the method used).

So, the answer to Geoff's question is a new method is not necessary
to distinguish one POST from another POST because this was already
considered by HTTP in 1992

   http://www.w3.org/Protocols/HTTP/Methods/Post.html

and codified in RFC 1945, 2068, and 2616.  The reason it allows
the server so much leeway in its handling of the request is because
only the resource can determine, according to its own implementation
and purpose in life, the most appropriate way to store subsidiary
resources.  ANY other specification would assume that an HTTP
server is just another form of file server like FTP (which is not
its purpose in life, as I've explained a million times before).

If the client sends random unsafe requests to a Web server,
it will obtain randomly unsafe results regardless of the method
used.  If the server provides a URI as an authorable resource,
then POST to a collection has the semantics of adding a member
to that collection -- the semantics have not changed just because
webdav chose to ignore that feature of HTTP.

....Roy

---

From: Geoffrey M Clemm <[email-masked]>
Date: Fri, 18 Feb 2005 00:10:51 -0500

My main concern is that because of the popularity of SOAP,
many/most implementations fall into category 2b (i.e., "clueless"),
so a resource will say that it supports the POST operation,
but will actually fail it with some kind of 4xx response
because it cannot parse the body, or in the worst case,
will successfully parse the body and execute some potentially
harmful SOAP operation that was never intended by the client
(the client just wanted a subsidiary whose content was that body
to be created).

I'm rather surprised that clueless (e.g., SOAP :-) implementations
would parse the body of an unknown method (e.g., ADDMEMBER)
and treat it as if it were a POST call.  I would have expected
SOAP implementations to limit themselves to parsing only POST bodies.
But Roy knows way more about that kind of thing than I do ... 

But even if a clueless implementation will try to parse the
body of an unknown method like ADDMEMBER, I assume it is very
unlikely that it would say that it supports the ADDMEMBER
method on that resource, so that at least would give the client
a way of avoiding the problem (i.e., by first asking the resource
if it supports the ADDMEMBER function, before attempting it).

Cheers,
Geoff

---

From: Mark Baker <[email-masked]>
Date: Fri, 18 Feb 2005 09:42:55 -0500

On Wed, Feb 16, 2005 at 05:45:46PM -0800, Roy T. Fielding wrote:
> POST is always an attempt to store state.

I meant "store" in the "file->save" sense of the word, where the
resulting state of the server includes *some* resource whose state is
represented by the representation included in the request.  For PUT, the
Request-URI would identify that resource, while for ADDMEMBER, the
server would mint a new resource with that state.

> how to process POST requests.

Right, but it seems to me that operation and media type should be
orthogonal.  So assuming they aren't would, at best, be limiting the
ability of the operation to be used with different media types.

> as "text/plain" -- the two messages have different semantics.

Understood.  I've championed that position before, in fact.

Mark.

---

From: Scott Lawrence <[email-masked]>
Date: Mon, 21 Feb 2005 15:31:54 -0500

> for the Content-Type request header, right?

I fear that I may have lost the thread of your comment... you cannot use
Content-Type to send anything _but_ the content type of your request
body.  

> with issues for a long time.

Then you've been living on a nicer part of the net than I do.  I run a
server that provides subversion over webdav methods, and it's a routine
item to tell people to use ssl to access it so that their proxy won't
reject the webdav methods.

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 21 Feb 2005 21:28:38 +0000

> Content-Type to send anything _but_ the content type of your request

Roy said that POST can take into account the request content type to
decide the action that is taken.

I believe Julian wants to "create resource" with an arbitrary content
type - _without_ the content type having any effect on the action that
is taken to create the resource.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 21 Feb 2005 21:32:47 +0000

>    (the client just wanted a subsidiary whose content was that body
..
>    and treat it as if it were a POST call.

Yes...  I have seen numerous CGI implementations which parse a request
as GET or POST, even if it has another method.

>    if it supports the ADDMEMBER function, before attempting it).

I think that's a reasonable assumption.

But you should always try to avoid accessing unknown resources, even
to query their capabilities:  There are implementations where sending
OPTIONS to them will be interpreted as a POST or stateful GET :)

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Mon, 21 Feb 2005 22:46:16 +0100

> is taken to create the resource.

Correct.

---

From: Roy T. Fielding <[email-masked]>
Date: Mon, 21 Feb 2005 16:34:48 -0800

>> is taken to create the resource.

General-purpose HTTP servers will not allow that for security
reasons.  They use the media type internally as an extensibility
mechanism.

....Roy

---

From: Scott Lawrence <[email-masked]>
Date: Mon, 21 Feb 2005 21:56:29 -0500

> is taken to create the resource.

But setting the content type of the resulting resource _is_ having an
effect !?!

The point that I think Roy and I have both failed to communicate is
this: POST does not in itself define what happens - the thing to which
the request is POSTed (identified by the full URL) makes that decision. 

That thing is free to use the URL, the content body, the time of day,
the phase of the moon, or whatever it wants to when deciding what to do
with the resource _and_you_get_to_choose_ because you are creating that
thing.  If you want to define how you use POST in such a way that you
say that the Content-Type of the body should be preserved in the new
resource you are adding to the server, that's fine.  If you want to post
an entity body in the request that encapsulates metadata about the
resource with the resource itself (as is done in rfc1867 form file
upload), that's fine too.  You already have what you need in POST -
defining a new method does not add any new capability - it just adds
ambiguity for the servers that don't know what it is (all intermediate
systems already know that POST is not idempotent and the response to it
can only be cached if the response so indicates - that's all you need).

---

From: Mark Baker <[email-masked]>
Date: Mon, 21 Feb 2005 22:46:55 -0500

Scott,

On Mon, Feb 21, 2005 at 09:56:29PM -0500, Scott Lawrence wrote:
> You already have what you need in POST -
> defining a new method does not add any new capability

Let me ask this; why do we have PUT when a valid effect of a POST
could also be to set the state of a resource?

IMO, it's because there are advantages to having messages which reflect
the expectations of their sender.

Mark.

---

From: Geoffrey M Clemm <[email-masked]>
Date: Mon, 21 Feb 2005 21:24:50 -0700

> OPTIONS to them will be interpreted as a POST or stateful GET :)

I'm not sure what you mean by "avoid accessing unknown resources".
The only way a client "knows" about a resource is by doing some discovery
method on it such as OPTIONS.

Cheers,
Geoff

---

From: Bernard Desruisseaux <[email-masked]>
Date: Tue, 22 Feb 2005 00:19:23 -0500

Ideally we would have a solution that would work for all methods that
may request the creation of a new resource (e.g, LOCK, PUT, COPY, MOVE,
MKCOL, MKWORKSPACE, MKACTIVITY, MKCALENDAR, ...).

Could the solution be as simple as specifying an empty query component
(e.g., "?") in the Request-URI for those methods, and have the methods
return the Location header as part of their response?

For instance,

   PUT /calendar/bernard/?

would request the server to create a new resource in the collection
/calendar/bernard/ and let the server chose the name of the resource.

Is there already a defined use for the query component for the methods
mentionned above?

Cheers,
Bernard

Cyrus Daboo wrote:

> also apply to COPY and MOVE methods too, not just PUT. So we do need to 
> come up with a solution that at least works with those methods too. Of 
> course it is possible to emulate COPY and MOVE using PUT (client 
> downloads original data, creates it in new location) and DELETE (remove 
> old data in the case of MOVE) - but we should really have the proper 
> atomic operations for this.
>

---

From: Roy T. Fielding <[email-masked]>
Date: Tue, 22 Feb 2005 00:21:05 -0800

> could also be to set the state of a resource?

Because PUT means set the state of this resource, whereas POST
does not.

> the expectations of their sender.

Umm, think about that sentence and you will find it has no content.
Messages reflect the instruction of the sender.  POST does that.

What you are really saying is that there are advantages to the
client knowing the nature of a resource, encoding that nature
into the set of methods used, and thus forcing how its state
will be impacted by each messaged action.  That is what Julian
wants in an ADDMEMBER method.  You, of all people, should
be the first to recognize that slippery slope.

Yes, it has advantages, but it also has far-reaching disadvantages
that HTTP was designed to prevent. It causes clients to become
coupled to particular "types" of resources on the server, which
is just another from of implementation-dependency.  It leads to
absolutely moronic implementations that have to test each resource
for the set of methods it implements before performing a set of
actions that would have the same result as a generic POST.
[Which is to say that it will suck every bit as much as the other
methods introduced by webdav and its prodigy that created arbitrary
resource types for collections, versions, and now calendars.]

POST, in contrast, is a generic semantic that allows the resource
to determine its implementation-specific effects.  POST to a
collection means add a member.  POST to a PUT-able resource means
an atomic append. POST to a messaging gateway means "post this".
POST to any other resource means "process this". Note, however,
that to a client they all mean the same thing: take this data
and apply it in accordance with your nature.  The client does
not need to know anything more because the client already knows
what it wants to do (and knowing more cannot help it, since
only the server is allowed to know the implementation).

PUT versus PATCH might lead you to an interesting comparison,
at least in terms of seeing when two methods do have different
semantics.  A better example is POST versus PATCH, given that
PATCHing a collection could be defined as a set of add/remove
operations on the collection state. One could easily define a
patch media type for that, but there are also good reasons not to.

POST versus ADDMEMBER, however, does not compare at all because
POST is already defined to be "add a member." POST only seems to
have multiple definitions because the other definitions are
examples of various forms of partial state change within a
resource that may consist of identifiable subcomponents. It is
really just one semantic definition that manifests in different
ways based upon how the implementer views the resource to which
the POST is applied.  A file is just an ordered collection of bytes.
A collection is just a file with a compound media type. A gateway
is just a pipe, and a pipe is just a file with scrolling-window
state over time.

The IETF could give every form of implementation a different
type and every type its own set of methods for manipulating
state, but all that will accomplish is the death of another
protocol.  And no, I don't care what SOAP does via POST,
since SOAP has already proven to be a miserable failure.

Cheers,

---

From: Mark Baker <[email-masked]>
Date: Tue, 22 Feb 2005 08:37:58 -0500

On Tue, Feb 22, 2005 at 12:21:05AM -0800, Roy T. Fielding wrote:
> client knowing the nature of a resource,
[lots of other stuff that I agree with snipped]

Ah, so you're saying that ADDMEMBER isn't uniform?  Sorry, I wasn't
able to extract that from your other messages.  But can you please
explain your reasoning behind that belief?  From my POV, the draft
defines ADDMEMBER semantics to be a small, (seemingly) uniform
adjustment upon PUT semantics.  Can you give an example of a resource
for which ADDMEMBER wouldn't make sense?

Mark.

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 22 Feb 2005 16:15:21 +0000

Geoffrey M Clemm wrote:
>    The only way a client "knows" about a resource is by doing some
>    discovery method on it such as OPTIONS.

No: the way a client knows about a resource is that something asks the
client to access the resource.

For example, a WebDAV client accesses a resource because the user
tells the client to.  The user is allowing the client to assume that
WebDAV methods are ok to try on the resource.

A CalDAV or Atom client accesses a resource that it is told to access.

In all cases, the client is told to try certain methods on certain
URLs, by the user (or other agent).

If the user asks a client to try WebDAV methods on a resource that
they don't know it's safe to try WebDAV methods on, then they can
cause all sorts of havoc.

For example, some URLs will treat any method as form submission, even
OPTIONS, so users should never ask a client which uses OPTIONS to
access those URLs, because that is dangerous.

There is simply no way for the client to "discover" whether a given
URL supports the behaviour it's been asked to do, without causing
potential harmful side effects.

This is why the client has to assume the user gave it a URL with known
behaviour in the first place.

This is why a client whose job is "add a member to a collection" may
_assume_ that it's ok to use POST for this, if POST is specified as
having the right behaviour _on that resource_.

The only technical reason why a new method would be required in HTTP
is if the message semantics were different.  For example, POST is
allowed to return 201 with Location; PUT is not.  Proxies and caches
may already understand this.  From a purely technical view (not
counting "intention") that's what makes POST appropriate.

The addition of a new method does not add anything technical, it only
expresses "intention".  Whereas the methods GET, POST, DELETE, TRACE
etc. _do_ have technical differences in terms of what HTTP messages
are expected.  That's why they exist.

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 17:36:02 +0100

> WebDAV methods are ok to try on the resource.

You're making assumptions about the knowledge a user has that IMHO are 
simply questionable. For instance, a user my use IE to follow a 
hyperlink to an Office document, and Office will (under some 
circumstances) LOCK the document, and later PUT and UNLOCK it. At least 
95% of the users will not be aware of what was going on technically.

> cause all sorts of havoc.

If that causes havoc, that's because the server is broken (be it 
misconfigured or just buggy).

See above.

> potential harmful side effects.

Jamie, lots of clients are doing this today. I'm not aware of any 
"havoc" causing this.

> having the right behaviour _on that resource_.

So what do I do if a have a resource that accepts HTML form posts, SOAP 
requests and ADDMEMBER-like semantics on the same URL?

> is if the message semantics were different.  For example, POST is

Of course. That's why they are defined to be different from POST.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 17:49:34 +0100

> reasons.  They use the media type internally as an extensibility

Yes. But I'm not speaking of general-purpose servers, but of servers 
that *do* want to allow that (just like every off-the-shelf WebDAV server).

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 18:10:16 +0100

> OPTIONS to them will be interpreted as a POST or stateful GET :)

Jamie,

I'm not sure what your point is. Of course if a server mishaves that 
way, a client accessing it may cause problems. But that's the problem of 
the server, and I don't think there's anything we can do about that when 
discussing HTTP based protocols.

What am I missing?

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 17:56:44 +0100

> the request is POSTed (identified by the full URL) makes that decision. 

Well :-) I certainly am aware of that fact. That's the *reason* of 
proposing an alternate method that has strict semantics instead.

> can only be cached if the response so indicates - that's all you need).

I am fully aware that the desired semantics *can* be achieved with a 
well-defined message used with POST. The issue is that if multiple 
protocols (such as Caldav or Atompub) want to use the same semantics, it 
would be nice if they could use the same format, so that general purpose 
servers can take advantage of that. Whether that is some specific kind 
of POST, ADDMEMBER or PUT+Redirect (already dissed by Roy :-) is a 
separate issue here.

Julian

---

From: Scott Lawrence <[email-masked]>
Date: Tue, 22 Feb 2005 13:36:34 -0500

> requests and ADDMEMBER-like semantics on the same URL?

Either:

- Implement it very very carefully.

or

- Change your server organization so that they use different URLs.

> Of course. That's why they are defined to be different from POST.

But that's just the point - from the perspective of HTTP, the proposed
ADDMEMBER is not fundamentally different from POST.

---

From: Jeffrey Mogul <[email-masked]>
Date: Tue, 22 Feb 2005 10:38:04 -0800

Jamie Lokier writes:

Let's be VERY careful about statements like this.  If Jamie means
his "should" in a normative sense, then this is flat-out wrong.

RFC2616, section 9.1.2, says:

   ... the methods OPTIONS and TRACE SHOULD NOT have side effects, ...

and section 9.2 says:

   [The OPTIONS] method allows the client to
   determine the options and/or requirements associated with a resource,
   or the capabilities of a server, without implying a resource action
   or initiating a resource retrieval.

So any implementation of HTTP (including either a server per se,
or its ancillary software!) that "treats any method as a form
submission" is manifestly non-compliant with the specification.

It is essentially impossible to specify a protocol, let alone
evolve it in a compatible way, if one cannot assume that
implementations are minimally compliant.  (Some might quibble
that the "SHOULD" in 9.1.2 gives stupid implementors some cover,
but the distinction between "SHOULD" and "MUST" in RFC2119
doesn't apply to stupid implementors.)  Without the assumption of
compliant implementations, there really is no point in even
trying to write specifications.

If Jamie is arguing that we need to add new method because
implementations are not compliant with the specs for the old
methods, that seems to be a recipe for doom.  (RFC2616 is already
encrusted with some features that were designed to get around the
fact that, prior to RFC2068, there was no formally accepted
"specification" for HTTP at all; these are the most ugly parts of
RFC2616.)

There might be other, more plausible arguments for ADDMEMBER
(although in this case, I agree fully with Roy: I haven't seen a
successful argument made yet).  But this argument against using
OPTIONS cannot be one of them.

-Jeff

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 19:52:09 +0100

> - Implement it very very carefully.

Hm. What exactly does that mean?

When POST is applied to that resource with a content type of 
"application/soap+xml", is it supposed to store the attached entity as a 
new resource (with content type "application/soap+xml", returning a 
Location response header with the URL of the new resource), or should it 
process it according to 
<http://www.w3.org/TR/2003/REC-soap12-part2-20030624>?

> ADDMEMBER is not fundamentally different from POST.

No, it's a subset, thus it can not be "fundamentelly different". That's 
on purpose.

Best regards, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 22 Feb 2005 18:56:42 +0000

Jeffrey Mogul wrote:
> methods, that seems to be a recipe for doom.

I'm arguing _against_ adding a new method.

-- Jamie

---

From: Jeffrey Mogul <[email-masked]>
Date: Tue, 22 Feb 2005 10:59:52 -0800

Sorry! ... I guess I lost track of the thread at some point!
I still don't accept the "don't use OPTIONS" line of argument.
(If I got that part right.)

-Jeff

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 19:59:41 +0100

> I'm arguing _against_ adding a new method.

:-) On the other hand, arguing against new methods by principle may be a 
recipe for doom as well.

Best regards, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 22 Feb 2005 19:02:28 +0000

> discussing HTTP based protocols.

You're missing that when you are doing CalDAV* requests, you _know_ the
URL you are accessing is CalDAV compliant already.  (* - example).

If you don't know that you have big problems.

So you don't need new methods just to express "intent".
(Maybe for other reasons, but not this one).

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 20:19:19 +0100

> That is not allowed in HTTP.

Thanks for the clarification.

>> POST, and to discover these service URIs by parting entity (feed)  

Sure. I didn't mean to imply otherwise.

>       in addition to its other duties, it will return 201 Created;

How is it (the resource) supposed to distinguish?

> webdav chose to ignore that feature of HTTP.

I still have a hard time understanding what to learn from this. It seems 
that we're now discussing "when to define new HTTP methods" which is 
definitively an interesting topic...

1) If I understand you correctly, WebDAV should have said that POST to a 
WebDAV collection resource indeed is the operation I called ADDMEMBER. 
RFC2518 says in 
<http://greenbytes.de/tech/webdav/rfc2518.html#rfc.section.8.5>:

"Since by definition the actual function performed by POST is determined 
by the server and often depends on the particular resource, the behavior 
of POST when applied to collections cannot be meaningfully modified 
because it is largely undefined. Thus the semantics of POST are 
unmodified when applied to a collection."

It would be interesting to find out how the working group came to that 
decision (I don't remember seeing that discussion when I went through 
the archives some time ago; but I may have missed it).

2) RFC2518 introduces WebDAV collections as a new resource type (with a 
distinct creation method, MKCOL). How would a client use the POST method 
(as described above) to create a new collection then? In WebDAV, the 
only difference between a collection and a non-collection resource is 
containment; thus a WebDAV collection may respond to GET with any 
content type the server likes.

3) RFC3253 (DeltaV) defines "VERSION-CONTROL", which is a well-defined 
operation that puts a resource under version control (and may be applied 
to WebDAV collections as well). Of course that operation *could* have 
been defined using POST, but assuming POST to a WebDAV collection is the 
same as ADDMEMBER, how could I possible use POST to do something else 
then? So should RFC3253 have used POST instead? If the answer is "yes", 
how should it have resolved the conflict between the two differing 
semantics that are needed? If the solution is to use different URLs for 
POST, what would have been a good way to discover these URLs?

Wondering,

Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 20:23:56 +0100

> URL you are accessing is CalDAV compliant already.  (* - example).

How does a client do *any* kind of server feature discovery if *any* 
method (including things like OPTIONS) may cause harm because of broken 
server implementations?

> If you don't know that you have big problems.

Such as?

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 20:49:50 +0100

> Messages reflect the instruction of the sender.  POST does that.

Agreed. But for POST, this only seems to work where there is a tight 
coupling between client and server; otherwise the client will have no 
clue about the server will indeed do with the request.

> is just another from of implementation-dependency.  It leads to

Well yes; let's take a look at WebDAV collections, part of the WebDAV 
requirements written down in RFC2291 
(<http://greenbytes.de/tech/webdav/rfc2291.html#rfc.section.5.8>).

By defining a way to discover WebDAV collections 
(DAV:resourcetype/DAV:collection in PROPFIND response), a client can 
find out whether a resource is a WebDAV collection or isn't. Also, 
WebDAV defines COPY and MOVE as namespace operations that take 
containment into account.

I'm not sure why this is a problem; and in particular I'd like to know 
what a better solution would have been.

> actions that would have the same result as a generic POST.

Right, those are moronic implementations (because in general, a client 
can safely try the method and then handle HTTP's "method not allowed" 
status code gracefully).

On the other hand, using distinct methods allows to use a 
discovery/failure signalling approach that uses plain HTTP (OPTIONS 
method, "Allow" response header, 405 status).

> resource types for collections, versions, and now calendars.]

Nit: Versions aren't special resource types.

> an atomic append. POST to a messaging gateway means "post this".

Why can't a collection be PUT-able?

> since SOAP has already proven to be a miserable failure.

OK, so that sounds to me like resources should fall into distinct 
categories:

(A1) Collections vs (A2) Non-Collections
(B1) Put-able vs (B2) Non-Put-able

- resources of type A1 would understand POST request as having 
"ADDMEMBER" semantics

- resources of type A2 can (within reason) do anything they like with 
POST (as long as compliant to RFC2616)

- resources of type B1 will allow PUT (as defined by RFC2616) and 
understand POST as "append" operation

Let's consider a simple WebDAV server that supports only resources of 
types A1 and B1. What's the recommendation to do authoring operations 
beyond PUT on these resources? Define new methods (such as 
VERSION-CONTROL) or mint new related URLs to which existing methods can 
be applied?

Best regards, Julian

---

From: Roy T. Fielding <[email-masked]>
Date: Tue, 22 Feb 2005 11:52:55 -0800

On Feb 22, 2005, at 5:37 AM, Mark Baker wrote:
> Ah, so you're saying that ADDMEMBER isn't uniform?

Actually, no, I was saying that ADDMEMBER as currently defined
is identical to POST.  Julian said that it wasn't identical because
his client would be able to expect one semantic, namely that a
201 result would cause the webdav collection to contain a new
member with the given media type.  That implies an additional
requirement that the target be a webdav collection, which isn't
uniform at all.

> Can you give an example of a resource

No, which is why it is POST by any other name.

....Roy

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 21:01:55 +0100

> requirement that the target be a webdav collection, which isn't

I'm not sure how you get to that conclusion. ADDMEMBER (as proposed) 
doesn't require the target resouce to be a WebDAV collection.

Best regards, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 22 Feb 2005 20:38:39 +0000

> 95% of the users will not be aware of what was going on technically.

It's not assumption about the user's knowledge.  It's an unavoidable
assumption the client must make, that it's invoked on an appropriate
kind of resource.

If a user does that, and the document came from a CGI script of the
low quality I've seen, there's a good chance the LOCK and PUT
operations will invoke unwanted side effects - or just overwrite the
script.

Of course if it's a better written web component then it won't have
those problems.  And something that generates and returns Office files
probably is better written in that respect.

> Jamie, lots of clients are doing this today. I'm not aware of any 

How often do you look at Joe Random form results and then select "edit
this page"?  If you do that with random forms from the web, I think
you'll be surprised at how many don't respond sensibly to OPTIONS,
LOCK and PUT.

The slackness doesn't cause problems because people don't do that.
(And in the cases where they do, because it's appropriate for a resource,
that resource implements a sensible response).

Similarly, how often do you expect CalDAV clients to be pointed at
non-CalDAV URLs?

If it's done, the behaviour of the server is unpredictable.  ADDMEMBER
doesn't fix that.

(Although it would shift the likelihood towards a simple failure
rather than an unwanted side effect.  Is it worth creating a new
method just to change the likelihood of a behaviour when a CalDAV
client is pointed at the wrong URL?)

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 22 Feb 2005 20:42:26 +0000

> method (including things like OPTIONS) may cause harm because of broken 

Even when you can trust OPTIONS, that's not going to tell you if it's
a CalDAV resource - only what methods are accepted.

You do feature discovery - in other words, which URL points to a
calendar - in other ways.

> >If you don't know that you have big problems.

Such as trying to treat a non-calendar resource as a calender! :)

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 21:47:43 +0100

> If it's done, the behaviour of the server is unpredictable.  ADDMEMBER

You keep saying that :-) So are you saying that defining new methods 
never ever is the right approach, because there are broken servers out 
there?

If that's the case, let's just agree that we disagree.

> client is pointed at the wrong URL?)

ADDMEMBER is a generic method. It doesn't have anything specific to do 
with CalDAV except that it *could* be used with CalDAV.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 21:49:19 +0100

> a CalDAV resource - only what methods are accepted.

I'm not interested whether it's a CalDAV resource. I'm interested in 
whether it supports ADDMEMBER.

> Such as trying to treat a non-calendar resource as a calender! :)

And the problem resulting from that would be what exactly? A CalDAV 
calendar (as currently proposed) is just a WebDAV collection with some 
specific features.

Julian

---

From: Justin Chapweske <[email-masked]>
Date: Tue, 22 Feb 2005 14:56:47 -0600

> misconfigured or just buggy).

Amen.  Allowing poor implementations to dictate the evolution of HTTP
can only lead to further inconsistencies and ambiguities which lead to
more bugs and security holes.

I'm still rather grumpy about how many dozens of hours my developers
wasted getting iTunes to work through our proxy due to the liberties
that Icecast has taken with HTTP.  I was appalled when I learned they
return "ICY 200 OK" for a valid HTTP request.

I think the PATCH proposal is good as it fills in a huge hole, but I am
much less excited about ADDMEMBER because now we have a whole new verb
that we have to understand the security ramifications and caching
semantics for.

-- 
Justin Chapweske <[email-masked]>

---

From: Roy T. Fielding <[email-masked]>
Date: Tue, 22 Feb 2005 13:01:57 -0800

> clue about the server will indeed do with the request.

The client doesn't need to know.  An HTTP client does not know the
reason why a request is being made, just as it does not know the
text inside the anchor of a hypertext link that was clicked on to
generate a GET request.  The user (agent) builds up state based on
a huge number of factors, including what is said in all past
representations received, degrees of trust, instructions that
may have been received via some other channel, configuration, etc.
That is what tells the user what to expect.  HTTP doesn't care.
What HTTP cares about is that the method is unsafe, the response
is not (by default) cacheable, a new resource has been created (201),
and it can be found at a given Location.

> containment into account.

Completely irrelevant -- COPY and MOVE are HTTP operations on any
collection, not just webdav collections, and will succeed or fail
regardless of whether the client previously checked its pacifier.

> what a better solution would have been.

There was no problem -- it was a solution to someone's angst.

> method, "Allow" response header, 405 status).

Which is completely unnecessary in HTTP.

> Nit: Versions aren't special resource types.

Sorry, I meant MKWORKSPACE.

> Why can't a collection be PUT-able?

It could have been, but MKCOL was defined instead (over my objection).

> VERSION-CONTROL) or mint new related URLs to which existing methods 
> can be applied?

Different URLs (see, for example, how the configurable name "index"
is a member of a collection and also the default response to GET
on that collection).  How a PUT or POST or any other method applied
to the collection is handled is just as configurable and may depend
on other aspects of the request (like content-type, authentication,
and more).  PUT is usually redirected to the index URL because
namespace operations are typically done one name at a time rather
than entire collections at a time, however that is dependent on the
implementation: when a client does a PUT of an XML document to a
JCR-backed implementation, it will create a collection (a document
is just one view of the collection).

Each resource has an implementation and that implementation defines
how the method impacts its state aside from what is already defined
by virtue of HTTP's method semantics.  The reason we don't need
ADDMEMBER is because POST already exists, a server must inspect
the media type already, and none of the overlapping uses of POST
are valid new members to add to a collection (soap envelopes or
www/x-form-urlencoded).  It is therefore possible to say that POST
has the add member effect on a webdav collection and be done.

....Roy

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 22 Feb 2005 21:19:08 +0000

> never ever is the right approach, because there are broken servers out 

No I'm not saying that.

One of the rationales given in this thread for ADDMEMBER is that the
tighter semantics mean that clients can depend on it literally adding
a new resource which is GETable, or failing with Method Not Supported.

I'm saying that particular rationale isn't logically sound.

It doesn't negate any other rationales you may have up your sleeve :)

> with CalDAV except that it *could* be used with CalDAV.

Indeed, but are there realistic applications in mind that would use
ADDMEMBER without caring what kind of container resource it is, but do
care that it is a container and not a form processor?

If there are it makes more sense.  If there aren't, it's unneeded.

I think your exploration of the WebDAV text concerning why POST isn't
used on containers is a good one, btw.

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 22:41:10 +0100

> I'm saying that particular rationale isn't logically sound.

ADDMEMBER is defined to do just that. If it would be accepted and become 
a standards-track RFC, yes, clients could rely on it just like they can 
rely on other HTTP methods today.

> care that it is a container and not a form processor?

Yes. For instance it could be used for Atom-Pub, and another use case 
was just suggested a few days ago internally at SAP (where a specific 
backend system has a concept of containment, but doesn't allow 
user-selected URIs).

> used on containers is a good one, btw.

I appreciate all the feedback I'm getting, and I'm very interested in 
ways that would avoid adding a new method while keeping the intended 
benefits.

Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Feb 2005 23:10:40 +0100

> regardless of whether the client previously checked its pacifier.

Yes, but that doesn't mean it's irrelevant whether a resource is a 
collection or not. It just means that you don't *have* to check before 
you apply namespace operations.

> There was no problem -- it was a solution to someone's angst.

So how *should* a WebDAV collection have been defined?

> Which is completely unnecessary in HTTP.

The 405 status and the Allow response header are unnecessary?

> Sorry, I meant MKWORKSPACE.

The alternative would have been just to use standard resources, but in 
that case RFC3253 would have had to define some new way to indicate that 
something indeed is a workspace.

> It could have been, but MKCOL was defined instead (over my objection).

There is nothing in RFC2518 that prevents collections from being 
PUT-able (however the rational for not defining PUT on collections is a 
bit weird, see 
<http://greenbytes.de/tech/webdav/rfc2518.html#rfc.section.8.7.2>).

> namespace operations are typically done one name at a time rather

...when you say "typically", what implementation are you referring to? 
Is this implemented in Apache/mod_dav?

> www/x-form-urlencoded).  It is therefore possible to say that POST

Interesting. Why wouldn't a SOAP envelope be a valid new collection 
member? And what about other potential content types resources could 
already implement for POST (that just happen to be proprietary)?

Best regards, Julian

---

From: Scott Lawrence <[email-masked]>
Date: Tue, 22 Feb 2005 17:15:57 -0500

> Hm. What exactly does that mean?

It means that that resource should have a well defined way of
determining what it is being asked to do for any given request.
Personally, I would choose the second alternative - use different URLs
for those different purposes, but...

> <http://www.w3.org/TR/2003/REC-soap12-part2-20030624>?

That W3C spec does not mandate that anything in particular be done with
a request body with that mime type.  It in no way requires that a
request body with that mime type invoke a SOAP operation or prevents a
given URL from treating that body as an object to be stored.

---

From: Mark Baker <[email-masked]>
Date: Tue, 22 Feb 2005 23:26:08 -0500

> requirement that the target be a webdav collection, which isn't

I get the DAV-collection-implies-non-uniform thing, and would be
against a revision which attributed that meaning to a 201 response.
But I'm just not seeing how ADDMEMBER==POST, sorry.

I still don't see how POST vs. ADDMEMBER is any different than POST
vs. PUT.  You previously said that PUT has "set the state of this
resource" semantics which is clearly different than POST.  IMO,
ADDMEMBER's semantics are very similar to PUT.  What (constraint?) am
I missing that suggests PUT is fine while ADDMEMBER isn't?

Mark.

---

From: Roy T. Fielding <[email-masked]>
Date: Wed, 23 Feb 2005 00:24:07 -0800

> I missing that suggests PUT is fine while ADDMEMBER isn't?

You are missing that the target of PUT is the new resource,
whereas the target of POST and ADDMEMBER are both the collection
resource.  As such, ADDMEMBER's semantics has very little in
common with PUT (almost nothing, in fact, since any unsafe
extension method can return 201 and communicate just as much,
whereas PUT is unique in what it communicates by the 200/201).

....Roy

---

From: Mark Baker <[email-masked]>
Date: Fri, 25 Feb 2005 00:35:09 -0500

> whereas PUT is unique in what it communicates by the 200/201).

In case anybody's still following this thread 8-) ... I now find myself
in agreement with Roy, that POST is appropriate and that ADDMEMBER ==
POST.

I was looking at the interaction from the POV of a tweak on PUT
semantics, rather than as the submission of a document to a
state-preserving container (which I should have recognized it as
immediately, given my previous attempts at describing such a
model[1]).

I'm still not convinced that a POST extension (which declared the
expected type of the target resource) wouldn't improve the
self-descriptiveness of the message, but practically, that's pretty
much irrelevant for the purposes of this discussion.

 [1] http://www.markbaker.ca/2001/09/draft-baker-http-resource-state-model-01.txt

Mark.