Subject: Re: i24: Requiring Allow in 405 responses

From: "Mark Baker" <[email-masked]>
Date: Mon, 4 Feb 2008 22:18:03 -0500

I'm with Roy.  The MUST is unnecessary because the response has
unambiguous semantics with or without the Allow header.  I'd prefer
"MAY" but could live with "SHOULD".

Mark.
-- 
Mark Baker.  Ottawa, Ontario, CANADA.         http://www.markbaker.ca
Coactus; Web-inspired integration strategies  http://www.coactus.com

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 4 Feb 2008 23:08:35 -0800

My thinking was that it may be necessary to preserve the MUST on the  
presence of the header (in case any software had been written to  
depend upon its presence), but to loosen the implied requirement that  
the list of headers be complete.

The text I proposed didn't really do that too well either, now that I  
look at it...

--
Mark Nottingham     http://www.mnot.net/

---

From: Henrik Nordström <[email-masked]>
Date: Tue, 05 Feb 2008 16:35:29 +0100

I meant replacing MUST by SHOULD, making the use of Allow in 405
responses a SHOULD level requirement.

This is a requirement that is in some cases impractical for servers to
implement properly. And it's also a case where it's most likely better
the sever doesn't say anything at all if it doesn't know than to try to
guess.. If it doesn't know let the client guess if it want.

But on the other hand a 405 without Allow is pretty much equivalent to a
403. So an alternative approach would be to add a note that if the
server can not provide a reliable list of allowed methods then 403
should be returned instead of 405, reserving 405 to be used only when
the server knows within rasonable doubt what methods it accepts on the
resource. And this is probably a better way to address the problem.

I do not think relaxing the meaning of Allow is a good idea. If Allow is
given then the client SHOULD assume it's the truth. Changing this would
render Allow as such pretty useless.

It's the same for Allow headers in response to GET btw. If the server
doesn't really know then there SHOULD NOT be an Allow header in the
response.

Regards
Henrik

tis 2008-02-05 klockan 06:39 -0800 skrev Mark Nottingham:
> Are you saying that s/MUST/SHOULD/ is adequate, or agreeing that  
> splitting it into two requirements, making the second a SHOULD, is  
> necessary?
> 
> 
> On 05/02/2008, at 4:47 AM, Henrik Nordström wrote:
> 
> >
> > mån 2008-02-04 klockan 23:08 -0800 skrev Mark Nottingham:
> > SHOULD is more than sufficuent for a such requirement level.
> >
> > Regards
> > Henrik
> 
> 
> --

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 5 Feb 2008 08:51:24 -0800

Fair enough. Henrik, any thoughts about SHOULD vs MAY? Roy and Mark  
both expressed a preference for MAY.

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 4 Feb 2008 18:27:18 -0800

Raised originally at <http://www.w3.org/mid/[email-masked] 
 >.

Current text:

> The response MUST include an Allow header containing a list of valid  
> methods for the requested resource.

My proposal:

The response MUST include an Allow header that SHOULD contain a list  
of valid methods for the requested resource.

Thoughts?

---

From: Henrik Nordström <[email-masked]>
Date: Tue, 05 Feb 2008 13:47:06 +0100

Regards
Henrik

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 5 Feb 2008 10:11:57 -0800

On Feb 4, 2008, at 11:08 PM, Mark Nottingham wrote:

> My thinking was that it may be necessary to preserve the MUST on  
> the presence of the header (in case any software had been written  
> to depend upon its presence), but to loosen the implied requirement  
> that the list of headers be complete.

The likelihood of real (non-testcase) software depending on Allow
being present is nil.  I can't even remember why the MUST was added
in the first place -- it certainly isn't useful for interoperability,
since a client can't very well choose from among the methods listed
as a replacement for the method attempted.  I added it in the first
pre-WG draft of HTTP/1.0 (Nov 94) as

    405 Method Not Allowed

        o  Required metainformation:  Allow

    The method specified in the Request-Line is not allowed for the
    object identified by the URI. The server should send back an Allow
    header containing a list of valid method's as explained in Section
    7.1.

which means it dates from back in the days when HTTP was expected
to be an object remoting protocol (many more methods).  Note that
this draft also had

    The methods GET and HEAD must be supported by all conforming
    HTTP/1.0 servers. The list of methods acceptable by a specific
    object can be specified in an "Allow" Object-Header (Section 7.1).
    However, the client is always notified through the return code of
    the response whether a method is currently allowed on a specific
    URI, as this can change dynamically. The set of common methods for
    HTTP/1.0 are specified below. Although this set can be easily
    expanded, additional methods cannot be assumed to share the same
    semantics for separately extended clients and servers. In order to
    maintain compatibility, the semantic definition for extension
    methods should be registered with the HTTP registration authority
    (Section 10). Servers should return the Status-Code "501 Not
    Implemented" if the method is unknown.

Even if any software does make use of the Allow field, the result
in its absence is the same -- an error has been indicated by the
server.  That is why I think including Allow should be MAY.

....Roy

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 5 Feb 2008 06:39:21 -0800

---

From: Henrik Nordström <[email-masked]>
Date: Wed, 06 Feb 2008 19:51:35 +0100

SHOULD. If the server can determine the valid list of methods it SHOULD
indicate this with Allow. This mechanism is used for enabling HTTP
extensions in the client such as WebDAV, CalDAV etc and a MAY level
requirement is too weak for this purpose as the client should not be
expected to guess that these features is available even if it MAY.

MAY level requirements is for optional features or alternative ways of
doing things, not things which should be done if you can.

SHOULD is for things which you should do, but where things will quite
likely cope even if you don't. I.e. sending the Allow header where not
sending the Allow header which in worst case makes clients capable of
using extensions not enabling these extensions.

tis 2008-02-05 klockan 08:51 -0800 skrev Mark Nottingham:
> >>>> the

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 12 Feb 2008 21:51:21 -0800

Mark, Roy, any thoughts? I'm inclined to go with SHOULD unless someone  
objects...

---

From: "Mark Baker" <[email-masked]>
Date: Wed, 13 Feb 2008 00:54:22 -0500

> Mark, Roy, any thoughts? I'm inclined to go with SHOULD unless someone

Like I said, I'm fine with SHOULD.

Mark.

---

From: Robert Sayre <[email-masked]>
Date: Wed, 13 Feb 2008 01:22:44 -0500

This is wrong, aiui. SHOULD is for things that will be harmful if not  
obeyed, but there may a possibility that your implementation knows  
better somehow. I prefer MAY.

- Rob

On Feb 13, 2008, at 12:51 AM, Mark Nottingham wrote:

>
> Mark, Roy, any thoughts? I'm inclined to go with SHOULD unless  
> someone objects...
>
>
>>>> the sever doesn't say anything at all if it doesn't know than to  
>>>> try
>>>> to

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 12 Feb 2008 22:26:36 -0800

RFC2119;
> 3. SHOULD This word, or the adjective "RECOMMENDED", mean that there  
> may exist valid reasons in particular circumstances to ignore a  
> particular item, but the full implications must be understood and  
> carefully weighed before choosing a different course.

On 12/02/2008, at 10:22 PM, Robert Sayre wrote:

> This is wrong, aiui. SHOULD is for things that will be harmful if  
> not obeyed, but there may a possibility that your implementation  
> knows better somehow. I prefer MAY.
>
> - Rob
>
>>>>> But on the other hand a 405 without Allow is pretty much  
>>>>> equivalent
>>>>> to a
>>>>>>>> depend upon its presence), but to loosen the implied  
>>>>>>>> requirement
>>>>>>>> that

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 12 Feb 2008 22:28:26 -0800

BTW, this is why I don't like linking conditional/unconditional  
compliance to SHOULD/MUST; it overloads 2119.

On 12/02/2008, at 10:26 PM, Mark Nottingham wrote:

>
>
> RFC2119;
>> 3. SHOULD This word, or the adjective "RECOMMENDED", mean that  
>> there may exist valid reasons in particular circumstances to ignore  
>> a particular item, but the full implications must be understood and  
>>>>>> the sever doesn't say anything at all if it doesn't know than  
>>>>>> to try
>>>>>> to

---

From: Robert Sayre <[email-masked]>
Date: Wed, 13 Feb 2008 02:11:58 -0500

>> carefully weighed before choosing a different course.

Sure, let's contrast this text with what we have now.

"You might help clients out if you send this, no one relies on it,  
most people don't bother."

Does not sound like something that must be "carefully weighed". I  
claim that it doesn't matter a lick, unless you're writing a WebDAV  
server or something. By all means, opt in to the MAY in that case.

  - Rob

---

From: Julian Reschke <[email-masked]>
Date: Thu, 14 Feb 2008 14:02:03 +0100

Robert Sayre wrote:
> "You might help clients out if you send this, no one relies on it, most 
> people don't bother."
> 
> Does not sound like something that must be "carefully weighed". I claim 
> that it doesn't matter a lick, unless you're writing a WebDAV server or 
> something. By all means, opt in to the MAY in that case.

Hm.

Sending an incomplete Allow header will affect clients that use it. 
WebDAV clients certainly use it to check for features, and I wouldn't be 
surprised if some AtomPub clients did as well (to check for PUT).

So it seems it *is* harmful not to send the header, or just to send an 
incomplete list.

 From a client's p.o.v., given the choice between an incorrect list and 
no list at all, I'd prefer the latter, though.

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 14 Feb 2008 14:11:33 +0100

> no list at all, I'd prefer the latter, though.

...what I said really applies to successful OPTIONS requests, not Allow 
in 405 response though...

BR, Julian

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 14 Feb 2008 10:10:23 -0800

On Feb 14, 2008, at 5:11 AM, Julian Reschke wrote:

> Julian Reschke wrote:
>> Hm.
>> Sending an incomplete Allow header will affect clients that use  
>> it. WebDAV clients certainly use it to check for features, and I  
>> wouldn't be surprised if some AtomPub clients did as well (to  
>> check for PUT).
>> So it seems it *is* harmful not to send the header, or just to  
>> send an incomplete list.
>>  From a client's p.o.v., given the choice between an incorrect  
>> list and no list at all, I'd prefer the latter, though.
>
> ...what I said really applies to successful OPTIONS requests, not  
> Allow in 405 response though...

Right -- I was going to say that yesterday but spent it on other things.

Personally, I don't care whether it is a SHOULD or a MAY, but I don't  
know
of any software that checks the 405 message's Allow field.  I do know  
that
Apache can't ensure completeness on its own because a method might be
disallowed by one part of the server (access control) even if the
resource is handled by code that might accept any method (e.g., CGI).
We would have to change the CGI interface (or restrict it to known
methods) to satisfy the original requirement.

What we might want to say is that the Allow field MUST be present if the
response comes from the origin server, and MUST NOT be present if the
method was disallowed by some intermediary without contacting the origin
server.  That does provide useful information even when incomplete.

....Roy

---

From: Henrik Nordström <[email-masked]>
Date: Fri, 15 Feb 2008 11:09:47 +0100

It's true that for the scope of the base HTTP/1.1 specifications Allow
is completely useless.

However, making it a MAY moves it out of the recommended set of
implementation, forcing every extension who want to use it to negotiate
it's precense to upgrade the requirement to a SHOULD. Quite noticeably
crippling the extension path of HTTP.

If we were talking about OPTIONS *, then I would fully agree with the
others that it's a MAY, but not GET or disallowed methods to a specific
resource. In most cases the server know pretty darn well what methods it
may handle on the resource.

I see no reason why making the requirements for 405 substantially
different than OPTIONS, just confuses implementers.

If the desire is to make Allow a MAY in general then we may just as well
remove it entirely, moving it completely down to extension
specifications, as it makes the feature completely useless from a
specifications point of view as each extension then either MUST upgrade
the requirement to a SHOULD/MUST, or invent their own negotiation
mechanism to announce the extension.

Regards
Henrik

tis 2008-02-12 klockan 23:28 -0800 skrev Mark Nottingham: 
> Thoughts? I have three people who say they prefer MAY, and Robert is  
> starting to convince me...
> 
> 
> Begin forwarded message:
> 
> > Resent-From: [email-masked]
> > From: Robert Sayre <[email-masked]>
> > Date: 12 February 2008 11:11:58 PM
> > To: Mark Nottingham <[email-masked]>
> > Cc: "Roy T. Fielding" <[email-masked]>, Mark Baker  
> > <[email-masked]>, HTTP Working Group <[email-masked]>, Henrik  
> > Nordström <[email-masked]>
> > Subject: Re: i24: Requiring Allow in 405 responses
> > Archived-At: <http://www.w3.org/mid/[email-masked] 
> > >
> >
> >
> >
> >>> there may exist valid reasons in particular circumstances to  
> >>> ignore a particular item, but the full implications must be  
> >>> understood and carefully weighed before choosing a different course.
> >
> >

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 28 Feb 2008 17:15:11 +1100

I don't feel like we're particularly making progress here (and I'm not  
trying to pick on Henrik in particular).

A number of dimensions have been talked about:

  1) Whether the list of headers MAY/SHOULD/MUST be present
     a) ... for 405 responses
     b) ... for OPTIONS responses
        I) ... that are generated by the origin server
        II) ... that are generated by intermediaries
  2) Whether the list of headers, in each of these cases, MAY/SHOULD/ 
MUST be complete

A few observations;

* (I) vs. (II) is IMO already taken care of; the text that mentions  
Allow in conjunction with OPTIONS starts "A 200 response SHOULD  
include..."
* There is no existing normative text for (b); OPTIONS lists Allow as  
an example, nothing more AFAICT. Expanding it to a SHOULD or MUST  
seems too aggressive, and software that depends upon it today is  
already taking liberties (unless a specific overlay protocol like  
WebDAV specifies otherwise).
* The spec is already crystal-clear about (a); it's a MUST.

That, in combination with the original issue description <http://www3.tools.ietf.org/wg/httpbis/trac/ticket/24 
 >, makes me believe that the best solution is to point out non- 
normatively in 14.7 Allow that the list doesn't have to contain all  
possible methods, since that kind of wiggle room is already --  
arguably -- implied. E.g., change

"The actual set of allowed methods is defined by the origin server at  
the time of each request."

to

"The actual set of allowed methods is defined by the origin server at  
the time of each request, and may not necessarily include all (or any)  
methods that the server would actually allow in a request if presented."

That's my proposal.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 28 Feb 2008 09:47:11 +0100

Mark Nottingham wrote:
> ...
> * There is no existing normative text for (b); OPTIONS lists Allow as an 
> example, nothing more AFAICT. Expanding it to a SHOULD or MUST seems too 
> aggressive, and software that depends upon it today is already taking 
> liberties (unless a specific overlay protocol like WebDAV specifies 
> otherwise).
 > ...

Hm. From 
<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.9.2.p.6>:

"A 200 response SHOULD include any header fields that indicate optional 
features implemented by the server and applicable to that resource 
(e.g., Allow), possibly including extensions not defined by this 
specification."

So it seems to me that is really *is* a SHOULD level requirement for 
include "Allow".

> ...
> ...

Me not happy. From 
<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.14.7.p.5>:

"This field cannot prevent a client from trying other methods. However, 
the indications given by the Allow header field value SHOULD be 
followed. The actual set of allowed methods is defined by the origin 
server at the time of each request."

So, if a server returns an incomplete list of methods -- for instance, 
not including "PATCH", and the client actually follows *this* 
requirement, then it wouldn't even try PATCH.

BR, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 28 Feb 2008 23:14:18 +1100

On 28/02/2008, at 7:47 PM, Julian Reschke wrote:

>
> Mark Nottingham wrote:
>> ...
>> * There is no existing normative text for (b); OPTIONS lists Allow  
>> as an example, nothing more AFAICT. Expanding it to a SHOULD or  
>> MUST seems too aggressive, and software that depends upon it today  
>> is already taking liberties (unless a specific overlay protocol  
>> like WebDAV specifies otherwise).
> > ...
>
> Hm. From <http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.9.2.p.6 
> >:
>
> "A 200 response SHOULD include any header fields that indicate  
> optional features implemented by the server and applicable to that  
> resource (e.g., Allow), possibly including extensions not defined by  
> this specification."

My bad - was looking at the wrong section when I wrote that. so, 405  
MUST, OPTIONS implicit SHOULD.

> So it seems to me that is really *is* a SHOULD level requirement for  

Well, it's an example of how to fulfil a SHOULD-level requirement.  
Almost the same, but not quite.

>> ...
>> "The actual set of allowed methods is defined by the origin server  
>> at the time of each request."
>> to
>> "The actual set of allowed methods is defined by the origin server  
>> at the time of each request, and may not necessarily include all  
>> (or any) methods that the server would actually allow in a request  
>> if presented."
>> That's my proposal.
>> ...
>
> Me not happy. From <http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.14.7.p.5 
> >:
>
> "This field cannot prevent a client from trying other methods.  
> However, the indications given by the Allow header field value  
> SHOULD be followed. The actual set of allowed methods is defined by  
> the origin server at the time of each request."
>
> So, if a server returns an incomplete list of methods -- for  
> instance, not including "PATCH", and the client actually follows  
> *this* requirement, then it wouldn't even try PATCH.

It's a SHOULD; there may be legitimate reasons for it not to be  
followed.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 28 Feb 2008 13:26:01 +0100

> Almost the same, but not quite.

Let me rephrase that: It's an example of a header that SHOULD be 
included; there may be others not defined in RFC2616. (The "DAV" header 
falls into that category).

>>> the time of each request, and may not necessarily include all (or 
>>> any) methods that the server would actually allow in a request if 
>>> presented."
>>> That's my proposal.
>>> ...
>>
>> Me not happy. From 
>> However, the indications given by the Allow header field value SHOULD 
>> be followed. The actual set of allowed methods is defined by the 
>> origin server at the time of each request."
>>
> It's a SHOULD; there may be legitimate reasons for it not to be followed.

It seems to me it would be unwise to say "clients SHOULD believe the 
Allow header", but "servers MAY leave of methods".

If we relax the requirement for the production, we also need to relax 
the requirement for the recipient.

BR, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 29 Feb 2008 10:25:07 +1100

On 28/02/2008, at 11:26 PM, Julian Reschke wrote:

> If we relax the requirement for the production, we also need to  
> relax the requirement for the recipient.

How about

Clients SHOULD believe the Allow header
Servers SHOULD send a complete Allow header

with weasel words on both sides to clarify that the client may try  
other ones in cases where it has external information / has explicit  
user input, and the servers may leave off methods for security, access  
control, or implementation reasons?

---

From: Stefan Eissing <[email-masked]>
Date: Fri, 29 Feb 2008 10:08:59 +0100

Am 29.02.2008 um 00:25 schrieb Mark Nottingham:

>> It seems to me it would be unwise to say "clients SHOULD believe  
>> the Allow header", but "servers MAY leave of methods".
>>

"SHOULD believe" is a new one to me.

How about describing the Server requirements and leaving it at that?  
Those rebel clients that just try methods, well, we could add a:  
"server SHOULD be able to handle methods not listed in their Allow  
header responses". But that is underestimating the reader, or?

//Stefan

--
<green/>bytes GmbH, Hafenweg 16, D-48155 Münster, Germany
Amtsgericht Münster: HRB5782

---

From: Julian Reschke <[email-masked]>
Date: Fri, 29 Feb 2008 11:35:31 +0100

Mark Nottingham wrote:
> with weasel words on both sides to clarify that the client may try other 
> ones in cases where it has external information / has explicit user 
> input, and the servers may leave off methods for security, access 

I think that's what we need to say.

BR, Julian

---

From: "Brian Smith" <[email-masked]>
Date: Fri, 29 Feb 2008 10:21:48 -0800

Stefan Eissing wrote:
> >> It seems to me it would be unwise to say "clients SHOULD 
> >> believe the Allow header", but "servers MAY leave of methods".
> >>
> >> If we relax the requirement for the production, we also 
> >> need to relax the requirement for the recipient.
> >
> >
> > How about
> >

The Allow header does not give enough information to be usefully
processed by a client. The absence of a method in an Allow header does
not mean that the method should not be used. It just means that the
server explicitly supported that method *at the time the response was
generated*, and it is unlikely to support the methods until some
*unspecified* server state changes.

How long should the client cache the value of an Allow header?
Currently, that is up to the client. Is a client *required* to cache the
Allow header from a previous response so that it can make use of it in a
future response? No. Should a client avoid methods that aren't listed in
a cached Allow header? No, because that requires the client to cache
previous Allow headers (plus, how did it get the Allow header that tells
it what is allowed, without making a request?).

This mechanism would little more sensible if there was a Disallow header
that specifically listed methods that the server intends to always
reject until some (unspecified) server state changes. Then you could say
that the client SHOULD NOT use a method listed in Disallow unless or
until it thinks the information in the header is invalid (stale).
"Disallow: *" would mean that *only* the methods listed in the Allow
header were valid at the time the response was generated, and only those
methods will be supported until some (unspecified) server state changes.
If the method is not explicitly allowed or disallowed, then the validity
of the method is considered to be unspecified. 

- Brian

---

From: "Roy T. Fielding" <[email-masked]>
Date: Fri, 29 Feb 2008 12:04:49 -0800

There is no point in arguing this.  Look at what has been implemented  
so far
and remove the cases that have not.

....Roy