Subject: Straw-man charter

From: Mark Nottingham <[email-masked]>
Date: Mon, 5 Mar 2007 22:54:26 -0800

Below, I've cut-and-pasted a straw-man charter along the lines that  
have been previously discussed. It's on a fairly short time-scale, to  
focus efforts on interop and editorial work, rather than spinning off  
into large-scale revisions or adding new features.

The tentative path forward is to discuss this informally in Prague,  
have a formal BoF in Chicago, and start thereafter.

Comments would be very much appreciated.

Cheers,

---8<---
HyperText Transfer Protocol Revision (http-bis) Charter

Last Modified: 2007-01-14

Chair(s):
[TBD]

Applications Area Director(s):
[TBD]
Lisa Dusseault <[email-masked]>

Applications Area Advisor:
[TBD]

Mailing Lists:
General Discussion: [email-masked]
To Subscribe: [email-masked]
In Subject: subscribe
Archive: http://lists.w3.org/Archives/Public/ietf-http-wg/

Description of Working Group:
HTTP is one of the most successful and widely-used protocols on the  
Internet today. However, its specification has several editorial  
issues. Additionally, after years of implementation and extension,  
several ambiguities have arisen, impairing interoperability and the  
ability to easily implement and use HTTP to its full potential.

The working group will refine RFC2616 to:
   * Incorporate errata
   * Improve editorial quality
   * Clarify conformance requirements and targets
   * Eliminate ambiguities where they affect interoperability
   * Document the extensibility model of HTTP
   * Add implementation advice (e.g., deprecating problematic  
optional features, if necessary)
   * Update to reflect current IETF practice
   * Identify mandatory-to-implement security mechanisms

In doing so, it should consider:
   * Implementer experience
   * Demonstrated use of HTTP
   * Impact on existing implementations and deployments

The working group must not introduce a new version of HTTP. It should  
not introduce new features or capabilities to HTTP, except where  
doing so is necessary to improve interoperability.

The Working Group's sole specification deliverable is a document that  
is suitable to supersede RFC2616.

Additionally, the working group may produce one or more test suites  
for HTTP conformance, if there is sufficient interest.

Goals and Milestones:
Sep 2007 - First HTTP Revision Internet Draft
Dec 2007 - IETF 70 Meeting, TBD
Mar 2008 - IETF 71 Meeting, TBD
Apr 2008 - Request Last Call for HTTP Revision
Jul 2008 - IETF 72 Meeting, TBD
Aug 2008 - Submit HTTP Revision to IESG for consideration as a  
Proposed Standard
--->8---

--
Mark Nottingham     http://www.mnot.net/

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 06 Mar 2007 08:09:28 +0100

>   * Eliminate ambiguities where they affect interoperability

Specifications should not have ambiguities (that is, cases where the
specification can be read in multiple ways but no one knows for sure
which reading is correct); presumably you mean ambiguities should be
removed, e.g. by explicitly making certain things implementation-de-
fined or by introducing new requirements if required for interop.

>   * Identify mandatory-to-implement security mechanisms

This sounds rather controversial.

>is suitable to supersede RFC2616.

I think the Working Group should be given the option to produce more
than one document to supercede RFC 2616; various suggestions what
could be factored out have been made here. The group might well decide
that keeping everything in one document is best, but it should not be
required to recharter if it decides otherwise.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: +49(0)621/4309674 · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Julian Reschke <[email-masked]>
Date: Tue, 06 Mar 2007 10:25:26 +0100

Mark Nottingham schrieb:
> ...
> Aug 2008 - Submit HTTP Revision to IESG for consideration as a Proposed 
> Standard
> ...

s/Proposed/Draft/

---

From: Julian Reschke <[email-masked]>
Date: Tue, 06 Mar 2007 11:14:47 +0100

> This sounds rather controversial.

True. Please make this an option.

> required to recharter if it decides otherwise.

In agree in theory. In practice, re-organizing the document that much is 
likely to collide with the goal to keep changes to minimum. That being 
said, I'd be ok with "one or more documents".

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 06 Mar 2007 11:16:02 +0100

>> Aug 2008 - Submit HTTP Revision to IESG for consideration as a 

Speaking of which: should we leave the door open for "full standard" 
(our downrefs may advance, and/or the IETF's policy on normative 
downrefs may change).

Best regards, Julian

---

From: "Anne van Kesteren" <[email-masked]>
Date: Tue, 06 Mar 2007 11:24:59 +0100

On Tue, 06 Mar 2007 07:54:26 +0100, Mark Nottingham <[email-masked]> wrote:
> Additionally, the working group may produce one or more test suites for  
> HTTP conformance, if there is sufficient interest.

I actually think this and maybe two interoperable implementations in  
popular browsers of the whole specification should be a requirement.  
Probably also for the server side of things. Otherwise the real issues  
probably never get fixed and deferred with arguments like: "We'll just  
wait until they get their act together..."

-- 
Anne van Kesteren
<http://annevankesteren.nl/>
<http://www.opera.com/>

---

From: Julian Reschke <[email-masked]>
Date: Tue, 06 Mar 2007 12:23:45 +0100

> wait until they get their act together..."

I think I strongly disagree. Requiring that things are indeed completely 
implemented will either result in lots of stuff being taken out, or the 
spec never be done. Both are IMHO non-goals. The agenda is *not* 
comparable to what the WHATWG is doing with HTML.

So what *are* the real issues, then? Have they been raised over here?

Best regards, Julian

---

From: "Larry Masinter" <[email-masked]>
Date: Tue, 6 Mar 2007 06:34:33 -0800

> The working group will refine RFC2616 to:

See below (RFC2617 too)...

>    * Incorporate errata

Add 'and updates' -- it's my impression that several other RFCs
update HTTP or attempt to clarify it, and these updates should
be incorporated. (WebDAV?)

>    * Clarify conformance requirements and targets

I don't know what 'and targets' means. How about just
'Clarify conformance requirements'?

>    * Eliminate ambiguities where they affect interoperability

How about just 'Remove ambiguities'? (As per previous
comments). And it is hubris to believe you can
eliminate them. 

>    * Document the extensibility model of HTTP

How about  'Clarify methods and requirements for extensibility'? 
I think there's a model there, it's just unclear. The
new document should reference the HTTP header registry,
for example.
 
> optional features, if necessary)

I think this is two separate bullets:

     * Remove or deprecate features that are not widely
       implemented interoperability and are not well specified.

and

     * Where necessary, add implementation advice

(Lots of implementation advice doesn't below in the spec.)

>    * Update to reflect current IETF practice

I'm not sure what this means, exactly, or why it is a goal
rather than a means. I suggest removing it from the charter.

>    * Identify mandatory-to-implement security mechanisms

I think, alas, it is necessary to update RFC2617, since the
two documents go together. And I think the goal should be
to address security in the context of HTTP's most common
application, namely 'web browsing'.  I'm not quite sure
how to rewrite this bullet. Best I can do for now is:

     * Identify security mechanisms appropriate
       (and mandatory to implement) for common applications
       of HTTP, including web browsing.

> doing so is necessary to improve interoperability.

I think 'improve' is too low a bar. And 'necessary to
achieve interoperability' seems a bit far-fetched, since
there's lots of interoperable HTTP today. I think taking out
the entire 'except where' clause is a good idea.

If you really want to allow new features or capabilities,
then take the whole feature out of base HTTP, and
put it in a separate Proposed Standard document
(using the now clarified extensibility mechanism).

> The Working Group's sole specification deliverable is a 
> document that is suitable to supersede RFC2616.

  'The Working Group's deliverables are a set of documents
  suitable to supersede RFCs 2616 and 2617.'

I understand the desire to avoid mission creep, but there might
be additional documents (e.g., Informational documents for
lengthier implementation advice, or Experimental or Proposed
Standard documents for solutions to currently poorly
specified features). But there's always the possibility of
revising the charter to include those, as necessary.
 
> for HTTP conformance, if there is sufficient interest.

I don't think this belongs in the charter as a deliverable
of 'the working group', and 'if there is sufficient interest'
means that it isn't really an IETF activity. 

If you want to say anything about test suites, how about:

 'The working group should review (and may document) 
 test suites for HTTP conformance, as they are made
 available.'

There is an extensive published literature on HTTP
implementation, conformance, operational behavior 
and performance. I'm not sure it belongs in the charter,
but I would think that an annotated bibliography
of _relevant_ documents would be a good thing to have.

> Aug 2008 - Submit HTTP Revision to IESG for consideration as a  

Internet Standard.  (Note that a single Internet Standard
may consist of several documents.)

Larry
-- 
http://larry.masinter.net

---

From: "Anne van Kesteren" <[email-masked]>
Date: Tue, 06 Mar 2007 15:44:29 +0100

On Tue, 06 Mar 2007 12:23:45 +0100, Julian Reschke <[email-masked]>  
wrote:
> spec never be done.

If that's indeed the result something is clearly wrong with the current  
specification. What's the use of a specification which authors and  
implementors can't actually rely on?

> So what *are* the real issues, then? Have they been raised over here?

I raised one (which was quickly rejected even though multiple people  
indicated it could not be implemented...), but there are others. Such as  
redirection of a POST as a GET etc. I'm not aware of a full list though,  
but such a list would probably become apparent once you start doing actual  
testing.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 06 Mar 2007 16:09:26 +0100

> be incorporated. (WebDAV?)

I would say WebDAV *extends* - using new headers and methods - but 
doesn't update, so I'm not sure it needs to be considered (and that 
coming from me :-).

> comments). And it is hubris to believe you can

Agreed. So: "Eliminate *known* ambiguities"?

> new document should reference the HTTP header registry,

Right. And the status code registry.

> rather than a means. I suggest removing it from the charter.

I though that refers, for instance, to the ABNF notation issue.

>        of HTTP, including web browsing.

I do agree that RFC2617 needs to be updated as well, but I'm not sure it 
  absolutely needs to happen at the same time, and whether the group 
working on that should be the same.

> (using the now clarified extensibility mechanism).

Wouldn't a new feature have to start as "proposed" on the standards 
track anyway?

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 06 Mar 2007 16:18:35 +0100

Anne van Kesteren schrieb:
>> I think I strongly disagree. Requiring that things are indeed 
>> completely implemented will either result in lots of stuff being taken 
>> out, or the spec never be done.
> 

That's the debate about whether to spec what people *should* do as 
opposed to what they do today. I disagree that it's useless to specify 
things even though existing implementations may get it wrong.

There's a problem if the spec defines something as mandatory, and nobody 
does it. Or if the spec says something that can't be implemented. Things 
like these should be discussed, agreed.

> indicated it could not be implemented...), but there are others. Such as 

That would be 
<http://lists.w3.org/Archives/Public/ietf-http-wg/2006OctDec/thread.html#msg224>, 
right? I wouldn't say it was rejected altogether, it's just that there 
was no agreement that Content-Location can be removed completely.

> but such a list would probably become apparent once you start doing 
> actual testing.

Is that about status 302 vs 303? Is there an open issue around here?

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 6 Mar 2007 09:52:48 -0800

> This sounds rather controversial.

Yes, indeed. I included it because we'll need to have *some* sort of  
conversation about it in the chartering process.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 6 Mar 2007 09:53:20 -0800

Hi Larry,

Thanks, that's very helpful. A few responses/comments below;

> be incorporated. (WebDAV?)

Generally I agree, except perhaps in the WebDAV case, as Julian says.

> 'Clarify conformance requirements'?

The target is the subject of the requirement; e.g., A Proxy MUST...

That said, "clarify conformance requirements" would do the same job.

>        of HTTP, including web browsing.

That sounds possible. I'm not sure whether that requires a revision  
of 2617 or not; TLS+Basic may be satisfactory.

> (using the now clarified extensibility mechanism).

I originally omitted the except clause, but I got some pushback  
because some people might interpret that as not allowing *anything*  
new on the wire, where we might find something is needed to fix  
something (e.g., pipelining). Since we're talking about folding in  
updates to HTTP, as long as it's backwards-compatible, it would seem  
to make sense to include such an update in the document, rather than  
folding in old updates, but putting new updates in new documents.

All that said, the intent here was to disallow people trying to do  
genuinely new features/capabilities that aren't interop-related to  
the protocol.

---

From: Jeffrey Mogul <[email-masked]>
Date: Tue, 06 Mar 2007 11:12:23 -0800

I'd like to suggest another deliverable (possibly not a true
"specification"), which could either be a separate document
or possibly a new section in rfc2616-bis.

This would be a description of which other HTTP-specific RFCs
are effectively part of the HTTP standard.  I can think of
a few off the top of my head:

   RFC2145: Use and Interpretation of HTTP Version Numbers
   RFC2817: Upgrading to TLS Within HTTP/1.1
	    (if only because it defines the HTTP Status code registry!)
   RFC4229: HTTP Header Field Registrations

I suspect there are others that could be included, but my memory
fails me for now.  And there are probably some for which there
will be signficant debate as to their value and status.  (What
about more recent standards that retroactively constrain HTTP
implementors, if any?  e.g., character sets?)

My suggestion is based on the observation that some implementors
look at RFC2616 as the entirety of the "HTTP specification", because
there are no obvious pointers from that document to the others.
Hence we have repeated discussions on this mailing list about
how proxies should handle version numbers (for example).

I'm not suggesting that we include true extensions, such as
WebDAV, in this category -- although I suppose that argument
could be made, I wouldn't be the one to make it.

-Jeff

---

From: "Justin Erenkrantz" <[email-masked]>
Date: Tue, 6 Mar 2007 13:40:52 -0800

> have a formal BoF in Chicago, and start thereafter.

Sounds reasonable to me.

I'd love to help out with this, however attending a BoF in Chicago
(69th IETF) is unlikely due to its conflict with OSCON (which takes
precedence for me).  The conference organizers really need to talk to
each other to avoid such lame conflicts...  =(  -- justin

---

From: Julian Reschke <[email-masked]>
Date: Wed, 07 Mar 2007 12:28:20 +0100

> 	    (if only because it defines the HTTP Status code registry!)

The status code registry is very well hidden in there... I'd propose to 
either integrate it into RFC2616bis, or to move it into a separate 
document (such as proposed with 
<http://greenbytes.de/tech/webdav/draft-reschke-http-status-registry-latest.html>).

Best regards, Julian

---

From: Henrik Nordstrom <[email-masked]>
Date: Wed, 07 Mar 2007 18:13:42 +0100

> was no agreement that Content-Location can be removed completely.

Content-Location servers an important role in HTTP cache correctness and
Vary logics and can not be removed without significantly crippling the
specifications. But it's a optional header, and servers are free not to
send Content-Location if there is no suitable location to refer to of
the response entity or if they consider it unneeded.

Things like reverse proxies remapping the URL name space etc all has to
consider their effects on Content-Location, just as they need for
Location, Request-URI etc. By no means an impossible task, just an often
overlooked or forgotten task as most things tend to work anyway..

But I think everyone agreed that it's due to the large number of broken
servers out there isn't viable to implement two clauses relating to the
Content-Location header:

a) Future requests MAY specify the Content-Location URI as the request-
   URI if the desire is to identify the source of that particular
   entity.

and

b) The value of Content-Location also defines the base URI for the
   entity.

I am not entirely sure why these two criterias was added in RFC2616 (is
not in RFC2068 from what I can tell), but a guess is to align HTTP with
the related MHTML definitions of the same headers. But the definition in
RFC2616 does not conflict with RFC2068, only extends it's use. In both
specifications Content-Location 

In terms of specification they both makes sense. But the fact that none
of the major user-agents have implemented them for many years after and
that there is quite many deployed servers handling this wrongly does
speak for that these two clauses should perhaps be removed.

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Wed, 07 Mar 2007 20:09:10 +0100

> Is that about status 302 vs 303? Is there an open issue around here?

Well, kind of. The specs and real world is still not aligned, just as in
the case of Content-Location.

RFC2616 added the 303 and 307 status codes trying to solve the problem
by defining new status codes replacing the earlier. But it did not
change 302 to reflect what the majority of the implementations actualy
do other than adding a note that many implementations is broken and uses
GET instead, and neither did it mark 301/302 as unusable.. And with most
implementers (all sides) still following the majority instead of
specifications..

And as result

In response to a POST

      2616    Most User-Agents
301   POST*   GET
302   POST*   GET
303   GET     GET?
307   POST    POST?

* with a note that there is implementations using GET.
? Possibly not implemented

As a result very few expects implementations to follow the RFC wrt
301/302 responses, and the new status codes is not used very much as few
see a need for them..

Here is some stats from a sample of 3xx responses to little more than 3
million POST requests:

301 1100
302 78793
303 664
304 25209
307 31

the complete breakdown in status codes in response to POST can be found
at http://www.henriknordstrom.net/code/http_post_status-20070307.txt

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Wed, 07 Mar 2007 20:13:28 +0100

> <http://greenbytes.de/tech/webdav/draft-reschke-http-status-registry-latest.html>).

My opinion is to integrate.

RFC2616bis should reference the IANA directly for the relevant
registries defined based on RFC2616, with a reference to the RFC where
the IANA responsibility was originally defined for that registry where
applicable.

With the IANA registries already existing and operational (even if not
well known) I don't see the point of duplicating their content in a new
registry type RFC (and therefore not in a draft either).

Regards
Henrik

---

From: "Anne van Kesteren" <[email-masked]>
Date: Wed, 07 Mar 2007 23:33:04 +0100

> implementers (all sides) still following the majority instead of

Following the spec for this is not really feasible. We tried and failed.  
It would make sense to simply change the specification imo...

---

From: Henrik Nordstrom <[email-masked]>
Date: Thu, 08 Mar 2007 00:57:14 +0100

> It would make sense to simply change the specification imo...

Hmm.. perhaps it would work to rewrite the specs to allow the request to
be retried using either the original request method or a GET after a
POST resulting in a 301/302 redirect. I.e. not say that there is broken
user agents around but to also accept that broken behavior as an
alternative action.

  ... Alternatively if the request was a POST request the user agent
  MAY automatically retry the request as a GET request without requiring
  user confirmation.

As it's been specified since day 1 (before HTTP/1.0 even) that the
original request should be retried with the new Request-URI without
changing the request method we can not change the specs in such manner
that this would not be a correct thing to do. In terms of spec writing
the only possible choices is to clearly deprecate the 301/302 responses
as "broken beyond repair" in favor of the new 303 & 307 status codes and
hope the industry won't trash these as well, or to convince the industry
to fix up their act and start following the specs.. and I suspect you
won't be happy with either choice..

In terms of spec writing it's quite disappointing that this industry
can't accept a clear and well specified "thats the wrong thing to do",
and instead continues doing the wrong thing forever countless software
generations after the implementation error has been pointed out.. This
attitude is quite saddening for the future of the web..

Regards
Henrik

---

From: Eric Lawrence <[email-masked]>
Date: Wed, 7 Mar 2007 16:05:08 -0800

<< As it's been specified since day 1 (before HTTP/1.0 even) that the original request should be retried with the new Request-URI without changing the request method we can not change the specs in such manner that this would not be a correct thing to do.>>

Do you know of ~any~ user-agent which will perform a POST after receiving a 302 in response to the POST?  I have yet to encounter any such client.

Thanks,

Eric

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Henrik Nordstrom
Sent: Wednesday, March 07, 2007 3:57 PM
To: Anne van Kesteren
Cc: [email-masked] Group
Subject: Re: Redirection of a POST as a GET

> It would make sense to simply change the specification imo...

Hmm.. perhaps it would work to rewrite the specs to allow the request to be retried using either the original request method or a GET after a POST resulting in a 301/302 redirect. I.e. not say that there is broken user agents around but to also accept that broken behavior as an alternative action.

As it's been specified since day 1 (before HTTP/1.0 even) that the original request should be retried with the new Request-URI without changing the request method we can not change the specs in such manner that this would not be a correct thing to do. In terms of spec writing the only possible choices is to clearly deprecate the 301/302 responses as "broken beyond repair" in favor of the new 303 & 307 status codes and hope the industry won't trash these as well, or to convince the industry to fix up their act and start following the specs.. and I suspect you won't be happy with either choice..

In terms of spec writing it's quite disappointing that this industry can't accept a clear and well specified "thats the wrong thing to do", and instead continues doing the wrong thing forever countless software generations after the implementation error has been pointed out.. This attitude is quite saddening for the future of the web..

Regards
Henrik

---

From: Adrien de Croy <[email-masked]>
Date: Thu, 08 Mar 2007 13:35:27 +1300

> Do you know of ~any~ user-agent which will perform a POST after receiving a 302 in response to the POST?  I have yet to encounter any such client.

there aren't any that I have found either, and IMO it makes no sense to 
retry with another POST, the result of the POST was the redirection.  
The redirection in this case is not used to indicate "try again some 
other location", it is used to indicate a result, which is obtainable at 
the new location.  So this is contrary to the spec, but everyone does 
it.  You'd have to ask website designers if there would be a use for a 
return code that meant "try the exact same request at this new location".

If any UA used the original POST method again, it would break most 
existing server/site implementations.  Software authors are quick to 
throw out the written specification in favour of what actually works.

So, I can see where the idea comes from about deprecating the 301/302 
codes, and using new ones, but that will create more problems than the 
hypothetical (nonexistent) problem it will solve.  In fact, the only 
problem does not lie in interoperability, but in conformance to the 
spec.  Much easier to change the spec in a way that will have no 
operational effect on anyone, than to force all authors to make 
modifications to code for no reason other than to then have it comply to 
the letter of a spec that there already is consensus on (albeit 
non-conformant).

What is a spec after all, but a statement of requirements for 
compliance, with the intent of interoperability - creating a consensus 
of operation.  if the consensus already exists outside the spec, then 
there is already a defacto spec.  Realigning the written spec to the 
defacto one would then seem to be the best path?

Of course if there are significant implementations out there that do 
rely on the written specified behaviour, then it's a different picture, 
but are there any, or is this just a purely hypothetical situation?

Adrien

---

From: "Mike Schinkel" <[email-masked]>
Date: Wed, 7 Mar 2007 23:58:34 -0500

Adrien de Croy wrote:
> If any UA used the original POST method again, 
> it would break most existing server/site 
> implementations.  

How would it break the implementations?

-- 
-Mike Schinkel
http://www.mikeschinkel.com/blogs/
http://www.welldesignedurls.org
http://atlanta-web.org - http://t.oolicio.us

---

From: Adrien de Croy <[email-masked]>
Date: Thu, 08 Mar 2007 18:10:27 +1300

In many if not most cases, the case where a response to a POST is a 
redirect is as a result of some CGI or server-side process running some 
web-based application.

in these cases, the web application (e.g. webmail, etc) really doesn't 
expect that the redirect they pass back to the client will result in a 
POST - they expect it to be a GET.  In fact, often the target location 
isn't even a script / executable resource, but a pure HTML resource.

Trying to do a POST request specifying a non-executable target resource 
I wouldn't expect to work.  And worse would be in cases where the target 
file would in other cases expect a POST request.  In this case, the POST 
data would be all wrong, being the data from a different resource.

It's an extremely common technique.  I've never seen a web application 
that accepts POST and returns a redirect expect anything other than a 
GET from the client in response.

most user login web pages do this for instance, shopping carts.. in fact 
most web forms that use POST.  The list is endless.

lots of things that would break in other words if the spec were changed 
to mandate a POST as a followup request in this scenario.

Adrien

---

From: "Mike Schinkel" <[email-masked]>
Date: Thu, 8 Mar 2007 03:32:35 -0500

Adrien de Croy wrote:
> In many if not most cases, the case where a response to a 
> POST is a redirect is as a result of some CGI or server-side 
> process running some web-based application.
> in these cases, the web application (e.g. webmail, etc) 
> really doesn't expect that the redirect they pass back to the 
> client will result in a POST - they expect it to be a GET.  
> In fact, often the target location isn't even a script / 
> executable resource, but a pure HTML resource.

What is the source of data for your assertion?  Why is possible for a web
developer wanting to move a resource?  Not all POSTs original from the same
source file on the same server. Consider a search request that allows
someone from one site to POST to another.  Without being able to redirect a
POST that resource is effectively either cemented and never able to move, or
if it has to move it breaks any links that exist to the resource.
 
> Trying to do a POST request specifying a non-executable 
> target resource I wouldn't expect to work.  And worse would 
> be in cases where the target file would in other cases expect 
> a POST request.  In this case, the POST data would be all 
> wrong, being the data from a different resource.

I can see the logic that a specific post being redirected to a random page
not designed to receive the post would be a bad thing, but I can't see why
anyone would ever configure a server to redirect to a resource that wasn't
design to receive the post.  

> It's an extremely common technique.  I've never seen a web 
> application that accepts POST and returns a redirect expect 
> anything other than a GET from the client in response.

Does HTTP support POST redirect?  I thought it did not.  If HTTP doesn't
support POST redirect, it would seem to me to be inappropriate to use the
fact you don't see anyone using it as justification for it not being needed,
right?

> most user login web pages do this for instance, shopping 
> carts.. in fact most web forms that use POST.  The list is endless.
> lots of things that would break in other words if the spec 
> were changed to mandate a POST as a followup request in this scenario.

I still don't see how anything could break unless the person issuing the 301
wants it to break.  Or maybe I'm just completely misunderstanding the issue?

---

From: Julian Reschke <[email-masked]>
Date: Thu, 08 Mar 2007 10:06:55 +0100

> but are there any, or is this just a purely hypothetical situation?

Be careful.

HTTP is not only about browsers:

1) WebDAV servers are known to use 302 on methods such as PROPFIND (for 
instance, Apache/moddav for PROPFIND on a collection URL with trailing 
slash missing), and expect clients to proceed with PROPFIND, not GET.

2) I'm pretty sure that APP clients are expected noz to follow a 302 
upon POST with GET.

The confusion seems to be caused by 302 being used for two separate 
things (moving a resource, and pointing to a retrievable result of a 
POST). Both deserve separate status codes, and have got them as 303 and 
307 since January 1997 (307) or even longer (303).

It seems to me that the right thing to do here is to clearly deprecate 
302 (describing both what it was supposed to do, and what it does in 
practice), and lobbying for proper use of 303 and 307 instead.

Best regards, Julian

---

From: "Robert Sayre" <[email-masked]>
Date: Thu, 8 Mar 2007 04:26:10 -0500

...
...
> The working group must not introduce a new version of HTTP.

It seems obvious that these two clauses are incompatible. We've
discussed this on the list before. I have a draft that states:

    It is possible that HTTP will be revised in the future. HTTP 1.1 [RFC2616]
    and Use and Interpretation of HTTP Version Numbers [RFC2145] define
    conformance requirements in relation to version numbers. In HTTP 1.1,
    all authentication mechanisms are OPTIONAL, and no single transport
    substrate is specified. Any HTTP revision that adds a mandatory security
    mechanism or transport substrate MUST increment the HTTP version
    number appropriately.

Does that paragraph contain incorrect information? If not, the charter
is inappropriate, because it disregards the IETF consensus recorded in
several documents.

Thoughts?

-- 

Robert Sayre

"I would have written a shorter letter, but I did not have the time."

---

From: Henrik Nordstrom <[email-masked]>
Date: Thu, 08 Mar 2007 11:35:09 +0100

tor 2007-03-08 klockan 03:32 -0500 skrev Mike Schinkel:

> Does HTTP support POST redirect?  I thought it did not.

It does. In several forms.

We have

a) The original 301/302 redirects, which was meant to tell the user
agent that the resource has moved and the sane request should be retried
at the new location. However, almost everyone (if not all) implements it
wrongly using a GET request instead.

b) To solve this RFC2616 adds the 303/307 redirects, where 303 SHOULD
result in a GET and a 307 SHOULD result in a POST (or whatever method
was used for the first request).

> wants it to break.  Or maybe I'm just completely misunderstanding the issue?

The issue is 'a'. Specs saying one thing, most user agents and servers
another...

Regards
Henrik

---

From: "Anne van Kesteren" <[email-masked]>
Date: Thu, 08 Mar 2007 11:37:36 +0100

> attitude is quite saddening for the future of the web..

In terms of implementing it's quite disappointing that spec writers don't  
always see a clear message that the industry doesn't want it the way  
things are specified and instead requests a different approach.

(This is also true, for instance, with the fatal error handling rules in  
XML which clearly don't work for the web.)

---

From: Adrien de Croy <[email-masked]>
Date: Thu, 08 Mar 2007 23:40:15 +1300

Julian Reschke wrote:
>> rely on the written specified behaviour, then it's a different 
>> picture, but are there any, or is this just a purely hypothetical 
>> situation?
> > ...
>
> Be careful.
>
understood... HTTP is used for many things not browser-related... (my
personal favourite is UPnP - HTTP over UDP multicast...urgh!).  Hence my
comment about implementations.

so I agree with you - not many options but a cleanup of the codes.  If
they have already been in the spec for years, hopefully they are
implemented (albeit dormant) in deployed software?  Can but test I guess.

Adrien

> 1) WebDAV servers are known to use 302 on methods such as PROPFIND 
> (for instance, Apache/moddav for PROPFIND on a collection URL with 
> trailing slash missing), and expect clients to proceed with PROPFIND, 
> not GET.
>
> POST). Both deserve separate status codes, and have got them as 303 
> and 307 since January 1997 (307) or even longer (303).
>

---

From: Henrik Nordstrom <[email-masked]>
Date: Thu, 08 Mar 2007 11:42:41 +0100

> slash missing), and expect clients to proceed with PROPFIND, not GET.

Yes. The industry standard deviation from the specifications is limited
to POST I think.

> practice), and lobbying for proper use of 303 and 307 instead.

Agreed.

Additionally we could accept the industry defacto standard, and adjust
301/302 to allow GET to be used automatically without user confirmation
if the request was a POST request making the current industry use
compliant.

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Thu, 08 Mar 2007 11:46:39 +0100

> implemented (albeit dormant) in deployed software?  Can but test I guess.

In the statistics I collected the other day (see the earlier post in
this thread, the first with this subject) both 303 and 307 responses was
seen. Not many, but they are in use at least somewhere..

Regards
Henrik

---

From: Julian Reschke <[email-masked]>
Date: Thu, 08 Mar 2007 11:47:38 +0100

Anne van Kesteren schrieb:
> In terms of implementing it's quite disappointing that spec writers 
> don't always see a clear message that the industry doesn't want it the 
> way things are specified and instead requests a different approach.
> 
> 

Well, I think it clearly does work (minus the Content-Type encoding 
issue) for many things, such as feed reading (*), client-side XSLT, or 
WebDAV. All of these are part of the web.

It doesn't work well for XHTML, but that's because the most widely used 
user agent doesn't support it. I think it's incorrect to blame fatal 
error handling here.

Best regards, Julian

(*) The RSS/Atom support in IE7 IMHO requires well-formed content, and 
gets away with it. Bravo!

---

From: Julian Reschke <[email-masked]>
Date: Thu, 08 Mar 2007 11:59:21 +0100

> is inappropriate, because it disregards the IETF consensus recorded in

I think that analysis is correct. The charter should state that the 
issue needs to be resolved, but should avoid saying anything about the 
outcome.

> Thoughts?

As far as I can tell, we have reached a deadlock here, so I'd be 
surprised if there was progress any time soon. That being said, I'd 
*love* to see an official IESG statement about how they think existing 
specifications are supposed to be revised and progressed while 
introducing new mandatory requirements.

Best regards, Julian

---

From: Adrien de Croy <[email-masked]>
Date: Fri, 09 Mar 2007 00:02:29 +1300

one thing - are there any security implications with a browser say 
automatically resubmitting some POST data to another server based on a 
redirect code?

Quite often this data will be usernames and passwords.

I'm not sure how comfortable I would be typing my username and password 
into a form, and then having my browser automatically sending that 
information off to another site without my knowledge because the site 
sent back a 307.  There's no obvious trust association.  By rights I 
really should have been redirected to the new site by the request that 
served the form.

For robots I can see the need to do that.  Maybe the answer is

1. Redefine 301/302 to allow GET after a POST, making the entire 
industry compliant.
2. Leave 303 as is.
3. MUST use 307 if you want it to be the same method.

servers and proxies and websites are going to bear the brunt of any 
changes to this - it's trivial to support both the new and old codes in 
a client, but when to choose which one to use in a server is another matter.

knowing there is widespread understanding of the new codes in browsers 
allows a server to start using them, but then there's the issue of the 
countless CGI applications that set their own return codes from script.  
A server hosting such a script could automatically translate the code I 
guess, with the expectation that if it was a POST, then translate 
301/302 to 303, with optional configuration override.

Adrien

Julian Reschke wrote:
>> don't always see a clear message that the industry doesn't want it 
>> the way things are specified and instead requests a different approach.
>>
>>
>> (This is also true, for instance, with the fatal error handling rules 
>> in XML which clearly don't work for the web.)
>
> It doesn't work well for XHTML, but that's because the most widely 
> used user agent doesn't support it. I think it's incorrect to blame 
> fatal error handling here.
>
> Best regards, Julian
>

---

From: Julian Reschke <[email-masked]>
Date: Thu, 08 Mar 2007 13:15:15 +0100

> automatically resubmitting some POST data to another server based on a 

Yes. That's why 10.3 
(<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.10.3>) says:

"The action required MAY be carried out by the user agent without 
interaction with the user if and only if the method used in the second 
request is GET or HEAD."

(this part needs to be fixed to say "safe method" instead of "GET or HEAD").

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 8 Mar 2007 09:22:51 -0800

I'd like to discuss this here and in the Sunday meeting, with the aim  
of taking a *productive* set of questions/statements to the APPS area  
open meeting the following Monday.

On 08/03/2007, at 2:59 AM, Julian Reschke wrote:

> issue needs to be resolved, but should avoid saying anything about  
> the outcome.
>
>> Thoughts?
>
> *love* to see an official IESG statement about how they think  
> existing specifications are supposed to be revised and progressed  
> while introducing new mandatory requirements.
>
> Best regards, Julian
>
>
>
>
>

---

From: "Larry Masinter" <[email-masked]>
Date: Thu, 8 Mar 2007 11:15:39 -0800

>    * Identify mandatory-to-implement security mechanisms

There is no deadlock, or really a contradiction
"Identify mechanisms" doesn't mean that there will be
a single mechanism, and "mandatory-to-implement" doesn't
mean "mandatory in all situations".

BCP 56/RFC 3205 ("On the use of HTTP as a Substrate")
section 2.3 ("Security") seems to me like a good start
on what the security requirements for HTTP should be,
and perhaps the charter item for the working group
should be to review that section and either reference
it or update it as necessary.

Larry

---

From: Eric Lawrence <[email-masked]>
Date: Thu, 8 Mar 2007 11:22:21 -0800

Unfortunately, I've come to the conclusion that notion that a general-purpose user-agent may be able to protect user information once the user has entered it, is likely to be fundamentally unsound in real-world use.

While, in the abstract, it seems reasonable that a cross-server redirection would notify the user, this assumes that:

1> the user understands the implications (unlikely)
2> the user had not previously disabled such warning (most users do)
3> "safe" methods cannot be used unsafely (untrue, nothing stopping servers from resubmitting postdata/creds as querystring)
4> cross-data posting is usually "evil" and isn't already commonly used by "legitimate" sites (e.g. login.example.com redirect to services.example.com)

Even if all of these factors didn't exist, the fundamental problem is that the user-agent cannot protect the user's data once it's in the server's control.  A poorly or maliciously written server can perform all manner of activities with the data, including posting it in plaintext, in an unsecure database, etc.  If the user-agent inhibits the server's ability to direct the browser to resubmit the data elsewhere, the server is likely to either workaround the restriction via a mechanism which is no more private or safe (e.g. backend web-service call) or to circumvent the restriction directly (e.g. by passing the data in a query string).  Furthermore, in the case of browser user-agents, we cannot simply rely on adding heuristics or restrictions for FORM tags, because a web developer could simply circumvent those restrictions using either an applet (e.g. Flash) or by submitting the data without using HTML Forms submission (e.g. use XMLHTTP).

Eric Lawrence

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Julian Reschke
Sent: Thursday, March 08, 2007 4:15 AM
To: Adrien de Croy
Cc: [email-masked] Group
Subject: Re: Redirection of a POST as a GET

Yes. That's why 10.3

Best regards, Julian

---

From: David Morris <[email-masked]>
Date: Thu, 8 Mar 2007 12:57:16 -0800 (PST)

Folks, it is about time we recognize that permission of the user is pure
nonsense. They choose whatever they believe will let them complete what
they are doing.

The only people who know if a redirect of a POST makes sense are the
authors of the compound web server (server and application). If my
web server has moved, it is quite reasonable to redirect a POST to the
new process. I might do it for load balancing as well... I wrote the darn
thing and won't be issuing a redirect if the request is subject to double
jeopardy.

Demanding user permision is like writing 5 page credit card agreements in
micro font and then blaming the individual for not understanding the
obtuse language.

The protocol MUST support responsible behavior by the application
developers.

---

From: David Morris <[email-masked]>
Date: Thu, 8 Mar 2007 13:12:17 -0800 (PST)

If you have a trust relationship with the original server, you darn well
better beable to trust what that server does with your data ... and in my
mind, that extends to trusting that server to not redirect to an untrusted
server.

In any case, if this data is sensitive, you should make sure it is sent in
an SSL protected session and it seems VERY reasonable to not allow the
scheme to change in a redirect ... certainly not a down grade in security
level.

Telling the average user there is a concern isn't worth the effort.

---

From: Eric Lawrence <[email-masked]>
Date: Thu, 8 Mar 2007 13:16:37 -0800

<< it seems VERY reasonable to not allow the
scheme to change in a redirect ... certainly not a down grade in security
level.>>

Even here there's likely to be a problem.  A common pattern for web-based logins is to send credentials to a HTTPS page, and that returns a redirect with a Set-Cookie header containing an authenticated token.  The redirected page is very often a HTTP page, and hence would trigger the warning in your model.

-Eric

---

From: "Mike Schinkel" <[email-masked]>
Date: Thu, 8 Mar 2007 16:32:26 -0500

David Morris
> If you have a trust relationship with the original server, 
> you darn well better beable to trust what that server does 
> with your data ... and in my mind, that extends to trusting 
> that server to not redirect to an untrusted server.
> 
> In any case, if this data is sensitive, you should make sure 
> it is sent in an SSL protected session and it seems VERY 
> reasonable to not allow the scheme to change in a redirect 
> ... certainly not a down grade in security level.
> 

I was going to say essentially the same, but since you already did I'll just
+1.

Also, as a user, I myself would get pissed if I had to fill out a login form
twice and be mad at the website, not realizing it was the specification's
fault.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 08 Mar 2007 22:42:38 +0100

> twice and be mad at the website, not realizing it was the specification's

Well.

If a server redirects POST request from /a to /b *on the same server*, 
blame the server. It could easily let the request on /a succeed, or 
shouldn't have exposed /a in the first place. You know, Cool URIs Do Not 
Change.

Best regards, Julian

---

From: David Morris <[email-masked]>
Date: Thu, 8 Mar 2007 13:51:32 -0800 (PST)

I agree ... that is perhaps a very well known special case which deserves
some form of distinguishing design ... perhaps a special redirect code
which would mean temporary change of security level for authentication.

In then end, I only want user choice required when they have a chance
of understanding the significance of the choice.

On Thu, 8 Mar 2007, Eric Lawrence wrote:

> Even here there's likely to be a problem.  A common pattern for web-based
> logins is to send credentials to a HTTPS page, and that returns a
> redirect with a Set-Cookie header containing an authenticated token.
> The redirected page is very often a HTTP page, and hence would trigger
> the warning in your model.
>
> -Eric
>
^>

---

From: Henrik Nordstrom <[email-masked]>
Date: Thu, 08 Mar 2007 23:56:29 +0100

fre 2007-03-09 klockan 00:02 +1300 skrev Adrien de Croy:

> sent back a 307.

And the specs do not allow it without user confirmation.

This security blanked has always been in the specs regarding automatic
redirection, only allowing it to take place for GET/HEAD requests
without user confirmation. Even the HTTP/1.0 specs has this security
restriction.

Regards
Henrik

---

From: Adrien de Croy <[email-masked]>
Date: Fri, 09 Mar 2007 12:13:09 +1300

thanks for that.

this may be a key factor for why we are in the state we are in (i.e. why 
everyone responds to a 30x from a POST with a GET) - because it's the 
"safe and obvious" option, albeit non-conformant.

Anyone who's run Vista knows about how painful nags can get.  You can't 
do hardly anything without being prompted to make this or that decision, 
even though you just tell it to do something, it still asks for security 
clearance (even when you're an admin).

Most people just want the thing to work with as little thought and extra 
effort required as possible.

So designing requirements into the spec to prompt users for decisions is 
kinda doomed I think.  There will be strong consumer pressure to disable 
the nags, which software writers will respond to by implementing nag 
disabling options, which people will then turn on.  Alternatively there 
will be pressure brought to bear on site writers to not return 307 
responses because it generates a nag.  either way the potential benefits 
of the 307 could end up being lost, in which case....

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 8 Mar 2007 17:04:39 -0800

> the way things are specified and instead requests a different  

The specification writers did more to implement HTTP than every single
browser vendor still in business.  It's not our fault that some folks
are too lazy to read a specification that clearly defined the redirect
method since 1995 (when Henrik convinced an entire WG that we should
define the codes as implemented in CERN/W3C libwww).  None of your
implementations predate the specification and everyone participating
in the process agreed to the status code definitions.  It's not our
fault that you have no spine and won't report errors to users.
It's not our fault that you prefer to act like lemmings instead of
standing up for your own businesses.

....Roy

---

From: Eric Lawrence <[email-masked]>
Date: Thu, 8 Mar 2007 19:11:56 -0800

At this point, I'm not sure it's productive to point fingers anywhere.  We're in a world where this a spec that mandates one behavior, and on the order of a billion client installations that behave in a way contrary to the spec.

The 303 and 307 status codes effectively clarify the exact behavioral specification without impacting legacy compatibility, so it's not clear to me that there's any benefit in further work here beyond documenting the inconsistency in implementation for 301/302 and recommending that user-agents and servers use 303 and 307 as appropriate.

-Eric

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Roy T. Fielding
Sent: Thursday, March 08, 2007 5:05 PM
To: Anne van Kesteren
Cc: [email-masked] Group
Subject: Re: Redirection of a POST as a GET

....Roy

---

From: "Mike Schinkel" <[email-masked]>
Date: Thu, 8 Mar 2007 22:31:18 -0500

Julian Reschke wrote:
> If a server redirects POST request from /a to /b *on the same 
> server*, blame the server. 

Why would I blame the server?  redirects POST requests allow the IA to
evolve and not require a server to be littered with lots of special rules
nearly as long as otherwise.

> It could easily let the request on 
> /a succeed, or shouldn't have exposed /a in the first place. 

Restructuring the IA can created these situation, and maintaining legacy
URIs makes for increased complexity that will likely never get cleaned up
given your scenarios.

> You know, Cool URIs Do Not Change.

But we all know that URIs *do* change and that pro-hibition is almost never
an effective solution. Better to accomdate the reasons why URIs change and
make the changes invisible to the users.

---

From: "Mike Schinkel" <[email-masked]>
Date: Sat, 10 Mar 2007 09:00:44 -0500

Henrik Nordstrom wrote:
> a) The original 301/302 redirects, which was meant to tell 
> the user agent that the resource has moved and the sane 
> request should be retried at the new location. However, 
> almost everyone (if not all) implements it wrongly using a 
> GET request instead.
> 
> b) To solve this RFC2616 adds the 303/307 redirects, where 
> 303 SHOULD result in a GET and a 307 SHOULD result in a POST 
> (or whatever method was used for the first request).
> 
> > I still don't see how anything could break unless the 
> person issuing 
> > the 301 wants it to break.  Or maybe I'm just completely 
> misunderstanding the issue?
> 
> The issue is 'a'. Specs saying one thing, most user agents 
> and servers another...

Thanks for the clarification. I made a bad assumption.

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Mon, 12 Mar 2007 21:01:54 +0100

> practice), and lobbying for proper use of 303 and 307 instead.

Aye, this seems like a good compromise.

-- 
Asbjørn Ulsberg     -=|=-    http://virtuelvis.com/quark/
«He's a loathsome offensive brute, yet I can't look away»