Subject: Extension methods & XMLHttpRequest

From: "Mark Baker" <[email-masked]>
Date: Sat, 10 Jun 2006 00:50:06 -0400

Folks,

The W3C WebAPIs WG is attempting to standardize the XMLHttpRequest
Javascript object[1], and part of that work involves deciding how to
handle extension HTTP methods.

Some of the WG is interested in establishing a "whitelist" of methods
deemed safe at the time of publication of our spec, with the intent
that all other methods would be disallowed.  Others would prefer a
"blacklist", whereby we specify that methods known to be a security
problem (in the context of the use of XHR, e.g. CONNECT) not be used,
but that unknown methods be allowed.

We would be interested to know what the HTTP community would recommend.

Thanks.

 [1] http://www.w3.org/TR/XMLHttpRequest/

Mark.

---

From: Lisa Dusseault <[email-masked]>
Date: Sat, 10 Jun 2006 12:07:56 -0700

A whitelist seems very poor for interoperability and future work.   
E.g. I've been talking recently about the need for HTTP clients  
sometimes to be able to request a login challenge, a request which  
might involve a new method.  The calendar access stuff in front of  
the IESG shortly, involves one new method.  OSAF is working on a  
WebUI CalDAV client that would usefully be able to use this new  
method as well as other non-2616 methods.

I wouldn't think that either a whitelist or a blacklist is necessary,  
but perhaps I haven't thought enough about the problem.  Can somebody  
provide a list of the methods which would be blacklisted, and why?   
E.g. can somebody expand on the security problem with CONNECT?
	- why is CONNECT always bad, aren't there cases where it's useful
	- why that security problem can't be dealt with through other client  
approaches (e.g.  explaining that the client code for XHR MUST NOT  
use CONNECT in certain ways)?

Thanks,
Lisa

---

From: Jamie Lokier <[email-masked]>
Date: Sat, 10 Jun 2006 23:19:42 +0100

> use CONNECT in certain ways)?

Perhaps the main thing is ensuring that proxies and servers can parse
HTTP message boundaries properly, so that requests from different
applications can't interfere with each other.

If request message boundaries are not maintained properly,
XMLHttpRequest can subvert connections from user-agent to an HTTP
proxy.

That can cause interference between different applications that should
have nothing to do with each other.  They might even be to different
servers, via the same connection to a proxy.  That is a real security hole.

The CONNECT method is special by virtue of having a different
effective syntax than all other methods - the octets sent after the
request are, effectively, part of the request.

No other methods have this property, and none will in future - for
compatibility with existing proxies and servers.  So a blacklist
ruling out CONNECT and no other methods makes sense.

Another way to get a similar effect to CONNECT is using the Upgrade
header, described in RFC2817 and reserved in RFC2616.

Therefore to prevent subversion of HTTP message boundaries,
XMLHttpRequest should prevent:

    - The CONNECT method
    - Setting the Upgrade header

I don't see any reason to disallow any other request methods.

By the way, it would be nice in future to be able to open a
bidirectional unbounded stream sometimes.  I suspect that's what you
had in mind when you said "[CONNECT], aren't there cases where it's
useful".  However, CONNECT with XMLHttpRequest is not usable for that
purpose.

-- Jamie

---

From: "Joe Gregorio" <[email-masked]>
Date: Sat, 10 Jun 2006 22:48:29 -0400

On 6/10/06, Mark Baker <[email-masked]> wrote:
> that all other methods would be disallowed.

The 'white list' approach is similar to the approach taken
by HTML forms which allows only GET and POST and which
has been disastrous, impeding progress on full usage of HTTP and
hobbling other specs that came later that tried to use methods
beyond GET and POST such as WebDAV. Please don't use a white-list.

> Others would prefer a

That would be a much better approach, and easier to explain, since
it matches the 'blacklist' approach taken by the XMLHttpRequest
specification with respect to HTTP headers.

  -joe

-- 
Joe Gregorio        http://bitworking.org

---

From: Jamie Lokier <[email-masked]>
Date: Sun, 11 Jun 2006 05:06:06 +0100

> I don't see any reason to disallow any other request methods.

Come to think of it, what about TRACE?

Google for TRACE and XMLHTTP.  The top results reveal some cross-site
scripting vulnerabilities whereby a script can deduce cookie values
that it shouldn't by using TRACE with Microsoft's equivalent to
XMLHttpRequest.

However Googling for TRACE and XMLHttpRequest, the top results reveal
that TRACE is usefully used.

-- Jamie

---

From: "Anne van Kesteren" <[email-masked]>
Date: Sun, 11 Jun 2006 10:34:39 +0200

> beyond GET and POST such as WebDAV. Please don't use a white-list.

The problem is that new methods (similar to TRACE and CONNECT) could be  
introduced which would require immediate patching to browsers. For these  
reasons, at least Opera and Internet Explorer 7 establish a white list at  
the moment.

(Please don't copy me on replies, I'm already subscribed to the list.)

-- 
Anne van Kesteren
<http://annevankesteren.nl/>
<http://www.opera.com/>

---

From: Jamie Lokier <[email-masked]>
Date: Sun, 11 Jun 2006 13:27:53 +0100

Anne van Kesteren wrote:
> introduced which would require immediate patching to browsers.

Are you sure it's possible to introduce new methods that have similar
problems to TRACE and CONNECT?

New methods must be compatible with existing deployed proxies, which
means they have to be end-to-end don't they?  And isn't the security
of end-to-end requests a matter between the client and server
applications?

On a related but different note:

XMLHttpRequest does not allow cross-domain requests.

People are working around this constraint by using the old methods of
IFRAME and fetching Javascript, because it is not (usually) subject to
the same restriction.

That means scripts are able to send GET and POST requests to any
domain, but they have limited control over the method and headers sent
and cannot examine the headers received.  And parsing the returned
value and being notified of when it's received are more difficult.

Therefore how about this proposal:

Relax XMLHttpRequest's constraints slightly to allow GET (only)
requests to any domain, with the constraint that in this case it's not
permitted to set arbitrary request headers or read most of the
response headers.  (Reading "Content-Type" should be allowed).

-- Jamie

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sun, 11 Jun 2006 15:29:53 +0200

>problems to TRACE and CONNECT?

Of course it is. There may be problems, but it certainly is possible.

>response headers.  (Reading "Content-Type" should be allowed).

Well, A is your client with a fixed IP, B grants access to A but no
one else, C wants data from B. To achieve that, you simply have to be
tricked into visiting a page on C, which is rather trivial. The only
way to prevent that is to deny (indirect) read access from C to A.

http://lists.w3.org/Archives/Public/public-webapi/2006Jun/0012 and
http://www.w3.org/TR/access-control/ might be interesting to you.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: +49(0)621/4309674 · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Mark Nottingham <[email-masked]>
Date: Sun, 11 Jun 2006 09:29:55 -0700

> of end-to-end requests a matter between the client and server

Exactly. New HTTP methods have to be backwards-compatible (i.e.,  
proxies have to be able to just pass them through like POST), unless  
you rev the *major* version number of HTTP itself (see RFC2145).

Lisa posed the most relevant question, IMO; if some uses of POST (and  
GET for that matter) are good and some are bad, and likewise some  
uses of CONNECT and TRACE are good, and some are bad, is it  
consistent or even a good idea to try to regulate them at such a  
coarse level (in an API standard)?

Also, AFAICT, all of the purported security problems (which are  
*still* hazy) occur with cross-domain XHR, when the spec at hand only  
specifies single-domain XHR.

--
Mark Nottingham     http://www.mnot.net/

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sun, 11 Jun 2006 19:09:32 +0200

>specifies single-domain XHR.

If you have a single host with a single entity responsible for all
script logic and the server setup is is indeed relatively safe to
assume this entity would not write script logic that poses risks
for the host or its users.

However, it is not safe to assume that this is always the case. As
an example, script injection vulnerabilities are rather common. It
is also common to attempt to work around such vulnerabilities, for
example "HttpOnly Cookies" are supposed to prevent cookie stealing
even if there is a script injection vulnerability.

And they do that, until you inject a script that does a TRACE. The
same goes for HTTP authentication credentials, they are not available
to scripts, so even with a script injection vulnerability you don't
have access to them, until you do a TRACE.

Also note that having a script injection vulnerability does not
necessarily mean the entity responsible for the web service made
any obvious mistake; browser vendors keep adding new features that
make previously safe codes unsafe and service providers have a
hard time to keep up with new threats.

You'll have to look at this from the user perspective. A browser
that does not allow (for example) TRACE requests is less risky to
use than one that does allow it. The question for browser vendors
is then "Do we want to make a product that's more risky to use
than competing products?"
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: +49(0)621/4309674 · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: "Roy T. Fielding" <[email-masked]>
Date: Sun, 11 Jun 2006 11:31:27 -0700

> than competing products?"

That is absolutely absurd.  How about making a product that obeys
the relevant standards?

As I've stated numerous times on this list, the only reason there
is any concern about TRACE is because one company that "specializes
in security" used it as a marketing promotion to point out a
bug in an old version of MSIE.  There is no reason for browsers
to block TRACE.

What browsers need to do is obey the specs.  TRACE is not a problem.
If the browser sends already-non-secure cookies on a TRACE request,
then the response is going to contain those cookies.  So don't do that.
Don't whitelist methods -- whitelist the information the browser is
allowed to send in *any* request.  Allow the user to configure
the browser differently when new information is needed.

CONNECT is a protocol switch.  The only real risks for CONNECT are
for proxies that are tricked into connecting to the reserved ports
of older protocols.  For that reason, good systems limit CONNECT
requests to the well-known SSL-using ports and do not send things
like cookies in the CONNECT request.  Again, this should be configurable
by the user.

All HTTP methods fall into safe/unsafe categories.  It is absolutely
necessary that the browser distinguish between safe requests
(hyperlinks) and unsafe requests (a la buttons).  That is a browser
GUI issue and, for scripting, requires a user ack prior to sending the
unsafe request.  By default, all unknown methods should be considered
unsafe until the user configures otherwise.  There is no need for the
*protocols* to define whitelists when the developers can do so on
their own, based on their own applications, and according to their
own user needs.

....Roy

---

From: Jamie Lokier <[email-masked]>
Date: Sun, 11 Jun 2006 21:06:12 +0100

Roy T. Fielding wrote:
> then the response is going to contain those cookies.

Eh?  What makes a cookie already-non-secure, as opposed to secure cookies?

The reasons given for blocking TRACE are to protect against a certain
type of cross-site scripting attack.  This is the example:

Site A is a wiki/blog which does not adequately sanitise content
posted to it.  There is no shortage of these.

A user logs on to site A, where a script maliciously posted loads into
the user's browser and starts doing things it shouldn't.  One of those
things is to send a TRACE to site A, look at the cookies returned
(cookies used to login to the site), and post those cookies to another
page on site A.

The hacker who posted the script can then retrieve the user's cookies,
and use them to impersonate the user at other times, if the cookies
are used as login credentials, which is of course common.

Your suggestion seems to be that the browser should allow TRACE but
not post cookies with the TRACE request.  There are two approaches to
that:

    1. The browser does not post cookies with _any_ XMLHttpRequest.
       That's very unhelpful - they are useful.

    2. The browser has a special rule for TRACE.

The third approach, which I suspect you have in mind, is that site A
should properly sanitise whatever is posted to it.  If site A doesn't
do that well enough, it's site A's own security hole.

Indeed, the fact that a malicious script can post to the site
indicates a serious hole.

But our job as designers of the Javascript sandbox is to limit the
damage which can be done by a script.

Being able to post to a site is one type of damange, which might be
considered serious or not, depending on the application.  Being able
to post the user's login credentials for someone else to use is
perhaps more serious.

That's the motivation for prohibiting TRACE, or for limiting the
headers which will be sent with TRACE to those which don't reveal
security information.  It's a type of damage limitation along a fairly
clear boundary: don't allow a script direct access to security credentials.

> So don't do that.  Don't whitelist methods -- whitelist the
> information the browser is allowed to send in *any* request.  Allow
> the user to configure the browser differently when new information
> is needed.

Perhaps.  But it's clear that ordinary cookies are useful with
XMLHttpRequest GET/POST requests.

> of older protocols.

No, there is another risk which is equally serious.

Example: Site A is a dodgy site which someone visits (let's say by
accident).  Site B is the user's bank.

The user visits site A, which sends a page containing a script which
uses XMLHttpRequest to send a CONNECT request.  The request is
processed by a proxy between the user and site A, and the proxy
processes it by connecting to port 443 on site A, successfully.

The user (shocked at the pictures) quickly closes their window, then
later connects to site B.

The browser then sends requests to site B *using the same connection
to their proxy*.  Persistent HTTP does this.

This assumes the browser did not recognise XMLHttpRequest + CONNECT as
different from other request methods.

This means the browser sends subsequent requests for site B to site A.
Site A can proxy them (silently) to site B, and thus accomplishes a
stealth phishing attack.  Nothing shows in the URL bar; only sniffing
the network would reveal the requests are hijacked.

There's a couple of obvious ways to prevent this:

    1. Don't allow CONNECT with XMLHttpRequest.

or

    2. Ensure the browser treats CONNECT differently than other
       methods with XMLHttpRequest: by marking the connection as
       non-reusable.

> their own, based on their own applications, and according to their

I agree with you, that the browser GUI should be the place where
ultimately whitelist/blacklist/whatever policy is configured.  This is
no different than, for example, whether a script is allowed to access
local files, or do cross-domain requests.

However I think it makes good sense for the XMLHttpRequest
recommendation to _recommend_ a common policy.

After all the whole point of the recommendation document is to provide
some common ground where people know what is likely to work (or will
when browsers claim compliance with the recommendation) without user
intervention, and what is not likely to work in that way.

In those terms, a whitelist makes sense.  So something like this:

    Implementations which conform to this recommendation support at
    minimum GET and POST requests to the same domain without user
    intervention.  Cookies, Accept etc. will be sent with these
    requests as with ordinary page requests.  (Script authors can
    depend on this).

    Implementations MAY allow other request methods, MAY allow
    cross-domain requests, MAY require user intervention to enable any
    of these things, and MAY apply implementation-defined restrictions
    on what combinations are allowed.  However, implementations do not
    have to allow any of these things to conform to this
    recommendation.

-- Jamie

---

From: "Anne van Kesteren" <[email-masked]>
Date: Sun, 11 Jun 2006 22:27:32 +0200

> problems to TRACE and CONNECT?

I'd rather have someone prove the opposite given the nature of the issue.  
Anyway, I'm by no means a security or HTTP expert, just saying what the  
problem is at the moment.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Sun, 11 Jun 2006 22:52:00 -0700

> Eh?  What makes a cookie already-non-secure, as opposed to secure  

All cookies are non-secure.  Using them for security purposes (like
access control) is just begging for security holes.

> are used as login credentials, which is of course common.

Yes, the world is full of people who don't care about security.

>        That's very unhelpful - they are useful.

Sure, they are useful for poorly designed sites that expect to receive
GET and POST requests.  They might even make sense for PUT and DELETE
and a few other methods.  But arbitrary methods that the browser has
no clue about the semantics?  Why would any client software want to
send supposed "security credentials" like cookie on a method without
knowing the semantics?

>     2. The browser has a special rule for TRACE.

Yes.

> do that well enough, it's site A's own security hole.

Yes, but that wasn't my point.  The point is that TRACE is a valuable
method and must be implemented by servers to have value.  The fact
that one browser sends cookie information on arbitrary requests is
not the fault of TRACE -- any new method might have the effect of
storing the headers on a public site, or perhaps broadcasting them
to an open network.  Fix the browser.

> to their proxy*.  Persistent HTTP does this.

No it doesn't.  The CONNECT request to a proxy terminates the HTTP
part of the connection -- it is no longer a proxy connection but a
tunnel.  If the browser sends later HTTP requests to that tunnel
then it is sending them *knowingly* to site A and the browser is
obviously broken.  Easy to fix, no need to change the protocol.

> different from other request methods.

That isn't possible -- the syntax of the CONNECT request is different
from that of other requests.  Any client that didn't understand it
would send an invalid message that the proxy will reject.

>        methods with XMLHttpRequest: by marking the connection as

Yes, that is how it was designed by Ari.

> recommendation to _recommend_ a common policy.

A browser represents its user on the Internet.  Any browser that sends
a method that it does not understand, without explicit consent from
the user, is entirely broken.  That is what the recommendation should
say.  Methods are designed to be extensible and modular, not safe.
A browser can implement new methods by allowing the user to install
extensions that understand those methods.  A browser should never
execute arbitrary, unapproved methods, for the same reasons that a
browser should not execute arbitrary code.

>     requests as with ordinary page requests.  (Script authors can

No, POST requests are inherently unsafe.  They cannot be made without
user intervention -- it violates all the other Web specs.

>     have to allow any of these things to conform to this

That is a waste of space.  The spec should say why methods exist and
that only known safe methods can be used without user intervention.
(Intervention includes such things a specific configuration prior
to running the application, not just pop-up boxes.)
That is what HTTP and HTML already requires.  What it should not do
is list a small set of methods and say implementations MUST (NOT)
implement them -- that is none of your business and simply sets up
the implementers to be fooled by unexpected extensions.

....Roy

---

From: "Sylvain Hellegouarch" <[email-masked]>
Date: Mon, 12 Jun 2006 08:18:56 +0100 (BST)

> the implementers to be fooled by unexpected extensions.

I agree here. However it seems the WEB-API is more interested in
documenting the choices made by browser vendors regarding issues such as
security or the way they interpret HTTP rather than producing a higher
level specification that would follow the spirit of the HTTP RFC which
would push them to modify their implementation.

Basically, I've seen a few times now some regular of the WEB-API group
saying "we've been doing that way for a while now. It works so there is no
reason to change and break implementations".

Therefore the ultimate purpose of that WG is fairly unclear to me.

- Sylvain

---

From: Stefan Eissing <[email-masked]>
Date: Mon, 12 Jun 2006 09:53:26 +0200

> http://www.w3.org/TR/access-control/ might be interesting to you.

Help me understand: The use cases for a more open XHR are motivated  
by application/server developers, right? Reading the above piece  
about access-control, it appears to me that it would not be an  
obstacle to implement something at the server side.

What I mean is that XHR would have the following behavior:
- Implement a "whitelist" of methods and uses which are known to be  
"safe"
- For all methods outside of this, let XHR ask the server if it ok.  
For example, let XHR send an OPTION request and look for an XHR-Allow  
header, listing the methods allowed to XHR. (or whatever, the key is  
that the server is in control)

Seems to me that this approach puts server application developers in  
the driver seat and lets browser developers stay safe by default, no  
matter what future http will bring.

//Stefan

---

From: Stefan Eissing <[email-masked]>
Date: Mon, 12 Jun 2006 11:12:30 +0200

Am 12.06.2006 um 10:47 schrieb Julian Reschke:

> Stefan Eissing schrieb:
>> ...
>> - Implement a "whitelist" of methods and uses which are known to  
>> be "safe"
>> - For all methods outside of this, let XHR ask the server if it  
>> ok. For example, let XHR send an OPTION request and look for an  
>> XHR-Allow header, listing the methods allowed to XHR. (or  
>> whatever, the key is that the server is in control)
>> Seems to me that this approach puts server application developers  
>> in the driver seat and lets browser developers stay safe by  
>> default, no matter what future http will bring.
>
> Can you give an example where a server that implements method X  
> would return it in the "Allow" header, but not in the "XHR-Allow"  
> header?

Sure. A server allowing POST for ordering you a book, but not  
allowing it from XHR requests from pages coming from a different site.

The last part is the key, of course. I am assuming that methods  
against the originating server of a page are always allowed and that  
we are talking about securing requests to other servers and methods  
used in them. Please correct me, if I got this wrong.

//Stefan

---

From: Julian Reschke <[email-masked]>
Date: Mon, 12 Jun 2006 10:47:08 +0200

Stefan Eissing schrieb:
> - Implement a "whitelist" of methods and uses which are known to be "safe"
> - For all methods outside of this, let XHR ask the server if it ok. For 
> example, let XHR send an OPTION request and look for an XHR-Allow 
> Seems to me that this approach puts server application developers in the 
> driver seat and lets browser developers stay safe by default, no matter 
> what future http will bring.

Can you give an example where a server that implements method X would 
return it in the "Allow" header, but not in the "XHR-Allow" header?

Regards, Julian

---

From: "Anne van Kesteren" <[email-masked]>
Date: Mon, 12 Jun 2006 11:42:14 +0200

On Mon, 12 Jun 2006 11:12:30 +0200, Stefan Eissing  
<[email-masked]> wrote:
> The last part is the key, of course. I am assuming that methods against  
> the originating server of a page are always allowed and that we are  
> talking about securing requests to other servers and methods used in  
> them. Please correct me, if I got this wrong.

You got this wrong. The discussion here is about (the first version of)  
XMLHttpRequest which will only allow same-origin requests.

---

From: Stefan Eissing <[email-masked]>
Date: Mon, 12 Jun 2006 12:12:26 +0200

Am 12.06.2006 um 11:42 schrieb Anne van Kesteren:

>> against the originating server of a page are always allowed and  
>> that we are talking about securing requests to other servers and  
>> methods used in them. Please correct me, if I got this wrong.
>
> You got this wrong. The discussion here is about (the first version  
> of) XMLHttpRequest which will only allow same-origin requests.

Thanks for the correction. Well, in that case I agree with Roys  
comment that instead of restricting methods it is superior to  
restrict the (manipulation of) information send to the server. So,  
basically a whitelist of settable/sent headers with some name prefix  
("x-"?) left open for individual applications/experimentation.

//Stefan

---

From: "Anne van Kesteren" <[email-masked]>
Date: Mon, 12 Jun 2006 12:22:19 +0200

> would push them to modify their implementation.

I would like to hear some specific examples as I'm not really convinced  
this is the case. For example, things like conditional GET are not  
implemented in browsers, but are pointed out in the specification. Same  
for transparently following redirects (just talking about XMLHttpRequest  
here).

> saying "we've been doing that way for a while now. It works so there is  
> no reason to change and break implementations".
>

The purpose of the WG is to standardize (existing) scripting APIs in a way  
that doesn't break them as used on the web today, but also to allow new  
things to be added and make them work better with existing specifications  
(when already deployed).

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 12 Jun 2006 12:15:41 +0100

> access control) is just begging for security holes.

Yes and no.  They're less secure in the sense that they're not
typically stored as carefully.  They're more secure in the sense that
with cookies a site can remove access under the site's control by
removing access from a particular cookie value, e.g. to implement a
login timeout policy, explicit logout button, single-client access or
whatever the site's policy.

It's no coincidence that almost every site on the net uses cookies for
access control, rather than HTTP user/password.  It certainly is not
due to lack of security considerations.

> send supposed "security credentials" like cookie on a method without

It's for the server to determine what constitutes a "credential", not
the client.

> >    2. The browser has a special rule for TRACE.

I agree :)

> would send an invalid message that the proxy will reject.

Ah.  Point to you.

Valid CONNECT looks like:
    CONNECT some.domain HTTP/1.1

whereas valid GET/POST/etc look like:

    GET /some/path HTTP/1.1

or

    GET http://some.domain/some/path HTTP/1.1

There is no other difference in the requests - headers can be the same.

The presence of "/" somewhere in the request-URI makes all the difference.

Seems a tad delicate that security hinges on that, but it works.

> A browser should never execute arbitrary, unapproved methods, for
> the same reasons that a browser should not execute arbitrary code.

The entire _point_ of XMLHttpRequest is to support the browser
executing arbitrary code with unknown effects with limitations -
scripts supplied by a server, in a sandbox.

> user intervention -- it violates all the other Web specs.

Specs: If anything, the necessary and common use of GET to send
state-changing messages to a resource from scripts is a violation of
HTTP specs.  I would have thought POST was the appropriate way to do
_all_ state-changing messages to a resource: they are messages sent to
the resource for processing after all, not messages to retrieve the
resource.

As RFC2616 says: GET: "retrieve whatever information (in the form of
an entity) is identified by the Request-URI", POST: "Providing a block
of data, such as the result of submitting a form, to a data-handling
process".

Most web apps use GET to perform the latter function.  If POST
required user intervention per message, they would _have_ to use GET
to send state-changing messages asynchronously.  Maybe violating
specs, but it's the whole reason for XMLHttpRequest existing.

If by user intervention you mean something less intrusive like
"configure the browser to allow scripts to use POST", that would be a
rather misleading use of "user intervention".  Every browser feature
including "running scripts" and "allowing GET" is equally covered by
that.

Security: POST requests are no more or less safe than GET requests,
when applied to a resource with known behaviour.  Since that's the
point of XMLHttpRequest (and the cross-domain restriction), no
problem.  They can and do both have insecure side effects when applied
to a resource with unknown behaviour.

> The spec should say why methods exist and

It would be daft for the recommendation to say nothing at all about
what methods web site authors should expect to be commonly supported.

I think that would be ignoring reality.  The entire practical purpose
of XMLHttpRequest is to permit the deployment of applications with
minimal or no user configuration.

In particular as you raise the point of POST requiring "user
intervention", if there is really is a need for user intervention the
recommendation ought to give guidance on what that means.

> the implementers to be fooled by unexpected extensions.

I agree.

-- Jamie

---

From: "Sylvain Hellegouarch" <[email-masked]>
Date: Mon, 12 Jun 2006 13:02:42 +0100 (BST)

> due to lack of security considerations.

Could not this be done via Digest Auth if browsers provided an API
(understand via accessiblity from Javascript) allowing web app developers
to nicely integrate the login/password fields in their page instead of
that ugly popup all browsers use?

- Sylvain

---

From: Julian Reschke <[email-masked]>
Date: Mon, 12 Jun 2006 15:05:58 +0200

> recommendation ought to give guidance on what that means.

I guess what it means is that a user agent should not send a POST 
request unless the user has clicked on a button (something that stands 
out from a regular link). That's also why allowing form.submit or 
XHR.send() from within a script running "onload" is problematic.

Best regards, Julian

---

From: David Morris <[email-masked]>
Date: Mon, 12 Jun 2006 10:05:47 -0700 (PDT)

> that ugly popup all browsers use?

I've forgotten ... did digest authentication end up solving the logoff
issue? There have been two classic issues related to using cookie/urlfield
based authentication:

  a.  Having a decent dialog which the end user can relate to, can
      include "forgot your password", etc.
  b.  Support of user logoff
an additional feature which can be implemented is periodic
re-authentication.

---

From: David Morris <[email-masked]>
Date: Mon, 12 Jun 2006 10:32:50 -0700 (PDT)

My first paragraphs are in response to the broader thread and not
Julian's specific comment ...

I'm so tired of protocol designers believing that if they dictate that
the user somehow be forced to approve an action, that users can be then
faulted for their stupidity when accepting a risky choice.

Informed consent really means being informed. Being informed means that
sufficient information must be provided AND that the information is
understandable by the average user. Protocols must be designed with
sufficient information capability to satisfy this requirement.

> XHR.send() from within a script running "onload" is problematic.

This is a specific case where the protocol design has ignored the
realities of application user interaction design. I have found
no alternatives to form.submit() within onload to achieving rich
and smooth applications.

Single domain restrictions, with clear definition of a domain and some
ability to over-ride the restriction.

a. The C++ notion of friend classes provides a decent paradigm for a
server to indicate partner domains. For example, merchant account and
credit card payments are often handled by a different domain.
b. Certain large web content providers have a pool of domain names which
can't satisfy any simple pattern match. google, yahoo, cnn, amazon are all
examples where their international servers may end up with local domain
names which need to intermingle with other local domains.
c. At the other end of the scale, small web providers are virtually hosted
on a shared server. Some aspects of the individual web sites may share
logic with the host.

Better to provide a clean way to handle common system issues than to
force creative hacks, false naming, etc. All of which lead to instability
and security issues.

Dave Morris

---

From: Julian Reschke <[email-masked]>
Date: Mon, 12 Jun 2006 19:45:40 +0200

> no alternatives to form.submit() within onload to achieving rich

Now that's an interesting statement. It would be nice if you could 
expand on this, because it would be useful to understand what kind of 
use cases are solved using this method.

Best regards, Julian

---

From: Stefan Eissing <[email-masked]>
Date: Mon, 12 Jun 2006 20:57:16 +0200

Am 12.06.2006 um 19:45 schrieb Julian Reschke:

> expand on this, because it would be useful to understand what kind  
> of use cases are solved using this method.

I am sure that David can speak for himself, however the benefits of  
IFrames over XmlHttpRequest are discussed in the "Comet" web pattern.  
For example here:

http://softwareas.com/index.php?s=comet

//Stefan

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 12 Jun 2006 20:01:24 +0100

> XHR.send() from within a script running "onload" is problematic.

What about a web-based instant messaging client, where the user types
some text and it must be relayed to the server as it is typed, perhaps
character by character?

There are basically two ways to do a "web application" like that now:
GET or POST with client-side scripts.

My reading of the HTTP spec. says that POST should be used for those
messages, because it's not fetching a resource, it's sending data to a
resource to be processed.

In practice of course, the author will use whatever works, which often
has to be GET for technical reasons.

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Mon, 12 Jun 2006 21:19:45 +0200

> has to be GET for technical reasons.

Of course I'm not suggesting to use GET instead of POST (in case anybody 
thought that :-).

A key press *is* a user interaction, so sending a POST upon pressing a 
key doesn't seem to be a problem to me.

What I'm concerned with is people following a link, opening a web page, 
and a script doing a POST without *any* further user interaction.

Best regards, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 12 Jun 2006 20:33:51 +0100

> http://softwareas.com/index.php?s=comet

Funnily enough I was working through the details last night of
streaming messages to a client asynchronously.  (For example, like in
an instant message client, live news feed, shared whiteboard, etc.)

I came to the same conclusion as Comet: you have to use an IFRAME and
stream SCRIPT elements to it as each message is to be transferred.

Technologically it doesn't look pretty, but the mess is really
confined to one small thing: the code responsible for streaming
messages.  The rest of the application code is decoupled from this
mechanism.

It's interesting that this method is quite portable, works with very
old browsers, and is close to optimal in terms of network performance
and responsiveness.  (Compare with polling using XHR every few
seconds, as some people do).

There's actually nothing to be gained by using XMLHttpRequest on
browsers that support incrementally viewing the response (not IE), and
plenty to lose.

Ways in which you lose with XHR are: you'd still have to poll the XHR
object often; xhr.responseText might get progressively slower if it
always re-parses the received bytes into characters; you can't easily
tell if a browser's XHR is incremental except by sniffing the User-Agent.

(Btw, Gecko and Microsoft's XHR objects do support some kind of stream
interface, but I don't think they're accessible from Javascript.)

Note that the IFRAME + streamed SCRIPT method is only needed for web
apps that need to receive asynchronous messages from the server -
without continuously polling the server.  XHR is fine for apps which
only need to fetch something immediately following a user action.

It leaves me wondering if XHR should be extended to support streaming
properly, in a standard way...

Unfortunately the whole idea is quite at odds with the "official" way
HTTP and HTML are supposed to be used.  But then we lost that as soon
as the DOM become accessible, and people noticed you can make fancy
interactive applications.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 12 Jun 2006 20:47:58 +0100

> key doesn't seem to be a problem to me.

Ok, so are you saying that from a technical point of view, a typical
browser will simply allow scripts to perform POST requests?

Because it's hard to see how anything but the script can determine
what counts as a user interaction for a specification application.
Mouse gestures, keys, resizing the window, are all examples.

> and a script doing a POST without *any* further user interaction.

Perhaps.  It seems analogous to popup windows: popups in response to
clicking something are ok.  Popups when you visit a page, leave a
page, or from a timer, are annoying and good browsers block them.

How do we draw a useful line between what counts as a user
interaction, and allowing "web applications" a rich set of interaction
methods which do count as user interactions for this purpose?

If it's to be specified, be careful, as the Firefox folks had a few
learning iterations before they got it about right for popups.

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Mon, 12 Jun 2006 22:01:36 +0200

> Mouse gestures, keys, resizing the window, are all examples.

 From a user's point of view, I would count pressing keys as 
interaction, but not moving the mouse (I may be old-fashioned here), or 
resizing the window.

> page, or from a timer, are annoying and good browsers block them.

Exactly.

> learning iterations before they got it about right for popups.

I think we at least are now asking the right question :-)

Best regards, Julian

---

From: David Morris <[email-masked]>
Date: Mon, 12 Jun 2006 18:40:35 -0700 (PDT)

> use cases are solved using this method.

Application Example: You have two frames where one depends on the other in
  some fashion. If the onload for FrameA initiates the load of FrameB,
  FrameB logic can be sure that FrameA data objects are present.
Application Example 2: The application has a main window and a data
  entry dialog. When the user input from the dialog is commited, the
  main window requires a reload. The onload for the dialog window
  initiates the refresh of the main window by calling a javascript
  method in that windows and then closes the dialog window.

Misc. Example: Timing web page loads in the style of www.numion.com's
stopwatch facility requires careful coordination of loading two visible
frames after the base frameset is loaded. Use of onload to trigger the
next step provides that synchronization.

Dave Morris

---

From: "Sylvain Hellegouarch" <[email-masked]>
Date: Tue, 13 Jun 2006 08:53:24 +0100 (BST)

> I think we at least are now asking the right question :-)

Indeed. However when I see the different directions taken by this thread,
I wonder if the question would not be more to clearly define what is
expected from web applications.

This discussion focuses on the XMLHttpRequest object and tries to use its
features to design web applications. But XMLHttpRequest is a tool. When I
see comparing IFrames to XMLHttpRequest, I see a discussion around "let's
find a hack to work around HTTP principles", not a discussion of "How
should we make HTTP evolving in the coming years so that it can cope with
what the Web is heading to?".

Folks who wrote the HTTP protocol designed it so well that today HTTP is
the most spreaded and used protocol. XMLHttpRequest and IFrames are not
protocols, they are tools and they will be eventually replaced by
yet-another-marketed-technology in the next few years. HTTP will still be
there.

It really sounds like no one is ready to undertake a process of updating
HTTP and prefer to patch around it and see no further than the coming
quarters which will eventually lead to a big mess of piled hacks.

- Sylvain

---

From: Stefan Eissing <[email-masked]>
Date: Tue, 13 Jun 2006 10:07:09 +0200

> expected from web applications.

I see your point. But such a architectural view is a luxury to people  
building web applications today.

I would not refer to XHR usage as a hack. The effort underway to  
define its behavior and security restrictions is removing any  
hackiness from its origin and giving webapp developers a stable API  
to work on. That is great.

The purpose of this thread was to check with a HTTP architectural  
view a specific detail of XHRs security considerations, namely how to  
handle "unknown" methods in the context of "same server" requests.

I think Roy pretty much made the points from HTTP point of view and I  
have not seen anyone arguing against it. (Which for some strange  
reason, rarely happens to Roy...)

---

From: "Sylvain Hellegouarch" <[email-masked]>
Date: Tue, 13 Jun 2006 09:13:35 +0100 (BST)

> building web applications today.

I can totally acknowledge that deciding to update HTTP is not something
that will pay off anytime soon. However I do hope at some point this will
happen.

> to work on. That is great.

Fair enough.
Let's agree that XHR is not a hack. What some people try to achieve with
it goes beyond its purpose (incidently showing that there is a real
potential for an update of the Web architecture) and this often becomes
ugly.

> handle "unknown" methods in the context of "same server" requests.

Point taken.

> reason, rarely happens to Roy...)

:)

Thanks for the feedback,
- Sylvain

---

From: Julian Reschke <[email-masked]>
Date: Wed, 14 Jun 2006 16:33:18 +0200

> next step provides that synchronization.

David, maybe I'm a bit slow, maybe there's a misunderstanding...

What I was saying is that calling an *unsafe* method such as POST upon 
"onload" is problematic. I didn't argue against "onload" in general. So 
do these use case require an unsafe action being performed 
automatically, or wouldn't GET be sufficient here?

Best regards, Julian

---

From: David Morris <[email-masked]>
Date: Wed, 14 Jun 2006 09:35:54 -0700 (PDT)

> automatically, or wouldn't GET be sufficient here?

Well, the notion of unsafe is really an application notion. Browsers
protect applications by attempting to warn the user before repeating a
POST which might repeat a data base change. The application designer needs
to make the decision as to whether POST or GET is appropriate. Because of
the wide variety of GET / no-cache / etc. implementations, I have much
more faith that a POST response will not be re-used than I do for a GET
response. So based on application requirements, the decision to use POST
or GET should be the application designers. There are also payload issues
between GET and POST.

I see no reason to distinguish between POST and GET in the case of onload
or other scripted execution of HTTP requests. My applications make heavy
use of the ability to process an ONCLICK event on an anchor to submit the
form. Perhaps because I want a pretty button or because a set of buttons
just makes a crummy layout. For example, we had several possible actions
on an object. As links, the actions laid out well, took minimal space,
etc. Use of buttons would have been horrible.

I have noted a rather significant move to this style of command object
on desk top guis as well.

POST is unsafe to repeat without user interaction (unless the experimental
SAFE rfc provisions are implemented). There should be no additional
implications related to POST.

Dave Morris

---

From: "S. Mike Dierken" <[email-masked]>
Date: Wed, 14 Jun 2006 18:41:04 -0700

> 
> Well, the notion of unsafe is really an application notion. 
> Browsers protect applications by attempting to warn the user 
> before repeating a POST which might repeat a data base 
> change. The application designer needs to make the decision 
> as to whether POST or GET is appropriate. 
The WWW is the application. 
It has already made the decision that POST is unsafe (modifies data) and GET
is safe (does not modify data and is predictably repeatable).

---

From: David Morris <[email-masked]>
Date: Wed, 14 Jun 2006 19:15:22 -0700 (PDT)

I disagree ...

www is the protocol not the application.

'unsafe' is, as you say, an indicator of how data updates might be
handled. There is no fundamental unsafeness outside of the context of the
web server receiving the POST. Only that server developer knows if there
is any negative impact of repeated use of POST and/or there is any other
reason why POST must be used. Unsafe merely means that the user agent may
not assume that repeating the post is safe. One way the server developer
can answer the question is to provide javascript as part of the web
content which repeats the post without bugging the user because the web
developer knows it is safe.

---

From: "S. Mike Dierken" <[email-masked]>
Date: Wed, 14 Jun 2006 20:15:16 -0700

> 'unsafe' is, as you say, an indicator of how data updates 
> might be handled. There is no fundamental unsafeness outside 
> of the context of the web server receiving the POST. Only 
> that server developer knows if there is any negative impact 
> of repeated use of POST and/or there is any other reason why 
> POST must be used. Unsafe merely means that the user agent 
> may not assume that repeating the post is safe. One way the 
> server developer can answer the question is to provide 
> javascript as part of the web content which repeats the post 
> without bugging the user because the web developer knows it is safe.
> 
<pedantic>http is the protocol. WWW is the application that results from the
use of that protocol</pedantic>

> Only that server developer knows if there is any negative impact 
> of repeated use of POST and/or there is any other reason why 
> POST must be used. 
Actually, any client developer knows that POST is unsafe because the
protocol specification says that it is unsafe (and also not repeatable). By
the same token, all developers know that GET is safe and repeatable, PUT is
unsafe but repeatable and DELETE is unsafe but repeatable.

> Unsafe merely means that the user agent may not assume that repeating the
post is safe. 
Unsafe means that the /first/ submission is not safe, not to mention
repeated submissions.

If you need to use POST and you want to ease the user experience in the face
of resubmitting the request, then script trickery works as an enhancement
but is not a general solution. Which isn't a big deal for nearly all web
(page) applications.

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 15 Jun 2006 11:57:32 +0100

> but is not a general solution. Which isn't a big deal for nearly all web

This is all very abstract.  It's quite clear how it makes sense to
traditional documented-oriented or page-oriented web sites, with
feedback through traditional forms.

But for highly interactive sites, with lots of site-created "widgets"
on the page, where the client sends a stream of messages to the
server, and the server sends a stream of messages to the client, there
are practical technical reasons why people are having to us GET when
POST should be used in theory, and using POST when GET should be used
in theory.

If it's mandated that, say, a POST initiated by XMLHttpRequest will
only succeed in response to a "user interaction", it really ought to
be defined what that means, because if that means "only accepted if
there was a mouse click or keyboard event triggering the script" or
something like that, people writing those sort of sites will just
carry on using GET for state-changing requests if that gives the
desired behavior in practice.

Here's an example: a site sends a page to the client, designed to
"gracefully degrade" to reasonable HTML on less capable clients.  The
script on that page checks the client's features, decides it can
enable fancy asynchronous messaging functions on that page, and then
sends an iframe GET to update the state associated with this session
and initiate the receipt of messages.

That GET must not involve user interaction - it's behind the scenes to
get the interactive bits of the page working after the page has
loaded.  It's probably idempotent but only the site author knows
whether it is.  And it should be a POST in theory, because it's
causing a state change on the server and it's not fetching a resource
or overwriting a resource.  But it's a GET in practice, because that works.

-- Jamie