Subject: [hybi] straw-poll on GET vs OPTIONS vs new method

From: Salvatore Loreto <[email-masked]>
Date: Fri, 21 Jan 2011 10:01:05 +0100

Hi there,

after the result of the first straw-poll on GET+Upgrade+Masking,
there were a discussion about GET not being the only method to "Upgrade" 
the connection;
other methods such as OPTIONS 
(http://www.ietf.org/mail-archive/web/hybi/current/msg05721.html)
and perhaps a completely new method could be used (as proposed months ago).

All the possibilities have pro and cons.

The mail thread(s) started to discuss the alternatives,
convinced Joe and myself, as HyBi wg co-chairs, that it make sense to 
run a straw-poll
to get as much information as we can  from people about technical reason 
(pro and cons)
for their preference(s).

So please provide your choices (if more then one, please rank them or if 
only one specify
"I can live only with") among the following

#1 - GET
#2 - OPTIONS
#3 - a complete new method (e.g. WEBSOCKET)

This poll will run until January 31st

cheers
/Sal

-- 
Salvatore Loreto
www.sloreto.com

---

From: Dave Cridland <[email-masked]>
Date: Fri, 21 Jan 2011 09:29:16 +0000

> #3 - a complete new method (e.g. WEBSOCKET)

It's not entirely clear to me what the significance of the method is.

In principle, a client makes a request, and includes an upgrade  
token, indicating its preference to fulfill this request by switching  
to a different protocol. If the server is unwilling or unable to make  
this switch, then the request is fulfilled as-is. That's the  
essential point of Upgrade, and so by picking Upgrade as the design,  
we've signed up for that.

So for many requests, a GET or an OPTIONS seems fine, and for others  
a POST might be more suitable - especially if, for instance, the  
request might be fulfilled by BOSH if a WebSocket upgrade fails.

So with that in mind, the existence of this poll somewhat confuses me  
- why do we want to mandate that a particular request method be used?

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Salvatore Loreto <[email-masked]>
Date: Fri, 21 Jan 2011 11:18:09 +0100

> - why do we want to mandate that a particular request method be used?
Hi Dave,

while I agree with your analysis about the usage of Upgrade
and the fact that we could live with different methods

I have to say that nobody till now raised your proposal (to not mandate 
a specific method) clearly,
and indeed the current version (04) as all the previous one mandate the 
usage of GET as the method
for the handshake.
So it seems to me that the poll has already started to show its own 
utility in getting as much information
as possible as we can from people about their preferences as well as 
technical reasons for their preferences

Having said that, as individual, I see different values on mandate a 
particular method to be used
for example
- different methods interact differently with intermediaries; so mandate 
the one that has the highest
chances to fail cleanly (and fast) with upgrade-incompatible 
intermediaries will provide a better user experience
to applications using the websocket protocol
- security...
- reduce the implementation complexity
etc.

cheers
/Sal

-- 
Salvatore Loreto
www.sloreto.com

---

From: Adam Barth <[email-masked]>
Date: Fri, 21 Jan 2011 02:30:27 -0800

> #1 - GET

This option is fine, but slightly less than ideal.

> #2 - OPTIONS

This option is semantically correct, as explained by Roy T. Fielding,
and the most secure, as I've explained before.  Note, I assume you
mean the following request-line:

OPTIONS * HTTP/1.1

> #3 - a complete new method (e.g. WEBSOCKET)

Using a novel method is unnecessary and provides little, if any,
additional benefit.

Adam

---

From: Julian Reschke <[email-masked]>
Date: Fri, 21 Jan 2011 11:38:26 +0100

On 21.01.2011 11:30, Adam Barth wrote:
...
> mean the following request-line:

I don't think this was implied, and furthermore it would be a bad idea 
(because you're losing the request path).

> ...

Best regards, Julian

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 21 Jan 2011 11:45:33 +0100

> (because you're losing the request path).

Agreed, OPTIONS for me means with the path.

Willy

---

From: Salvatore Loreto <[email-masked]>
Date: Fri, 21 Jan 2011 11:45:47 +0100

> (because you're losing the request path).
(for clarification)

#2 - OPTIONS does not imply "*" in the URI.
that can be a possibility but needs to be discussed

cheers
/Sal

-- 
Salvatore Loreto
www.sloreto.com

---

From: Roberto Peon <[email-masked]>
Date: Fri, 21 Jan 2011 04:25:55 -0800

> Agreed, OPTIONS for me means with the path.

I take it that we'll always be including the Host: header (i.e. a HTTP/1.1
request, not HTTP/1.0 or "HTTP/0.9").
If so OPTIONS with path seems fine by me.
-=R

>
> Willy
>
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: Joe Mason <[email-masked]>
Date: Fri, 21 Jan 2011 09:16:17 -0500

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Dave Cridland
>

Using POST to fallback seems like a useful technique to me.  If possible I'd like to support it.  However, if there are reasons to restrict to one other method, this shouldn't take precedence.

I'd suggest the spec allow any method but recommend use of OPTIONS unless the script wants to take advantage of a specific feature of another method (and use POST to fallback to BOSH as an example of such a feature).

Joe

---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.

---

From: Patrick McManus <[email-masked]>
Date: Fri, 21 Jan 2011 10:59:38 -0500

On Fri, 2011-01-21 at 09:16 -0500, Joe Mason wrote:
> Using POST to fallback seems like a useful technique to me.  If
> possible I'd like to support it.  However, if there are reasons to
> restrict to one other method, this shouldn't take precedence.
> 
> I'd suggest the spec allow any method but recommend use of OPTIONS
> unless the script wants to take advantage of a specific feature of
> another method (and use POST to fallback to BOSH as an example of such
> a feature).

+1 - we should note OPTIONS as a useful default, but the method to use
in conjunction with Upgrade is something the app designer can consider
in the fullness of her knowledge of the whole application. I don't see
any reason why we need to tie her hands.

---

From: Scott Ferguson <[email-masked]>
Date: Fri, 21 Jan 2011 09:21:38 -0800

Salvatore Loreto wrote:
> So please provide your choices (if more then one, please rank them or 
> if only one specify
#3 (WEBSOCKET) is preferable, but all 3 are workable.

The two advantages for WEBSOCKET on the server side are primarily 
administrative

  1. For logging, particularly access logging, marking the WebSockets 
clearly in the log is useful for debugging, statistics gathering, and 
scanning for unauthorized/unexpected use.

  2. For security/authorization configure, most servers have a 
method-based authorization configuration. With "WEBSOCKET", it's 
straightforward to configure policies specific to websockets, which fit 
into existing frameworks.

Actually, there's a third, though it's more speculative.

  3. For WebSocket web-services (generally non-browser), it may be 
useful for the "GET" to provide something like metadata about the 
WebSocket service. That pattern has been used for web-services in the past.

-- Scott

---

From: Dave Cridland <[email-masked]>
Date: Fri, 21 Jan 2011 17:25:39 +0000

> WebSocket service. That pattern has been used for web-services in  

Isn't that done by just GET *without* an Upgrade token?

Dave.

---

From: Scott Ferguson <[email-masked]>
Date: Fri, 21 Jan 2011 09:45:29 -0800

> Isn't that done by just GET *without* an Upgrade token?
That helps show why I'd prefer the WEBSOCKET.

With a separate method, the logic is simple:

  If method="WEBSOCKET", start websocket channel
  If method="GET", show metadata

  (And if method="WEBSOCKET" and broken/missing Upgrade, return error.)

With the Upgrade, you have:

  If request is a valid WebSocket upgrade, start websocket channel.
  Otherwise, show metadata

When "Otherwise" is an error/broken WebSocket upgrade, not a metadata 
request, it's unclear which return is the correct one: error or metadata.

All three method systems are workable, but using a separate method 
removes any ambiguity between errors in an upgrade and something like a 
metadata GET.

-- Scott

---

From: Joel Martin <[email-masked]>
Date: Fri, 21 Jan 2011 12:23:08 -0600

> #3 - a complete new method (e.g. WEBSOCKET)

My preference order is #3, #2, #1.

GET and OPTIONS are (originally) intended as idempotent methods that don't
change the state of the server. That seems out of line with the spirit of
web sockets or at least it is ambiguous. I think #3 removes ambiguity at
many levels (logging, implementation, etc) so it's my preference. However,
it's not a strong preference so any compelling reason for the other two
(especially security) would change my vote.

Joel Martin
https://github.com/kanaka

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Fri, 21 Jan 2011 10:47:55 -0800

> If so OPTIONS with path seems fine by me.

I also prefer OPTIONS with path. Creating a new method seems unnecessary.

---

From: Bruce Atherton <[email-masked]>
Date: Fri, 21 Jan 2011 11:13:14 -0800

On 21/01/2011 10:23 AM, Joel Martin wrote:
> On Fri, Jan 21, 2011 at 3:01 AM, Salvatore Loreto 
> <[email-masked] <mailto:[email-masked]>> 
> wrote:
>
>     So please provide your choices (if more then one, please rank them
>     or if only one specify
> GET and OPTIONS are (originally) intended as idempotent methods that 
> don't change the state of the server. That seems out of line with the 
> spirit of web sockets or at least it is ambiguous. I think #3 removes 
> ambiguity at many levels (logging, implementation, etc) so it's my 
> preference. However, it's not a strong preference so any compelling 
> reason for the other two (especially security) would change my vote.

+1.

---

From: "Joonas Lehtolahti" <[email-masked]>
Date: Fri, 21 Jan 2011 21:17:31 +0200

>> reason for the other two (especially security) would change my vote.

I agree with this too. WEBSOCKET method is clear on what it is for. What  
WebSocket is used for doesn't exactly match the usage of GET or OPTIONS  
methods so "semantically" using them for WebSocket is kind of wrong. But  
then again, one of the key points of WebSocket was that it is compatible  
with HTTP infrastructure, so in that view it could be more practical to  
reuse existing methods. And of course if there is any security issues that  
using one method over another has, that would also be a good point. So  
personally I would prefer #3, but I have no problem with the other options  
either.

---

From: John Tamplin <[email-masked]>
Date: Fri, 21 Jan 2011 14:38:30 -0500

> #3 - a complete new method (e.g. WEBSOCKET)

I personally would be happy with any of these (assuming we use a real path
with OPTIONS).

However, I would slightly prefer to not create a new WEBSOCKET method, since
I suspect many existing implementations would treat it as equivalent to GET
anyway.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Dave Cridland <[email-masked]>
Date: Fri, 21 Jan 2011 19:41:46 +0000

On Fri Jan 21 17:45:29 2011, Scott Ferguson wrote:
> When "Otherwise" is an error/broken WebSocket upgrade, not a  
> metadata request, it's unclear which return is the correct one:  
> error or metadata.
> 
> removes any ambiguity between errors in an upgrade and something  
> like a metadata GET.

No, there are three cases:

1) No Upgrade => Do metadata (or fallback, or whatever - it is at  
this stage "just" an HTTP request).
2) Upgrade, but no Sec-* => Error!
3) Upgrade and Sec-* => Transition to WebSocket.

With a different method, you still have three cases, but you also  
have an interesting range of silly states, such as WEBSOCKET with no  
Upgrade token.

The point of Upgrade is precisely to switch protocol to satisfy the  
request. Putting in a new method is a tortology if you're already  
using Upgrade.

Others are quite right when they say that GET would be idempotent and  
a WebSocket might not be. Luckily, after the Upgrade, we're no longer  
in HTTP so this wouldn't matter, but if it did due to the fallback  
method, then you could just pick the right method.

Dave.

---

From: John Fallows <[email-masked]>
Date: Fri, 21 Jan 2011 13:45:47 -0800

> #3 - a complete new method (e.g. WEBSOCKET)

I prefer OPTIONS (with path).  A new method seems unnecessary.

John
-- 
>|< Kaazing Corporation >|<
John Fallows | CTO | +1.650.960.8148
444 Castro St, Suite 1100 | Mountain View, CA 94041, USA

---

From: Senthilkumar Peelikkampatti <[email-masked]>
Date: Fri, 21 Jan 2011 23:09:46 -0600

I prefer #2 OPTIONS
         OPTIONS method description fits more than others -- "The OPTIONS
method represents a request for information about the communication options
available on the request/response chain identified by the Request-URI. This
method allows the client to determine the options and/or requirements
associated with a resource, or the capabilities of a server, without
implying a resource action or initiating a resource retrieval.".

--Senthil

On Fri, Jan 21, 2011 at 3:01 AM, Salvatore Loreto <
[email-masked]> wrote:

> other methods such as OPTIONS (
> http://www.ietf.org/mail-archive/web/hybi/current/msg05721.html)
> convinced Joe and myself, as HyBi wg co-chairs, that it make sense to run a
> straw-poll

-- 
Regards,
Senthilkumar Peelikkampatti,
http://pmsenthilkumar.blogspot.com/

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sat, 22 Jan 2011 10:38:58 +0100

* Salvatore Loreto wrote:
>only one specify "I can live only with") among the following
>
>#1 - GET
>#2 - OPTIONS

When implementations fail to agree to switch protocols, that's pretty
much a fatal condition. The browser API does not account for that, and
it would probably be unwise to even attempt to re-use the existing HTTP
connection after an upgrade attempt failed while the connection remains
open. Our current draft even recommends closing the connection instead
of generating some HTTP error response if the server refuses to upgrade
for some reason. That does not fit very well with the Upgrade design in
HTTP, and so none of the options seem very appealing.

It seems unlikely the working group would agree to OPTIONS if it does
not include a path, and it seems nobody is advocating for keeping GET,
or a fourth option not included in the poll (well, not mandating but
merely suggesting a method has been proposed; that seems sound design
to me, but in practise it would lead to interoperability and other
failures, so I do think the specification needs to communicate clearly
that failure should be expected if different methods are used, whether
we make this a conformance issue or not).

That basically leaves OPTIONS with path and WEBSOCKET with path. To me
it seems the benefits of OPTIONS are vague and minor, while the bene-
fits with WEBSOCKET are pretty clear and noticable. Scott Ferguson has
summarized some of them already.

I would add that combining the OPTIONS method with Upgrade strikes me
as a bit of a hack, "The OPTIONS method represents a request for
information about the communication options available" like you'd do
for instance to find out if a server is willing to handle certain kinds
of cross-origin requests, like http://www.w3.org/TR/cors/ uses OPTIONS
for that purpose. Adding to that "Oh, it's also how Websocket clients
connect to your server" would certainly lead to confusion.

A particular worry is that some people might mistake this to mean they
should put the "CORS" headers in the handshake to make the Websocket
service available to third parties, which could lead to security issues
(though I haven't thought through the implications of this yet.)

I am open to arguments why OPTIONS is best, or that there is a very
significant cost associated with introducing a new method, but so far
it seems clear that WEBSOCKET would be best.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 22 Jan 2011 11:20:58 +0100

Hi Bjoern,

> summarized some of them already.

In fact I find the benefits around WEBSOCKET very small, especially since
they were compared to GET. We could as well argue that OPTIONS provides
the same advantages.

> as a bit of a hack,

It is not a hack, it's even how the Upgrade mechanism was documented.

(...)
> it seems clear that WEBSOCKET would be best.

I still see some advantages to using OPTIONS over the two other ones :

1) more chances of cleaner failure than GET on non-compatible intermediaries ;
   some of them might even respond themselves.
2) ensure that no intermediary will cache the response
3) ability to return 200 to indicate supported options when an
   alternative scheme might be considered
4) no need to update the equipments that already support the Upgrade
   mechanism but which block unknown methods

The big disadvantage I see in having a new method is that from a product
editor's point of view, it's a new feature and this will take much more
time to integrate into their products. If you own a product that does not
work with Upgrade+101, you can contact you vendor and report a bug that
blocks WS traffic due to their blatant incompatibility with existing HTTP
specs. The fix is simple and might very well happen in a minor release.

Now if you contact them saying "I noticed that your product does not
support the new WEBSOCKET method, can you tell me when it's available",
they'll obviously respond "it will be the shiny new feature of version
4.x that's planned for next year".

Sticking to existing standards is often more efficient when your goal
is to get things to work.

Thus I'd select OPTIONS first, otherwise GET.

Regards,
Willy

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 22 Jan 2011 11:26:50 +0100

> I'd suggest the spec allow any method but recommend use of OPTIONS unless the script wants to take advantage of a specific feature of another method (and use POST to fallback to BOSH as an example…

We should avoid mixing POST with any request that's expected to return
a 101 response. Some intermediaries just forward the request with an
"Expect: 100-continue" header, waiting for the server to return 100
and only then they forward the data, waiting for a response.

This can cause some "HTTP/1.1 100 Continue" responses to appear on the
response path where the client would only expect a 101. It could also
confuse those intermediaries if the WS server immediately responds 101
disregarding the Expect header. The intermediary might also very well
consider the 101 as a 1xx response validating the Expect then wait for
any additional data to be sent as the final response.

There is a number of undesirable situations this can cause and I think
we'd prefer to stay away from those.

Regards,
Willy

---

From: Julian Reschke <[email-masked]>
Date: Sat, 22 Jan 2011 11:31:57 +0100

> 4.x that's planned for next year".

I'd argue that intermediates are supposed to handle extension methods, 
thus not handling them can be considered a bug as well.

Best regards, Julian

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sat, 22 Jan 2011 12:03:49 +0100

>It is not a hack, it's even how the Upgrade mechanism was documented.

(RFC 2068 does not seem to mention using OPTIONS+Upgrade specifically.)

>   mechanism but which block unknown methods

You would have to put some argument or data behind that (for the sake of
argument, let's say I don't understand the first point with respect to
WEBSOCKET, find bad caching behavior equally likely between WEBSOCKET
and OPTIONS, you could use a 200 response with WEBSOCKET aswell, and not
supporting some unknown method might be a policy decision that should
not be worked around by using a supposedly known method.)

>specs. The fix is simple and might very well happen in a minor release.

Adding a method to a white list is easy compared to figuring out how to
properly support the Upgrade mechanism, I would in fact expect there to
be a configuration file that permits people actually running the soft-
ware to do that. A vendor that requires you to wait for the next version
of their product whenever you'd like to use a new method also does not
seem likely to patch Upgrade support right in for you without making you
wait for the next version.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 22 Jan 2011 12:50:45 +0100

[changed the subject to ease counting for Chairs]

> (RFC 2068 does not seem to mention using OPTIONS+Upgrade specifically.)

I meant the first reference to an Upgrade handshake appeared in RFC2817
coupled with OPTIONS.

> not be worked around by using a supposedly known method.)

For the first point, OPTIONS and WEBSOCKET are equivalent, that's why I
compared that to GET. I'm not sure what arguments you need for the other
points though.

> Adding a method to a white list is easy

I would agree with you if it was just a matter of adding a method to a white
list. Unfortunately it's not always the case, as some products support a
predefined list of methods. For instance Pound seems to support various
sets of methods but does not appear to offer support for unknown methods.
Nginx also has a set of predefined methods and does not appear to offer
any option to add more.

> compared to figuring out how to
> properly support the Upgrade mechanism,

Having a specific method is not an alternative for getting the Upgrade to
work as designed. The later is a prerequisite anyway, and it's quite easy
to do in intermediaries that are susceptible to work transparently because
they already have to support tunnel-like behaviours when they're installed
in front of proxies. I had to do it for haproxy and basically at minima,
it's just a matter of adding a check for 101 in responses where you already
check for 200 in response to CONNECT. This is really easy to do in this
type of product and can be done in a minor release (as I did).

> I would in fact expect there to
> ware to do that.

A configuration file will unfortunately not implement behaviours that a
product does not have. HTTP methods follow some semantics. There is one
message in response to one request message. Opening a tunnel is different,
either it was already implemented and we can already use it, or it was not
and the configuration itself will not make that possible without code
changes.

I feel like what you liked in the WEBSOCKET method was the ability to
*just* have to configure it to make an intermediary comply with its
semantics, but unfortunately, this will not be the case, and it will
require more code in more products (having to implement tunnelling plus
support for the method in some of them).

> A vendor that requires you to wait for the next version

Then you have never worked with vendors :-)

Vendors are always looking for new features to advertise with new versions.
Putting a big "WEBSOCKET" stamp on a new version is fine, it requires little
marketting to sell it. Since it cannot be demonstrated there is a bug in the
version without it, there is no reason why they'd offer you with a fix for
your version.

As Julian said, it could be argued that the lack of support for unknown
methods is a bug, but that's harder to use to make a vendor move, especially
when this method comes with a different use than other ones.

Regards,
Willy

---

From: Julian Reschke <[email-masked]>
Date: Sat, 22 Jan 2011 13:41:42 +0100

> when this method comes with a different use than other ones.

We probably should clarify whether, when we say WEBSOCKET, we mean 
*just* the new method, or that new method + Upgrade.

I think Roy mentioned once that just a new method, with both sides just 
sending using chunked encoding, might be good enough. However I believe 
this is less likely to work though intermediaries (due to the chunked 
request body).

Best regards, Julian

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sat, 22 Jan 2011 13:56:10 +0100

>compared that to GET. I'm not sure what arguments you need for the other

You said, for instance, that OPTIONS helps making sure intermediaries
will not cache the response. Why can't we make WEBSOCKET ensure that
just as much? I can see that OPTIONS is defined as non-cacheable and
that WEBSOCKET has yet to be defined, but beyond that I don't see
much difference. Why would a 101 response to an unknown method ever
be put in a cache, why can't we use Cache-Control to work around it?
Would you agree that if caching is a concern, GET is probably worse
than the other two?

>support for the method in some of them).

Intermediaries so far don't play much of a role in my preference in
the poll. My point was that supporting the Websocket protocol using
method X in the handshake should not be much harder than supporting
the Websocket protocol using method Y in the handshake, and how the
marketing dynamics in "bug vs feature" work out is rather unclear.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 22 Jan 2011 13:59:48 +0100

> *just* the new method, or that new method + Upgrade.

I agree with your point. But having a new method without the Upgrade will
definitely require upgrading *all* intermediaries, which is what we've been
trying to avoid since the beginning by remaining HTTP-compliant.

> this is less likely to work though intermediaries (due to the chunked 

This will also cause other new issues, mainly intermediaries which don't
forward the response until all the request is sent, or inversely stop
forwarding the request as soon as the server starts to respond. Also
another issue with chunked encoding is that any intermediary is free to
bufferize a complete request before forwarding it, possibly changing
the chunking or even replacing it with a content-length. The variety
of issues depending on implementations makes this solution look scary
to me, to be honnest. Probably that if it worked well enough we wouldn't
be trying to define a tunneling protocol to work over HTTP :-/
One could also argue that the chunked encoding still adds bytes to the
framing :-)

Regards,
Willy

---

From: Julian Reschke <[email-masked]>
Date: Sat, 22 Jan 2011 14:02:28 +0100

> One could also argue that the chunked encoding still adds bytes to the

I agree with all you said -- just wanted to make sure that it's clear 
what people are talking about (similar to the OPTIONS * discussion).

Best regards, Julian

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 22 Jan 2011 14:11:55 +0100

On Sat, Jan 22, 2011 at 01:56:10PM +0100, Bjoern Hoehrmann wrote:
> much difference.

That's what I said above too. For that point, OPTIONS and WEBSOCKET are
equivalent.

> Why would a 101 response to an unknown method ever be put in a cache,
> why can't we use Cache-Control to work around it?

When thinking about intermediaries, you should always think about having
two of them in the path, no just one. So you have this :

   Client  --->  intermediary 1  --->  intermediary 2  ---->  server

This is what makes the worst scenario happen. For instance, a GET from
client 1 might be forwarded to the server without the Upgrade headers,
and the 101 response might not be understood by intermediary 2 and be
replaced with a 200, in turn transferred to intermediary 1 which could
cache it. That's just an example of course.

> Would you agree that if caching is a concern, GET is probably worse

Yes, that's what I said twice (three times if we count this mail :-)).
That's what I'm not much fond of GET compared to others. GET could cause
some caching to occur while others will not.

> the poll.

It's your choice but I pretty sure you won't find that many WebSocket server
deployments without a load balancer in front of them. The two products I gave
as example of intermediaries are used as load balancers and don't support
adding new methods by configuration.

> My point was that supporting the Websocket protocol using
> the Websocket protocol using method Y in the handshake,

As explained above, it's not method X vs method Y, it's existing method
vs new method which sometimes will require new code to be written.

> and how the

When a customer asks a vendor to develop something, it's better for the
vendor to be able to justify that he won't do it for free. If it's a fix
for a bug, he will have to do it for free. If it's a new feature, he will
charge for the work and most often delay it to the next major release.
That's not something new, all products work that way. Obviously there are
variations everywhere, such as editors who don't fix bugs and others who
accept to provide minor feature improvements in minor versions. But adding
a new method with new semantics (and possibly security impacts) certainly
is something that can justify delaying to a new major version.

Regards,
Willy

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Sat, 22 Jan 2011 14:50:54 +0100

>That's what I said above too. For that point, OPTIONS and WEBSOCKET are

We seem to be talking past each other. I agree with you that there are
certain setups where using WEBSOCKET would make things more difficult
compared to using OPTIONS due to different types of necessary software
changes, including no necessary software changes with OPTIONS. Are
there other significant points in the comparison between OPTIONS and
WEBSOCKET? I am not sure whether you are saying among the four points
you listed, the first three don't make much of a difference with these
two methods specifically.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 22 Jan 2011 15:24:17 +0100

> two methods specifically.

Indeed, the 3 first points are somewhat equivalent between OPTIONS and
any unknown method. The 3rd one already corresponds to OPTIONS however,
so it's probably not worth reimplementing it with another method.

Regards,
Willy

---

From: Joe Mason <[email-masked]>
Date: Sat, 22 Jan 2011 11:31:23 -0500

That's a good reason to avoid it, and points out a general drawback to allowing any method: some of them have non-obvious semantics.

I change my vote to requiring one specific method. OPTIONS is my first choice but I can see the benefits in clarity of WEBSOCKET.

Joe

----- Original Message -----
From: Willy Tarreau [mailto:[email-masked]]
Sent: Saturday, January 22, 2011 05:26 AM
To: Joe Mason
Cc: Dave Cridland <[email-masked]>; Salvatore Loreto <[email-masked]>; Joe Hildebrand <[email-masked]>; SM <[email-masked]>; Server-Initiated HTTP <[email-masked]>
Subject: Re: [hybi] straw-poll on GET vs OPTIONS vs new method

Regards,
Willy

---

From: Emmanuel BUU <[email-masked]>
Date: Sat, 22 Jan 2011 18:10:51 +0100

Hello everybody,

I am studying WebSocket as part of a research project. I subscribed 
recenctly to the mailing list and did not have the occasion to follow 
all the standard devepment. I have a couple of questions:

1/ Is WebSocket is a message oriented protocol or a stream oriented 
protocol.

2/ Authentication: is there a standard way for a WebSocket client to 
authenticate against a WebSocket server?

3/ WebSocket may be seen as a replacement for the use of Flash remoting, 
shared objects and RTMP (and RTPMT protocol) for HTML5. What are the 
possible comparison between the two worlds?

4/ What lead to the decision to build a simple "transport" protocol vs 
to provide higher level primitives such  remote procedure call and event 
notification?

Emmanuel BUU
IVèS
http://www.ives.fr/

---

From: Adam Barth <[email-masked]>
Date: Sat, 22 Jan 2011 11:24:20 -0800

On Sat, Jan 22, 2011 at 9:10 AM, Emmanuel BUU <[email-masked]> wrote:
> recenctly to the mailing list and did not have the occasion to follow all
> the standard devepment. I have a couple of questions:
>

Message.

> authenticate against a WebSocket server?

Nope.

> possible comparison between the two worlds?

Not sure.

> 4/ What lead to the decision to build a simple "transport" protocol vs to
> provide higher level primitives such  remote procedure call and event
> notification?

You can build the a high performance version of the later with the
former but not the reverse.  We're in the business of providing
simple, hard-working primitives that authors can use to create new,
innovative applications.  The more genearl a tool we provide, the
better off the ecosystem is.

Adam

---

From: Senthilkumar Peelikkampatti <[email-masked]>
Date: Sat, 22 Jan 2011 15:57:15 -0600

I am thinking of another issue with new method (WEBSOCKET), it needs to be
defined and added to http spec, it crosses different boundary.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Sat, 22 Jan 2011 14:20:44 -0800

On Jan 22, 2011, at 1:57 PM, Senthilkumar Peelikkampatti wrote:

> I am thinking of another issue with new method (WEBSOCKET), it needs to be defined and added to http spec, it crosses different boundary. 

HTTP methods are extensible -- any document can define a new one.

HTTP methods do not evolve -- no document can change an already
defined method's meaning once that meaning is deployed on the Internet,
since we have no means of updating the already deployed servers.

We can, however, optionally tweak the meaning of methods via request
header fields like If-Modified-Since, but only when the tweak remains
inside the scope of the original method definition (i.e., nothing
breaks if any intermediary or the origin server do not understand
the tweak).

....Roy

---

From: Jamie Lokier <[email-masked]>
Date: Sun, 23 Jan 2011 19:55:45 +0000

> One could also argue that the chunked encoding still adds bytes to the

It could be sent with a large Content-Length (e.g. 2^31-1)
Content-Length.  That removes request chunking issues, but the other
concerns about buffering or stopping forwarding still apply.

But then again, neither WebSocket nor HTTP specify what kind of
buffering to expect with Upgrade, so unintended buffering could occur
with method+Upgrade too.

Has anyone done any actual tests with chunked requests, and/or
simultaneous streaming request+response?

I've done some tests like that with browsers and other clients, and
the behaviour was quite varied.  But I haven't had the occasion to do
those tests with intermediaries.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Sun, 23 Jan 2011 20:14:57 +0000

Willy Tarreau wrote:
> response path where the client would only expect a 101.

That should not be visible at the client.  If the intermediary
generates Expect: 100-continue itself, it should be prepared to absorb
the first 100 response; it is wrong to forward it.  If it's generating
it because the client sent Expect: 100-continue, then the client will
see it but that is correct.

In the case of POST+Upgrade, a client may actually choose to send
Expect: 100-continue, if it wants the semantics associated with that.
That's fine, there's nothing wrong with that; it's a client choice.

Do you know of buggy intermediaries which generate Expect: 100-continue
(when the client hasn't sent it), and then forward the 100 response
back to the client even though the client hasn't sent the Expect header?

We could always add another variable to the mix: Expect: 101-websocket
:-) (Except we know that intermediadiaries/servers don't all implement
the mandatory HTTP spec about unrecognised expectations :-/)

> It could also confuse those intermediaries if the WS server
> immediately responds 101 disregarding the Expect header. The
> intermediary might also very well consider the 101 as a 1xx response
> validating the Expect then wait for any additional data to be sent
> as the final response.

I agree, that might be a problem.

The WS server should probably send 100 first in this scenario.  It
could be specified or recommended that way.

Bearing that in mind, do you know if it's a problem with any
intermediary (that you know of), after the intermediary has received
100, that it'll misinterpret or fail to forward the _subsequent_ 101?

In which case, do you think such intermediaries will forward 101
fine in the first place, with non-POST requests and no 100?

What about intermediaries which send Expect: 100-continue when
forwarding *any* request that has a body (perfectly reasonable)?

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Sun, 23 Jan 2011 20:22:35 +0000

> any reason why we need to tie her hands.

+1 to this.

There is no reason to tie the implementors hands on this.  Instead,
specify that WebSocket *servers* should accept any reasonable method
provided it has the right headers (and a path), and clients may use
any reasonable method, including POST, OPTIONS, WEBSOCKET if they wish.

Browsers will no doubt settle on a single method at first, but if the
method turns out to have intermediary problems (which we can't be sure
of until it's more widely used), it can be changed with a browser
upgrade - only if the WS servers are accepting connections that use
any reasonable method.

So for maximum plausible compatibility I favour

   (a) tell servers to accept any reasonable method, with a path and
       the right headers.
   (b) tell clients they can use any reasonable method as in (a)
   (c) see what happens; clients can be narrowed down if necessary
       as a result of experience.

-- Jamie

---

From: John Tamplin <[email-masked]>
Date: Sun, 23 Jan 2011 15:26:00 -0500

>       as a result of experience.

For this to be useful, the JS API would have to expose the method
choice to the app, which it doesn't currently.

I would prefer this spec to pick one, and then later if it is deemed
useful, we could change this spec and the JS API simultaneously to
expose the method choice to the app.

---

From: Dave Cridland <[email-masked]>
Date: Sun, 23 Jan 2011 20:44:52 +0000

On Sat Jan 22 16:31:23 2011, Joe Mason wrote:
> That's a good reason to avoid it, and points out a general drawback  
> to allowing any method: some of them have non-obvious semantics.
> 
> I change my vote to requiring one specific method. OPTIONS is my  
> first choice but I can see the benefits in clarity of WEBSOCKET.

Right, I've been convinced too - I'm happy with recommending OPTIONS,  
although I'd prefer the choice of method to be SHOULD and not MUST,  
and the text to alert the implementor of the pitfalls of using GET or  
POST with naïve intermediaries.

Dave.

---

From: Dave Cridland <[email-masked]>
Date: Sun, 23 Jan 2011 20:51:31 +0000

> choice to the app, which it doesn't currently.
Client SHOULD use OPTIONS, but servers MUST accept any method?

> expose the method choice to the app.

I think Jamie Lokier is suggesting that we should require servers to  
be flexible on the method choice, but clients can choose a single  
fixed one if that makes sense - and I'd expect all browsers to use  
the RECOMMENDED method we select.

Dave.

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 23 Jan 2011 22:40:13 +0100

> buffering to expect with Upgrade, so unintended buffering could occur

I disagree here Jamie. Upgrade is supposed to offer us with an equivalent
tunnel as it does with CONNECT through proxies. So it leaves the scope of
HTTP messaging and is bidirectional by nature.

> those tests with intermediaries.

Same here. I used to abuse proxies in the past by passing them POST
requests with very large content lengths in order to telnet through
them. At that time it was possible to get anything in return through
some proxies as long as you did not hit the content-length limit,
even if the response did not look like HTTP at all.

There are some which like to re-chunk the response stream eventhough
there's a content-length. And I've seen some immediately stop sending
data to the server once the server started to respond.

Regards,
Willy

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 23 Jan 2011 23:03:04 +0100

Hi Jamie,

> see it but that is correct.

I agree with the principle, but given the numerous conditions around
the 1xx status handling in RFC2616, I'm pretty sure all combinations
do happen depending on the components. For instance, it's said in
section 10.1 that a proxy must forward 1xx to the client unless it
adds it itself, in which case it "need not" forward the response.
You can see that nothing prevents it from doing so, and for
implementers, this keeps the requirement to forward 1xx if the
client put it anyway, so the easiest implementation consist in
unconditionally setting it in the request and always letting it
pass in the response as long as the client talks 1.1.

> back to the client even though the client hasn't sent the Expect header?

I'm pretty sure I've encountered this behaviour in the context of a
webservice-based application about 2 years ago, but I can't put names
on the components involved. I remember that because it was my first
encounter with a 100 that was causing failures through haproxy and I
had to implement it there. Also, I would not say the intermediaries
are necessarily buggy if they do that considering it's not forbidden.
I'd just say they're lazy.

(...)
> could be specified or recommended that way.

You see, we're progressively complexifying the protocol with this design.

> 100, that it'll misinterpret or fail to forward the _subsequent_ 101?

Those who correctly process 100 (ie don't forward it to the client)
but don't know about 101 will consider it as 1xx (= 100) and will
eat it.

> fine in the first place, with non-POST requests and no 100?

I have no idea. The principle of laziness implies that all the responses
should be forwarded to the client, which would let the client see 100
and 101. I'd bet that most of those which forward 101 will forward 100,
but that's a pure guess from someone who would proceed like this would
the situation appear.

> forwarding *any* request that has a body (perfectly reasonable)?

Those are the ones I'm worried about precisely. We're talking about
POST because most often it's POST which holds a body, but we can
have it with PUT as well as many other methods. That's why I'd
really prefer avoiding to rely on the body and establish a tunnel
instead. That really matches the purpose we're looking for.

Regards,
Willy

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 24 Jan 2011 01:16:53 +0000

> the RECOMMENDED method we select.

Yes exactly.

If a server goes ahead with WebSocket, it SHOULD NOT use the method to
in deciding WebSocket negotiation or behaviour, except what might be
necessary to satisfy HTTP (e.g. consuming a request-body or not), or
if there is any security consideration.

At the client, a pure WebSocket app should not be selecting the method
choice.  That should be up to the client's WS implementation, and
would, as Dave says, be the recommended method unless some good
technical reason is discovered after deployment why a different method
should be used.

It's intended that if there is a reason to change the recommended
method, ideally it would be done by the rollout of new WS expert
implementation updates (e.g. in browsers), not changing all the apps.

Although this is a reason not to give apps a choice, it must be said
that if some apps (browser or non-browser) would like a way to fall
back to BOSH over POST, (or WebSocketProprietaryAlternative etc.)
when a WS request is refused, if a client WS implementation wants to
connect the WebSocket and XHR APIs somehow to make that possible, that
would expose the method via XHR.  Obviously pure WS apps still
shouldn't care about the method, and if there's a WS method ("for
debugging or special cases"), they should avoid using it without a
specific reason to use it.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 24 Jan 2011 03:09:45 +0000

> HTTP messaging and is bidirectional by nature.

You're right.

Though all implementations have _some_ kind of short term buffering
strategy, even if it's just an implicit side effect of the way they
work internally.  Nothing forwards each byte the moment it is received.

In future, a proxy which recognises WebSocket might, quite reasonably,
attempt to recognise frames and _deliberately_ hold back the first
small half of a frame when it knows a second half is due, in order to
make more efficient use of TCP and under some circumstances, reduce
the overall latency.  I don't know if that would improve latency in
practice but it's something I've thought about implementing.

I'm not suggesting any current proxy does that (obviously).  But I
think it's fair to ask if we know of any proxies doing other strange
kinds of buffering in "Upgrade tunnel mode" at the moment - or if we
can assume they forward every byte reasonably eagerly, as they are
clearly known to do for HTTPS over CONNECT?

-- Jamie

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 24 Jan 2011 14:11:30 +1100

>>       as a result of experience.

+1

> choice to the app, which it doesn't currently.

True - so it is unlikely to happen in the near future, but that should
not rule out future use of other methods.

> expose the method choice to the app.

but you would then need to change all the existing implementations
that were not aware that any method may be used.

regards

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 24 Jan 2011 07:36:24 +0100

> work internally.  Nothing forwards each byte the moment it is received.

I obviously agree :-)

> practice but it's something I've thought about implementing.

yes this is something we will likely have, just as some proxies currently
handle HTTP with TCP_CORK / MSG_MORE : tell the system not to send an
incomplete segment when we know some data is still coming. This is
something to think about with WebSocket, especially because we'll have
many small frames. Maybe it will make sense to have an equivalent of
the TCP PUSH bit so that both ends can say "forward it now!" at the
end of a message when their output buffer is empty.

> clearly known to do for HTTPS over CONNECT?

They will obviously buffer, and at worst they will rely on Nagle,
occasionally causing some short delays. But that should be rare and
most proxies I've seen put a TCP_NODELAY on their sockets as soon as
they're created.

Regards,
Willy

---

From: Joe Mason <[email-masked]>
Date: Mon, 24 Jan 2011 08:55:20 -0500

> You see, we're progressively complexifying the protocol with this

This is my concern - even if we can convince ourselves that 100 handling will not break the protocol in this case, it's better to specify a protocol that doesn't need to be reasoned about like this.

Joe

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 24 Jan 2011 17:41:46 +0000

Joe Mason wrote:
> This is my concern - even if we can convince ourselves that 100
handling will not break the protocol in this case, it's better to
specify a protocol that doesn't need to be reasoned about like this.

I agree, we should not depend on unknowns and complexities.

However I think it should be fair game if client implementors want to
experiment with POST or method+body and deal with whatever
consequences they find.  If they find it works, they should be allowed
to use it and assume WS servers will behave with it.

We can RECOMMEND something more conservative, without making it a hard
requirement, and we can specify enough of the client and server
behaviour for all methods.

In other words, don't overconstrain WS implementations from trying
things that may or may not work out.

Separate hard requirements that are needed for client-server interop
and security etc., from advice (recommendations) that doesn't need to
be rigid to achieve those goals.

We don't really know what works best yet - we won't know until it's
much more widely used - and what works best over the infrastructure
may change over time anyway.

We can't prevent proxies from sending Expect: 100-continue and
mishandling 101 for _any_ method that _may_ include a request body.
Without further data, since we can't rule it out, we shouldn't be
making hard specifications based on the presumption.  Leave that to
best practice, to be discovered, and advice refined with experience.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 24 Jan 2011 18:53:29 +0000

> end of a message when their output buffer is empty.

Don't we have a message-fragment "more/done" bit already?  Maybe I'm
confusing framing proposals :-)

(Related: I discussed here last year why it would make sense to be
_two_ bits: "forward eagerly" (like TCP PUSH) and "message finished",
so that remuxing/rebuffering intermediaries can safely coalesce
message fragments as well as splitting them.  With only one bit,
coalescing wrecks the forwarding guarantees and timing of byte
stream-like messages, such as would be needed for generic
HTTP-over-WebSocket (muxed) or TCP-over-WebSocket, so coalescing would
have to be forbidden at proxies (causing pointless excessive framing
overhead), or the sender would have to use fine-grained messages in
the first place (also causing unnecessary framing overhead)).

> most proxies I've seen put a TCP_NODELAY on their sockets as soon as

I'm thinking of eager forwarding of incomplete message parts causing
longer TCP stalls.  WebSocket advertising on the net likes to talk
about how it's going to be nice for low latency... so let's aim for that.

Background, Google for (I have no browser right now, sorry):

    "Performance Interactions Between P-HTTP and TCP Implementations",
    John Heidemann, USC / Information Sciences Institute, May 19, 1997.

    Abstract (partial):

    "This document describes several performance problems resulting
    from interactions between implementations of persistent-HTTP
    (P-HTTP) and TCP. Two of these problems tie P-HTTP performance to
    TCP delayed- acknowledgments, thus adding up to 200ms to each P-
    HTTP transaction. A third results in multiple slow-starts per TCP
    connection. Unresolved, these problems result in P-HTTP
    transactions which are 14 times slower than standard HTTP and 20
    times slower than potential P-HTTP over a 10 Mb/s Ethernet."..

The paper's a bit dated (1997) and modern TCP implementations are
better at avoiding the various problems described.

But still, I'm not happy to assume that eager forwarding of all
partial-frame bytes is a good low-latency strategy for WS-optimised
proxies.  Timing-wise, TCP does not behave like a simple byte pipe
with fixed latency - and this is generally true of similar protocols.

-- Jamie

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 24 Jan 2011 20:16:46 +0100

> confusing framing proposals :-)

As you suggested below, yes those are two distinct things as soon as you
multiplex or simply send chunks in multiple frames for whatever reason
(eg: a sequence of multiple opcodes, let's say text then binary).

> the first place (also causing unnecessary framing overhead)).

indeed.

> about how it's going to be nice for low latency... so let's aim for that.

No quite the opposite instead. I'm not talking about buffering everything,
just about sending full-sized TCP segments when that makes sense. It's
extremely important in mobile environments where you need a time slot for
each frame sent, and where a PUSH implies an ACK that consumes bandwidth
and time slots.

>     times slower than potential P-HTTP over a 10 Mb/s Ethernet."..

I have also observed similar issues with components which blindly apply
the Nagle algorithm. If you correctly disable it on each last chunk,
this problem cannot appear because there will always be a PUSH causing
the frame to immediately be sent.

But the behaviour above is often encountered when TCP proxies are used
to forward HTTP with Nagle disabled. Nagle should only be disabled when
the proxy has knowledge of the application layer and framing. Some
implementations clearly are abusive.

> better at avoiding the various problems described.

Not all, even last week I observed one, but I can't tell the name here ;-)

> with fixed latency - and this is generally true of similar protocols.

Anyway, since we already support fragmentation, probably that later
it will be possible to implement WS over UDP !

Cheers,
Willy

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 24 Jan 2011 21:09:21 +0000

> each frame sent, and where a PUSH implies an ACK that consumes bandwidth

I believe you are describing Linux's TCP_CORK/MSG_MORE and BSD's TCP_NOPUSH?

I.e. write what you like to the kernel socket, but tell TCP not to
send a partial segment until you signal an explicit message boundary
to be pushed eagerly.  And even then, don't do that if you already
have part of the next message to forward, meaning you are expecting
more soon?

> the frame to immediately be sent.

Since you've observed it - do you know if correct use of
TCP_CORK/MSG_MORE/TCP_NOPUSH (where available) prevents the problem
consistently (presumably with Nagle also disabled), instead of
depending on toggling TCP_NODELAY optimistically on each write and
hoping that does the right thing?

> implementations clearly are abusive.

Leaving Nagle on can add horrible latency jitter to "smooth"
interactive applications on LANs.  I've played a simple game of
"snake" - two players each guiding their growing snake around and
trying not to hit the other, every move received from a shared server.
With Nagle, the motion of the snake head was very jittery, very
visible, very unpleasant.  With Nagle turned off it was nice and
smooth.

That wasn't in a browser, but I'd expect exactly the same in a browser
game with similar "smoothness" constraints.

(But as you note, turning Nagle off can cause problems too.)

> Not all, even last week I observed one, but I can't tell the name here ;-)

Hmm.

So, based on your observations, do you basically agree with me, that
blindly forwarding bytes after CONNECT/Upgrade negotiation (which
proxies aren't precisely specified to do but generally do), is not
ideal for consistent and low WebSocket message latency?

-- Jamie

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 24 Jan 2011 22:33:51 +0100

> I believe you are describing Linux's TCP_CORK/MSG_MORE and BSD's TCP_NOPUSH?

exactly.

> hoping that does the right thing?

Yes, that what I've done in haproxy (I suspect we're becoming off-topic
here, we can continue off-list if you want). Basically I'm using MSG_MORE
on Linux which is stronger than TCP_NODELAY. So I set TCP_NODELAY on all
sockets and adjust the MSG_MORE flag on each send() depending on whether
I know there are more data behind it or not. That way the last segment
does not have the flag and goes out with TCP_NODELAY, leading to a PUSH.
That's extremely efficient, it sends full-sized segments without any added
latency since the last segment is never delayed. It's also important for
the receiver to get all segments at once, it generally allows its TCP
stack to deliver them all at once. This also presents the benefit of
saving some ACKs in the other direction : the receiver ACKs all it got
at once instead of ACKing them 2 by 2. Once again this is very important
in mobile environments.

> interactive applications on LANs.

I agree. I discovered TCP_NODELAY 15 years ago when I was wondering why
my telnet sessions were jerky on top of a simple proxy I wrote.

> I've played a simple game of

This is normal. Nagle waits for the last small frames to be ACKed before
sending more, in the hope that it can fill them in the mean time. This
causes undesired delays. This must never be enabled when quick response
or delivery is expected. That's why I was saying that it must be disabled
unless the application understands the protocol in order to adjust it at
will.

> game with similar "smoothness" constraints.

You'd have the same issue with a browser. Try to enable nagle on a keep-alive
HTTP connection and get 100 objects on the same connection, you'll get old
memories :-)

> ideal for consistent and low WebSocket message latency?

It's the only way to ensure low delivery latency without understanding
what is spoken there. However, it will inevitably cause many PUSH flags
to be emitted, and as many ACKs the other way around. That's already
what happens with many proxied protocols.

Anyway, don't stress, for the proxy to receive the data in small chunks,
the sender will already have had to send them chunked the same way with
TCP_NODELAY as well. So the proxy won't cause more PUSH/ACKs to be
exchanged than if the communication happened directly between the sender
and the receiver.

A correctly designed proxy (basically one that does not set TCP_NODELAY
when it has more data in the buffers) does not generate more PUSH flags.
It can be seen as a repeater : what it gets small is sent small, what
it gets large is sent large.

Regards,
Willy

---

From: Gabriel Montenegro <[email-masked]>
Date: Sat, 29 Jan 2011 07:03:42 +0000

Unfortunately, I can't say which I'd absolutely prefer, but I can say that OPTIONS is at the very bottom.

In other words, my preference is:

#3 (WEBSOCKET) over #2 (OPTIONS)
#1 (GET) over #2 (OPTIONS)

Here's some rationale: 

The point that previous uses of OPTIONS can lead to confusion with previous usage was made in reference to its use in CORS (http://www.w3.org/TR/cors/). There's also WEBDAV (http://tools.ietf.org/html/rfc4918) and others.  This point argues for WEBSOCKET (a brand new method with no other previous usage) over either GET or OPTIONS. 

+1 WEBSOCKET

Even though both GET and OPTIONS have existing usage, they are not equally safe. GET and POST (which are by far the most used) undergo much more scrutiny than other lesser-known or completely arbitrary methods. Lesser methods can be used to exploit vulnerabilities in faulty ACL checking:
http://www.owasp.org/index.php/Testing_for_HTTP_Methods_and_XST_(OWASP-CM-008)#Arbitrary_HTTP_Methods

This argues for GET over either OPTIONS or WEBSOCKET.

+1 GET

The reason why the straw poll was initiated is that it was argued that OPTIONS is what RFC2817 uses.  This is true, but only partly so. RFC2817 uses *both* GET and OPTIONS.

GET is used in an "Optional Upgrade" per http://tools.ietf.org/html/rfc2817#section-3.1. This means that the GET request could be satisfied by the server either (1) without heeding the Upgrade, or (2) by first switching protocols as requested by the Upgrade, and then answering the original request, per the order specified in http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-12#section-9.8:
 	
   ... although the first action
   after changing the protocol MUST be a response to the initial HTTP
   request containing the Upgrade header field.

OPTIONS is used in a "Mandatory Upgrade" per http://tools.ietf.org/html/rfc2817#section-3.2. The Upgrade is still optional at the server, of course, but if it's not heeded, the client could abort at this time. This would make the upgrade a precondition to fetching the resource ("mandatory" in that respect).

So RFC2817 is not an argument for either method, but to the use of more than one. The WG seems to have spoken against this, and I agree, it's best to have only one and avoid more complexity.

Another point against GET and OPTIONS is that both are supposed to be idempotent. This has been misconstrued as a requirement to cause no changes at the server. http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-12#section-7.1.2 clarifies:

   ...  It is important to
   note that idempotence refers only to changes requested by the client:
   a server is free to change its state due to multiple requests for the
   purpose of tracking those requests, versioning of results, etc.

Since the Upgrade effects a change on the connection, not on the resource, I don't see idempotence as an argument against GET, OPTIONS or any other idempotent method.

That's why I prefer WEBSOCKET or GET (and only one) over OPTIONS. 

Gabriel

> Sent: Friday, January 21, 2011 1:01 AM
> To: [email-masked]
> Cc: Joe Hildebrand; SM
> Subject: [hybi] straw-poll on GET vs OPTIONS vs new method
> 
> 
> Hi there,
> 
> after the result of the first straw-poll on GET+Upgrade+Masking, there were a
> discussion about GET not being the only method to "Upgrade"
> the connection;
> The mail thread(s) started to discuss the alternatives, convinced Joe and myself,
> as HyBi wg co-chairs, that it make sense to run a straw-poll to get as much
> information as we can  from people about technical reason (pro and cons) for
> their preference(s).
> 
> 
> So please provide your choices (if more then one, please rank them or if only
> one specify "I can live only with") among the following
> 
> #1 - GET
> #2 - OPTIONS

---

From: Julian Reschke <[email-masked]>
Date: Sat, 29 Jan 2011 10:04:11 +0100

> The point that previous uses of OPTIONS can lead to confusion with previous usage was made in reference to its use in CORS (http://www.w3.org/TR/cors/). There's also WEBDAV (http://tools.ietf.org/h…

I don't think there's a problem with OPTIONS and WebDAV. WebDAV uses 
OPTIONS just the way it's designed for, and I'm not aware of any 
potential problems using OPTIONS in other protocol exchanges as well. If 
you do, you need to elaborate.

OPTIONS and CORS is a slightly different story because of the stupid 
requirement CORS makes on OPTIONS not to require authentication. That's 
a design flaw in CORS. I'm not sure how it would affect usage of OPTIONS 
over here, though.

> http://www.owasp.org/index.php/Testing_for_HTTP_Methods_and_XST_(OWASP-CM-008)#Arbitrary_HTTP_Methods

Not convinced. The linked text correctly reports that some frameworks do 
only ACL on certain methods, but then fallback to execute unknown 
methods as GET. That's a bug, indeed. Do we have evidence that 
Websockets is going to be deployable on frameworks like these anyway? 
Also, these frameworks are vulnerable already, how does Websockets 
change the situation?

> Since the Upgrade effects a change on the connection, not on the resource, I don't see idempotence as an argument against GET, OPTIONS or any other idempotent method.

That's correct.

Best regards, Julian