Subject: Re: [hybi] Can Websockets be used peer to peer? Was Straw Poll: is Upgrade...?

From: Bruce Atherton <[email-masked]>
Date: Thu, 06 Jan 2011 16:37:47 -0800

On 06/01/2011 1:43 AM, Greg Wilkins wrote:
>
> a difficulty of this working group is that we were never able to even
> agree on a requirement document as the various constituencies
> suspected each other of steering the requirements towards their own
> use-cases. Hence it is difficult to really say what is in scope or out
> of scope.

Thanks for the explanation, Greg. That explains a lot about why this 
group is so dysfunctional. If you haven't agreed on what your goal is, 
no wonder you are all pulling in different directions. A trade-off that 
seems obvious to someone with a specific goal in mind will seem 
insensible to another person with a different goal.

Should the design of Websockets consider uses of peer-to-peer? Should it 
give consideration to issues that are specific to Intranet use? Should 
designers of the spec worry about non-browser clients like custom 
written Websocket clients and Websocket servers that act as clients to 
other servers? Or is the only thing that really matters when designing 
Websockets the browser and server interactions, and any other use that 
happens to work is just gravy?

I think that agreeing on these basic questions will likely eliminate a 
lot of the conflict on this list. Also, as a potential customer of this 
specification it would be very useful to me to have a clear 
understanding of what the intended uses for Websockets are, so that I 
can better understand where I am supposed to be applying it in my future 
designs.

So far it appears that Greg, Brodie, Jerod, and I favour including 
peer-to-peer design considerations in the spec. I am not sure, but it 
sounds like John considers it out of scope and Scott considers it in 
scope. What about the rest of you?

---

From: John Tamplin <[email-masked]>
Date: Thu, 6 Jan 2011 20:23:57 -0500

On Thu, Jan 6, 2011 at 7:37 PM, Bruce Atherton <[email-masked]> wrote:

> peer-to-peer design considerations in the spec. I am not sure, but it sounds
> like John considers it out of scope and Scott considers it in scope. What
> about the rest of you?
>

I am not opposed to it exactly, but given how long everything has taken I am
keen to keep the base protocol minimal so we can actually get something out
the door.

Just like when we were discussing compression, multiplexing, and metadata
during the design of the basic framing, we weren't getting anywhere until we
stripped it down to the minimum.  To overcome objections about stripping out
too much, I wrote up examples of how the base framing could be extended in a
compatible fashion to incorporate those use-cases and we were able to agree
on the base framing.

If you think peer-to-peer WebSockets is important, I suggest talking about
specific things that need to be represented in the base protocol in order to
allow that be added at a later time.  So far the suggestion has been that
masking is mandatory and an extension could be defined in the future to
remove it has been met with skepticism that people who build infrastructure
pieces will build in that extensibility (if I am mis-stating the objections,
please correct me).  I'm not sure this is any different than the
compression/mux/metadata cases we already dealt with, and my feeling is if
such equipment has to be upgraded later to support new things so be it.  I
personally am fine with adding language into the spec saying what things are
expected to be changed so implementors should be prepared to update the
handling if that helps address the concern.  However, I am really loathe to
try and define peer-to-peer operation in the base spec because I think it
will unreasonably extend the process of getting the base protocol out the
door.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Gabriel Montenegro <[email-masked]>
Date: Fri, 7 Jan 2011 01:44:30 +0000

John Tamplin wrote:
…  However, I am really loathe to try and define peer-to-peer operation in the base spec because I think it will unreasonably extend the process of getting the base protocol out the door.

[gab] I agree. WGs are typically chartered with a specific focus in order to ease progress. The charter as approved by the IESG includes this:
  The Hypertext-Bidirectional (HyBi) working group will seek
  standardization of one approach to maintain bidirectional
  communications between the HTTP client, server and intermediate
  entities, which will provide more efficiency compared to the current
  use of hanging requests.

This prominently talks about client-server not peer-to-peer.  When people have asked me if p2p is in scope, I’ve always answered “no, by charter” as that is how I’ve interpreted the above. Given the difficulties in this group, I agree that is best left after we deliver on the current items. We can then tackle a re-chartering operation and move on to other efforts, but for now I think that is not only out of scope, but would delay us even further.

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 06 Jan 2011 18:00:00 -0800

John Tamplin wrote:
> On Thu, Jan 6, 2011 at 7:37 PM, Bruce Atherton <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     peer-to-peer design considerations in the spec. I am not sure, but
>     it sounds like John considers it out of scope and Scott considers
>     it in scope. What about the rest of you?
>
>
> I am not opposed to it exactly, but given how long everything has 
> taken I am keen to keep the base protocol minimal so we can actually 
> get something out the door.

+1

> If you think peer-to-peer WebSockets is important, I suggest talking 
> about specific things that need to be represented in the base protocol 
> in order to allow that be added at a later time....   However, I am 
> really loathe to try and define peer-to-peer operation in the base 
> spec because I think it will unreasonably extend the process of 
> getting the base protocol out the door.

+1

As long as the unmasked framing can be used for a clean peer-to-peer 
protocol, the WebSocket handshake/client-masking can be focused on the 
browser case.

(It would be a problem if the client masking were to change the current 
framing significantly, but I don't think that kind of change has been 
proposed.)

-- Scott

> ------------------------------------------------------------------------
>
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 6 Jan 2011 18:09:20 -0800

I agree. This is clearly out of charter scope.

Even were it not out of scope, the P2P problem is clearly distinct, for a
number of reasons:

1. There is no expectation of being able to travel over an HTTP substrate,
even initially.
2. The issue of NAT traversal must be addressed and turns out to require a
confirmation handshake
    in any case.
3. Many of the most interesting cases involve UDP which raises another set
of issues.

As I said earlier, there is already an effort to address this, but it's not
appropriate for this WG.

-Ekr

On Thu, Jan 6, 2011 at 5:44 PM, Gabriel Montenegro <[email-masked]>wrote:

>
>
> John Tamplin wrote:
> *…*  However, I am really loathe to try and define peer-to-peer operation
> in the base spec because I think it will unreasonably extend the process of
> *[gab] *I agree. WGs are typically chartered with a specific focus in
> order to ease progress. The charter as approved by the IESG includes this:
>
> This prominently talks about client-server not peer-to-peer.  When people
> have asked me if p2p is in scope, I’ve always answered “no, by charter” as
> that is how I’ve interpreted the above. Given the difficulties in this
> group, I agree that is best left after we deliver on the current items. We
> can then tackle a re-chartering operation and move on to other efforts, but
> for now I think that is not only out of scope, but would delay us even
> further.
>

---

From: Bruce Atherton <[email-masked]>
Date: Thu, 06 Jan 2011 19:19:32 -0800

I am not talking about designing the specifics of peer to peer into the 
protocol. I am talking about everybody being clear on what they are 
designing. The question about peer to peer is just one of several in 
that vein. Perhaps you think that everybody is in agreement in their 
vision of Websockets, but it sure doesn't look that way to me, and I 
think that is where some of the polarization is coming from.

If everyone agrees that design decisions should be based only on 
browser-server interaction, then there is no need to talk about optional 
masking. If peer to peer is out of scope, then there is no need to worry 
about the asymmetry of the client and server. Suddenly progress can be 
made much more easily because a lot of objections will simply disappear. 
When you introduce intermediaries it can get more complicated, depending 
on what you decide is in scope. Greg's muxer probably is. Willy's WS 
Accelerator and my usecases with servers acting as clients, perhaps or 
perhaps not. Which ones you include for consideration are going to have 
an impact on what issues should be given weight when designing the protocol.

Correct me if I am wrong, but my understanding from what you have 
written is that you are principally concerned with browser to server 
interactions, and other scenarios that happen to work out are fine with 
you but need not be designed for. So it makes sense that you don't see a 
particular need for optional masking or have much concern about clients 
masking their frames and servers not. This view of the protocol is going 
to inform many more of your decisions as they come up.

But it is clear from Greg's description that he has a very different 
vision of where he sees Websockets being used. You are designing a 
screwdriver and he is designing a drill that can hold screwdriver bits. 
Both are useful tools, but the trade-offs for each are going to be 
different.

So all I'm suggesting is that you all clarify among yourselves what is 
in scope and what is not. That way, there is no need to argue about 
design decisions that impact uses that are out of scope. As a result, I 
would hope that progress would be sped up.

On 06/01/2011 5:23 PM, John Tamplin wrote:
> Just like when we were discussing compression, multiplexing, and 
> metadata during the design of the basic framing, we weren't getting 
> anywhere until we stripped it down to the minimum.  To overcome 
> objections about stripping out too much, I wrote up examples of how 
> the base framing could be extended in a compatible fashion to 
> incorporate those use-cases and we were able to agree on the base framing.
>
> in order to allow that be added at a later time.  So far the 
> suggestion has been that masking is mandatory and an extension could 
> be defined in the future to remove it has been met 
> with skepticism that people who build infrastructure pieces will build 
> in that extensibility (if I am mis-stating the objections, please 
> correct me).  I'm not sure this is any different than the 
> compression/mux/metadata cases we already dealt with, and my feeling 
> is if such equipment has to be upgraded later to support new things so 
> be it.  I personally am fine with adding language into the spec saying 
> what things are expected to be changed so implementors should be 
> prepared to update the handling if that helps address the concern. 
>  However, I am really loathe to try and define peer-to-peer operation 
> in the base spec because I think it will unreasonably extend the 
> process of getting the base protocol out the door.
>
> -- 
> John A. Tamplin

---

From: Bruce Atherton <[email-masked]>
Date: Thu, 06 Jan 2011 19:23:08 -0800

Thanks for the reference, Gabriel. That is very helpful. In that case, I 
won't raise any more issues about design that impacts peer to peer 
scenarios.

On 06/01/2011 5:44 PM, Gabriel Montenegro wrote:
>
> John Tamplin wrote:
> *…*  However, I am really loathe to try and define peer-to-peer 
> operation in the base spec because I think it will unreasonably extend 
> the process of getting the base protocol out the door.
>
> */[gab] /*I agree. WGs are typically chartered with a specific focus 
> in order to ease progress. The charter as approved by the IESG 
> includes this:
>
> This prominently talks about client-server not peer-to-peer.  When 
> people have asked me if p2p is in scope, I’ve always answered “no, by 
> charter” as that is how I’ve interpreted the above. Given the 
> difficulties in this group, I agree that is best left after we deliver 
> on the current items. We can then tackle a re-chartering operation and 
> move on to other efforts, but for now I think that is not only out of 
> scope, but would delay us even further.
>

---

From: John Tamplin <[email-masked]>
Date: Fri, 7 Jan 2011 00:41:02 -0500

On Thu, Jan 6, 2011 at 10:19 PM, Bruce Atherton <[email-masked]> wrote:
> Correct me if I am wrong, but my understanding from what you have written is
> that you are principally concerned with browser to server interactions, and
> other scenarios that happen to work out are fine with you but need not be
> designed for. So it makes sense that you don't see a particular need for
> optional masking or have much concern about clients masking their frames and
> servers not. This view of the protocol is going to inform many more of your
> decisions as they come up.

Mostly what I care about are achieving consensus and getting
WebSockets deployed.  I am not opposed to leaving extension hooks for
other things if that helps us get consensus.

> But it is clear from Greg's description that he has a very different vision
> of where he sees Websockets being used. You are designing a screwdriver and
> he is designing a drill that can hold screwdriver bits. Both are useful
> tools, but the trade-offs for each are going to be different.

As mentioned, there are some particular "bits" I want to add to the
screwdriver later such as multiplexing and compression, and I made
sure that the base screwdriver could be extended for those bits later.
 However, I don't want to have to design and get consensus on all
those other bits now -- I just want to ship the screwdriver and one
bit, and then we can worry about designing the other bits for it.

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 7 Jan 2011 07:27:19 +0100

On Fri, Jan 07, 2011 at 01:44:30AM +0000, Gabriel Montenegro wrote:
> 
> John Tamplin wrote:
> ???  However, I am really loathe to try and define peer-to-peer operation in the base spec because I think it will unreasonably extend the process of getting the base protocol out the door.
> 
> This prominently talks about client-server not peer-to-peer.
> When people have asked me if p2p is in scope, I???ve always answered ???no, by charter???
> as that is how I???ve interpreted the above.

I agree with that but the term "p2p" can be vague in some circumstances,
as it tends to be used in opposition with browser<->server. Some server
to server architectures (such as webservices) are sometimes seen as p2p
by some people because they don't understand that some of their servers
can also be a client of another one.

So I'd say no to the real p2p usage for now, but yes to the server-to-server
(eg: have a front HTTP server use WS messages over a pool of established
connections to validate auth tokens). This is indeed a client->server
relation where the client here is not a browser. Probably that this part
is not very clear in the current description as the work is mostly focused
on browsers.

Regards,
Willy

---

From: Gabriel Montenegro <[email-masked]>
Date: Fri, 7 Jan 2011 07:18:57 +0000

> connections to validate auth tokens). This is indeed a client->server relation
> where the client here is not a browser. Probably that this part is not very clear in
> the current description as the work is mostly focused on browsers.

The charter is clear about HyBi targeting "clients other than Web Browsers", so this seems like a fine usage of client-server websockets.

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 7 Jan 2011 09:57:37 +0100

I think the in-scope/out-of-scope issue comes back to the points made
recently about flexibility.

I think it is pretty clear that our charter is for a HTTP client (not
necessarily a browser) talking to a server via intermediaries.  But I
think it is also very clear that there is demand for all sorts of
different uses for a protocol like WS that can safely and standardly
penetrate firewalls with bidirectional connections.

So when we are faced with a choice between left or right, and there is
no clear "winner" between the choices, then I think it is entirely
valid to discuss potential out-of-scope use-cases like p2p to evaluate
the choices.   If left prohibits p2p use case, but right is useful for
p2p, then I don't see why we cannot consider that for the sake of
flexibility.

I certainly hate the dismissal of concerns simply because they are out
of scope.  It is one thing to propose a whole bunch of mechanisms for
p2p, and another thing entirely to note that some mechanisms being
proposed will hinder out of scope usage.

regards

---

From: Andy Green <[email-masked]>
Date: Fri, 07 Jan 2011 09:09:17 +0000

> proposed will hinder out of scope usage.

If there's a smart way to integrate everything cleanly in a way that 
doesn't invalidate growth that's great.

But otherwise you have to limit what is under consideration to get an 
actual result shipped.

Or you end up with 80+ versions and a bunch of live competing specs for 
the same thing and no convergence, and no end to the disruptive demands 
rewriting even the most basic stuff from scratch.

-Andy

---

From: Salvatore Loreto <[email-masked]>
Date: Fri, 7 Jan 2011 10:30:54 +0100

> scope. What about the rest of you?

as chair I have to say that at moment "peer to peer" is out of the scope
as drafted in the charter: http://tools.ietf.org/wg/hybi/charters

of course the charter is something that can be changed at anytime,
if there is working group consensus on doing it.

/Sal

---

From: Salvatore Loreto <[email-masked]>
Date: Fri, 7 Jan 2011 10:34:37 +0100

> get something out the door.
I agree with John,
lets keep the base protocol minimal at moment.
Once we have reached consensus on the minimal base protocol we can
start to add other stuff on top of the minimal base.

If there is consensus in doing something in the future we can track it 
in the issue tracker
so we won't forget.

cheers
/Sal

-- 
Salvatore Loreto
www.sloreto.com

---

From: Brodie Thiesfield <[email-masked]>
Date: Mon, 10 Jan 2011 11:02:33 +1000

I understand that this can be addressed later, however supporting peer
to peer in the protocol appears to only require a client to also
support the optional masking of the server responses.

In the case of a browser acting as the server, it will need to
implement the server side of things like handshake and unmasking
client messages. However, given that it is really also a client (with
the same unknown intermediataries, etc), I assume that it would be
desirable to have the same security protections as a simple client,
and so it would also want to send messages using masking.

As long as simple clients can also receive messages from a server with
masking enabled, then there appears to be no reason that peer to peer
wouldn't work with the same security guarantees (for the protocol)
that client->server does.

i.e.
client -> server = masked
server -> client = cleartext | masked

How the server notifies the client that it is going to send masked
responses remains unspecified.

Regards,
Brodie

On Fri, Jan 7, 2011 at 7:34 PM, Salvatore Loreto
<[email-masked]> wrote:
> If there is consensus in doing something in the future we can track it in
> the issue tracker
> so we won't forget.
>
>
> cheers
> /Sal
>
> --
> Salvatore Loreto
> www.sloreto.com
>

---

From: Eric Rescorla <[email-masked]>
Date: Sun, 9 Jan 2011 18:39:01 -0800

Not really. Meaningful p2p operation requires nat traversal

Ekr

---

From: Brodie Thiesfield <[email-masked]>
Date: Mon, 10 Jan 2011 12:00:14 +0900

Where clients aren't behind NAT a direct connection can be used, when
they are then a central hub relay. More complex schemes could be added
later, however simply trying for a direct connection and falling back
to a central relay is possible with the existing protocol.

However this problem is solved though, we will still have a client is
acting as a server and (I am assuming) that client will still want to
send masked data too. In the case of direct connection, where no
central server can modify the data stream, the client on the other end
must unmask the data coming from the server. So the most basic support
for p2p with the current protocol maintaining security guarantees
would only require a client to be able to accept either masked or
unmasked responses from the server.

The security paranoid could then also optionally configure a server to
send masked responses (if supported).

Regards,
Brodie

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 10 Jan 2011 07:39:14 -0800

On Sun, Jan 9, 2011 at 7:00 PM, Brodie Thiesfield <[email-masked]> wrote:
> they are then a central hub relay.

A better approach is to actually do hole punching. Seriously, there is
already IETF work
being spun up in this area. It will be informed by this WG, but it's a
different set of problems.

Given that it's out of scope for this WG, we really should not be
considering it.

-Ekr

More complex schemes could be added

---

From: Brodie Thiesfield <[email-masked]>
Date: Tue, 11 Jan 2011 09:12:07 +0900

> Given that it's out of scope for this WG, we really should not be

Yes, I agree that hole punching would be better, and when the work is
done it can be used. I'm not asking the WG to design p2p into the
protocol. I'm suggesting that the protocol have a flexibility that
would permit a basic p2p usage, rather than precluding it by design.

The essential points are:
* basic direct + central hub p2p is almost possible with the current protocol
* since both client and server are browsers, both client and server
will be wanting to send masked data
* therefore the client needs to be able to receive and decode masked frames

All that is required to enable simple p2p then is for the masking to
be designed such that the recipient can detect that it is being used
and unmask the data.

To do this, either the client must be informed out of band by the
server that masking will be used for the server->client channel too,
OR a frame must be self-describing with the masking details
(extensions, reserved bits). The current push for masking to be placed
into the extension data would do this.

Regards,
Brodie

---

From: John Tamplin <[email-masked]>
Date: Mon, 10 Jan 2011 21:11:56 -0500

> into the extension data would do this.

It seems to me to be totally separate as to how the masking is
actually applied -- you could easily define an extension "servermask"
which specifies that server->client frames will be masked (probably
the same mechanism as client->server, but it doesn't have to be), and
then if that extension is negotiated then you proceed with masking in
both directions.

---

From: Brodie Thiesfield <[email-masked]>
Date: Wed, 12 Jan 2011 09:54:49 +0900

> then if that extension is negotiated then you proceed with masking in

In the current protocol there is no mechanism that allows the server
to inform the client of a required extension. Currently, the server
can only respond with a subset of the extensions that the client has
offered (and according to the /extensions/ paragraph, only a single
item may be selected). This appears to be an oversight as there server
should have a way of requiring extensions from the client, however at
the moment there isn't, which precludes the "servermask" method from
working.

Alternatively, if the masking was made part of the framing or an
extension using the reserved bits whereby the presence or absence
could be simply determined by examining the frame, then the server
could just send the masked frames and the client could detect and
decode them.

Regards,
Brodie

---

From: John Tamplin <[email-masked]>
Date: Tue, 11 Jan 2011 20:22:31 -0500

> the moment there isn't, which precludes the "servermask" method from

Allowing the server to request an extension would require an extra
round-trip.  However, in this case, the client can't possible receive
the masked server frames if it doesn't know about it, so a client
which implements this extension simply advertises it.  Most servers do
not accept the servermask extension (by default, since they don't know
about it).

> could just send the masked frames and the client could detect and

One objection is that is using one of our precious reserved bits for
something which is generally going to be always on (for the client) or
always off (for the server) -- that doesn't seem like a good use of
resources.  I believe others also raised the objection that it
shouldn't be optional at all for particular classes of client/servers,
so again it is allocating a bit in every frame for something known to
be constant.

I think the existing extension negotiation mechanism suits this case
fine already.

---

From: "Alexander Philippou" <[email-masked]>
Date: Wed, 12 Jan 2011 09:15:04 +0200

> Currently, the server can only respond with a subset of the extensions that the client has offered (and according to the /extensions/ paragraph, only a single item may be selected). This appears to be an oversight as there server should have a way of requiring extensions from the client, however at the moment there isn't, which precludes the "servermask" method from working.

I believe that it does permit multiple extensions, but you are right that it may be read otherwise too. May I suggest a minor change to the text:

1 /extensions/
2    A (possibly empty) list representing the protocol-level
3    extensions the server is ready to use.  If the server supports
4    multiple extensions, then the value of each extension should be
5    derived from the client's handshake and match exactly one of the
6    values in the "Sec-WebSocket-Extensions" field.  The absence
7    of such a field is equivalent to the null value.  The empty
8    string is not the same as the null value for these purposes.

The changes are in lines 4 and 5.

Alexander

---

From: Salvatore Loreto <[email-masked]>
Date: Wed, 12 Jan 2011 08:24:07 +0100

On 1/12/11 2:22 AM, John Tamplin wrote:
> On Tue, Jan 11, 2011 at 7:54 PM, Brodie Thiesfield<[email-masked]>  wrote:
I fully agree with John here:
we do not need to complicate the extensions negotiation part.
Based on the 04, in the case we want the client be able to unmask
(or if you prefer the server to mask)
the client, willing to receive masked frames, has to advertise
that it supports this particular extension during the handshake.

cheers
/Sal

-- 
Salvatore Loreto
www.sloreto.com

---

From: Brodie Thiesfield <[email-masked]>
Date: Wed, 12 Jan 2011 20:13:12 +0900

> not accept the servermask extension (by default, since they don't know

That makes sense and fits with the HTTP model of Accept headers.

> I think the existing extension negotiation mechanism suits this case

Yes, I agree.

So to support this basic level of p2p, a well known extension (i.e.
"servermask") would need to be defined, and the client would need to
be able to handle server masked frames.

Regards,
Brodie