Subject: Re: [hybi] Hello / idle timeout

From: Willy Tarreau <[email-masked]>
Date: Tue, 1 Mar 2011 09:48:43 +0100

On Tue, Mar 01, 2011 at 07:30:46AM +0000, Andy Green wrote:
> On 03/01/2011 06:15 AM, Somebody in the thread at some point said:
> >On 1 March 2011 17:03, Willy Tarreau<[email-masked]>  wrote:
> >>BTW I'd prefer to see a suggested heartbeat interval than a
> >>timeout,
> >
> >work for me.
> 
> Isn't it better to think of this as "idle timeout" for the channel?  If 
> there's normal traffic coming from the other guy at any time, actually 
> the receiver can take that as a PONG for a PING he didn't have to send 
> and reset his idle timeout.  It's only after the channel is idle for the 
> idle timeout he has to think about making synthetic traffic via PING to 
> confirm all is still well.

Yes, I agree this is an idle timeout. I thought it was obvious but in fact
it was not, given that not all components behave the same.

Regards,
Willy

---

From: <[email-masked]>
Date: Tue, 1 Mar 2011 12:49:25 +0000

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> ext Greg Wilkins
> Sent: 01 March, 2011 08:16
> To: Willy Tarreau
> Cc: Hybi; Gabriel Montenegro
> Subject: Re: [hybi] Hello frames?
> 
> On 1 March 2011 17:03, Willy Tarreau <[email-masked]> wrote:

Hi, I haven't participated in the discussion before, but the issue of heartbeat is so close to my heart it merits a few comments:

Being able to suggest a heartbeat interval is indeed very valuable for mobile devices. How the negotiation actually takes place, and which value gets chosen is a more complex issue. I'm hoping the websocket client on a mobile device would have priority in setting the value since it might know something about the access network characteristics in order to select an optimal (=long) interval. But I understand the server side is keen on getting rid of dead connections sooner than later, preferring perhaps a shorter heartbeat. But, for now I think it is enough to have a placeholder in the protocol where the proposed interval values can be exchanged.

Being able to use a single heartbeat for multiple websocket connections might also be beneficial, but not that critical. In cellular networks, waking up the radio frequently is clearly worse than sending out some more bytes once the radio is on. Clever operating systems might also offer an API for queuing multiple heartbeat messages and sending them out at one go.

> > I'm even thinking that if we want to announce that, we'd better do
> that
> > in the initial HTTP handshake as a header,
> 
> Also fine with that

Is there any difference in practice in having the interval in the HTTP handshake or in a websocket message? In terms of interworking with legacy proxies and other middleboxes there might be? I guess not?

Regards,
Simo

> > That also means that it'd leave the ping usable for any purpose we
> > want :-)
> 
> for a hello frame for example.
> 
> cheers
> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi

---

From: Andy Green <[email-masked]>
Date: Tue, 01 Mar 2011 12:59:17 +0000

On 03/01/2011 12:49 PM, Somebody in the thread at some point said:

Hi -

> Is there any difference in practice in having the interval in the
> HTTP handshake or in a websocket message? In terms of interworking
> with legacy proxies and other middleboxes there might be? I guess
> not?

In practical terms it's probably easier and simpler to define and sell
people on the idea that there's another optional header that can be sent
out at the same time as the client and server handshake headers.

Otherwise you'd have to use one of the scarce reserved opcodes for
setting this kind of thing or overload PING in an ugly way, etc.

Plus the idle limit is really for the entire connection like the initial
headers are.

-Andy

---

From: John Tamplin <[email-masked]>
Date: Tue, 1 Mar 2011 10:28:37 -0500

On Tue, Mar 1, 2011 at 7:49 AM,  <[email-masked]> wrote:
> Being able to use a single heartbeat for multiple websocket connections
> might also be beneficial, but not that critical. In cellular networks, waking
> up the radio frequently is clearly worse than sending out some more
> bytes once the radio is on. Clever operating systems might also offer an
> API for queuing multiple heartbeat messages and sending them out at
> one go.

I don't see how the OS can get involved in it -- imagine the mobile
browser has 10 apps running, and each of them have a hearbeat interval
of 60s.  If they happen to get started at regular intervals, you could
wind up with the case of having to send a keepalive frame every 6
seconds.  The OS only knows that packets are being sent over separate
TCP connections -- how does it know it can delay some of those to
avoid powering up the radio?  If it delays some of them too much, the
connection could be dropped as well.

I really think all of this is a lot more complicated to get right, and
I don't think we should try and put it into the base spec with no
experience of deploying WebSocket on mobile devices to see how it is
going to get used.

Also, applications are likely to want their own timeout, so if you
don't expose some way for apps to control the interval and what
actually happens on it (perhaps a no-op call to their server), then
you are likely to wind up with multiple layers of heartbeat messages.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: John Tamplin <[email-masked]>
Date: Tue, 1 Mar 2011 10:32:12 -0500

> setting this kind of thing or overload PING in an ugly way, etc.

I would still like to move all control frames under a single CONTROL
opcode, and use the first byte of the payload to indicate the control
frame type -- I think it is useful to group control opcodes together
since we specifically do things differently for control frames (they
can't be fragmented, must use the short length format, and can be
injected in the middle of a fragmented data message), plus it frees up
some scarce opcodes and makes it nearly free to add more control
opcodes.

However, there has never seemed to be much support for the idea when
proposed - maybe if we are wanting to add more control opcodes
(including Hello), the idea becomes more interesting.

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 2 Mar 2011 08:39:44 +1100

> some scarce opcodes and makes it nearly free to add more control

I really don't think opcode space is going to be a problem.... it is so hard
to get agreement on even basic features, I don't see any rampant featurism
about to start soon.

But I do think partitioning the opcodes is a good idea:

frame-opcode = %x0 ; continuation frame
             / %x1 ; text frame
             / %x2 ; binary frame
             / %x3-7 ; reserved
             / frame-control-opcode

frame-control-opcode = %x8 ; ping
                     / %x9 ; pong
                     / %xA-E ; reserved
                     / %xF ; close

cheers

---

From: Brian McKelvey <[email-masked]>
Date: Tue, 1 Mar 2011 13:39:50 -0800

I agree.  For mobile developers, for now it seems reasonable that they could use their own custom scheme in coordination with their own server implementation if they really need to specifically tune the heartbeat interval for now, assuming that they're writing a native app and not a mobile web app, and control their own server, which seems likely in this scenario.  Long running background web apps on mobile that continuously maintain a background websocket connection is very unlikely to be a possibility right now for other reasons.

Once we learn more about how this all works together in practice we can define an extension or update to the protocol then.

Brian

Sent from my iPhone

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 2 Mar 2011 09:51:18 +1100

On 2 March 2011 08:39, Brian McKelvey <[email-masked]> wrote:

>
> Once we learn more about how this all works together in practice we can
> define an extension or update to the protocol then.
>
>
But do you think we should be disclosing idle timeouts at this stage, to
help us develop those future plugins.

For example if we see that the clients are declaring idle timeouts of 300s
and the servers declaring 200s, yet we see connections closing frequently at
a 60s, then we will know that we have an intermediary with it's own idle
timeout and that will shape the design of any future keep alives.

So currently the proposal is simply to disclose timeouts (and buffer
sizes).

cheers

---

From: John Tamplin <[email-masked]>
Date: Tue, 1 Mar 2011 18:00:51 -0500

> So currently the proposal is simply to disclose timeouts (and buffer

But what exactly would we be collecting?  If there is no application control
over those values, it will simply be what the user agents send by default
(if they do at all).  Also, how would we actually collect the data?

I just don't see it buying much to put it in before we have anything to do
with it.

---

From: Brian McKelvey <[email-masked]>
Date: Tue, 1 Mar 2011 15:02:38 -0800

I'm not opposed to it.  It seems super simple to me especially if it's just informational and there is no requirement to act on the information.  Someone developing an application where it is more important (such as a mobile app) can choose to act on the data in the fields in their implementation, and/or implement the heartbeat at the application level.

If it's just to provide data points for future evaluation and optimization it seems like a pretty low cost.

The proposal ought to be simple enough that i don't think it should reasonably meet too much resistance, especially if the fields are optional and if someone submits a proposal for how to word it in the draft.

Fwiw, I'd like to see this in the http headers rather than as some kind of hello frame.

Brian

Sent from my iPhone

On Mar 1, 2011, at 2:51 PM, Greg Wilkins <[email-masked]> wrote:

> But do you think we should be disclosing idle timeouts at this stage, to help us develop those future plugins.

---

From: Brian McKelvey <[email-masked]>
Date: Tue, 1 Mar 2011 15:05:43 -0800

Right.  I kindof feel the same way but it seems simple and unobtrusive enough that if Greg wanted to be able to write his server to log the data for research purposes and be able to provide some amount of concrete information to the dialog in the future, im not opposed.

Sent from my iPhone

On Mar 1, 2011, at 3:00 PM, John Tamplin <[email-masked]> wrote:

> I just don't see it buying much to put it in before we have anything to do with it.

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 2 Mar 2011 10:37:06 +1100

On 2 March 2011 10:00, John Tamplin <[email-masked]> wrote:
> (if they do at all).

Indeed - I think a lot more thought would be needed before exposing
such information to the application.

The information would be known only to the websocket layers of the
client and the server.  I assume that initially they would simply be
ignored or logged.  Perhaps the servers could align their idle times
with that of the client.

>  Also, how would we actually collect the data?

Logs

> I just don't see it buying much to put it in before we have anything to do with it.

I don't see how any future keep alive mechanism will be able to be
well designed without the knowledge gained from such disclosures.  It
will have to be based on guess work, or we will just set a stupidly
low timeout value and continue to waste bandwidth.

If we start declaring timeouts now, then intermediaries written to
understand WS will know that timeouts will eventually be explicitly
managed.  They may then be encouraged to declare their own timeouts or
to respect the server/client timeouts.

I know that there is no perfect solution here - but giving up and just
pinging every 30s, does not feel like the right approach.

---

From: John Tamplin <[email-masked]>
Date: Tue, 1 Mar 2011 18:57:27 -0500

> ignored or logged.  Perhaps the servers could align their idle times

But my point is you will just be collecting what we already know (assuming
any user agents provide the information) -- default compiled-in values that
are in the user agents.  So, I don't see the benefit of sending and
collecting something that has no API to change.  At the end of it, there
will be no new information gathered, and in the meantime we just sent lots
of constant bytes over the wire.

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 2 Mar 2011 11:23:47 +1100

On 2 March 2011 10:57, John Tamplin <[email-masked]> wrote:
> are in the user agents.

We don't already know it.   I don't even know my own default values,
as they are taken from configuration that every deployment of jetty
may change.
More importantly, I do know that users do change the values.  Those
that are concerned about working no matter what typically have 30s
timeouts.  Others that are working in intranets or don't care too much
about a non zero drop out rate, will often have timeouts up to 250s.

I have no idea what the timeouts are in chrome, FF, opera etc.   Are
they the same?   Are they configurable?  Are they dynamic?
What about libwebsocket? what about home grown clients?

This is a real problem for comet servers already and I don't see why
websocket will be any different.

> So, I don't see the benefit of sending and

Just because data is not available to the application, does not mean
that it is not meaningful.
The bytes are neither known or constant.

I'm not sure that a

  Sec-Websocket-IdleTimeout: 200

in the handshake could be described as "lots of bytes".
It will certainly be less bytes than if connection keep getting closed
due to no know idle timeouts and the client just opens then up again.

regards

---

From: John Tamplin <[email-masked]>
Date: Tue, 1 Mar 2011 19:31:21 -0500

> about a non zero drop out rate, will often have timeouts up to 250s.

If they don't actually do anything, why would anyone go change it?

Right now they don't exist because the current spec doesn't have them.  If
the spec were to require browsers to send timeout values that aren't
actually used, it seems likely they would send some fixed value.

I'm not sure that a

> due to no know idle timeouts and the client just opens then up again.

I mean collectively lots of bytes across all the WebSocket connections until
a standard exists that makes use of them.  Since as proposed the timeouts
aren't to be used for anything, I don't see how providing them can prevent
any connections from being closed.

To be clear, I am only talking about the time between when they are put in
the spec (now, as you are suggesting) and when they are actually used for
some purpose (some arbitrary time in the future, since you appear to agree
we don't want to wait and try and define how to do something useful with
them in the base spec).

---

From: John Tamplin <[email-masked]>
Date: Tue, 1 Mar 2011 19:37:54 -0500

On Tue, Mar 1, 2011 at 4:39 PM, Greg Wilkins <[email-masked]> wrote:

> I really don't think opcode space is going to be a problem.... it is so
> hard to get agreement on even basic features, I don't see any rampant
> featurism about to start soon.
>

Perhaps, or perhaps once we get the base spec out the door everybody will
propose extensions that were deferred from the base spec.  Even if they are
private extensions, many of these will still need opcode space.

> frame-control-opcode = %x8 ; ping

Fragmenting the limited opcode space seems like a bad idea -- what if we
don't define any new control opcodes but want to define more data opcodes or
vice-versa?  We will start having to have an extension opcode sooner than
otherwise necessary.

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 2 Mar 2011 11:48:33 +1100

> If they don't actually do anything, why would anyone go change it?

Currently the jetty configuration has a maxIdleTime for it's HTTP
connector that is inherited by the upgraded websocket connection.
There are plenty of reasons for users to tune maxIdleTIme for HTTP
connections.

Also, when we use websockets as a transport for cometd, we use the
configuration of the interval and poll times to configure the max idle
time for the connection.   Users are already tuning these parameters
for their environments and the websocket implementation inherits
these.

Besides,  if you really think there is no reason to change - then we
should hard code a timeout value into the spec.

> actually used, it seems likely they would send some fixed value.

If an implementation has no timeout for a connection, then I'd
certainly want to know that.
But I expect that most implementations do have an idle timeout, even
if it is not explicitly in the spec.

> any connections from being closed.

Well they can be used by infrastructure  For example, if the browser
tells me that it has a 45s idle timeout, then I'm going to raise an
error if I see a cometd configuration with a poll timeout of 60s (or
automatically adjust it to < 45s ).

Eventually I'd like to see the protocol use ping/pongs to do this, but
currently we need the frameworks (or applications )to do keep alives,
and on the server side, these frameworks would have access to the
client declared timeouts.

> we don't want to wait and try and define how to do something useful with

Put them in the spec today, and cometd will start using them tomorrow.

cheers

---

From: John Tamplin <[email-masked]>
Date: Tue, 1 Mar 2011 19:52:55 -0500

> Put them in the spec today, and cometd will start using them tomorrow.

And that is a problem -- what has been proposed is agreed to be a half-baked
thing just to collect data, yet here you are suggesting something will
actually start using it when we haven't addressed the related issues that
have been discussed on this thread.

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 2 Mar 2011 12:29:49 +1100

> have been discussed on this thread.

No that is not what I'm saying.

So I never said that declaring timeouts is half baked.  It is valuable
meta information that can be used today.
Declaring that the protocol MUST send ping/pongs at that rate would be
half baked.

It is probably too hard for us to achieve consensus for a keep alive
mechanism in the protocol itself, but that declared timeouts would be
a good first step towards a standard keep alive mechanism.

Failing a standard keep alive mechanism, frameworks and applications
will have to implement their own keep alives - websocket applications
will just not work without them!    Declaring that a problem is too
hard for us to solve, does not make the problem go away, so there will
be regular keep-alive-like messages sent no matter what we do in the
specification.

If we provide a standard way for timeouts to be declared, then at
least on the server side, that information can be used to guide the
frameworks and applications in choosing a frequency (which may still
prove to be wrong due to intermediaries, but that is valuable
information to know and at least the framework could start any backoff
strategy at a realistic timeout).

If you are really worried about the bytes - then we could define
control packets to query configuration parameters, but that sounds
like a lot more faff than just

  Sec-WebSocket-IdleTimeout: 200s

cheers

---

From: John Tamplin <[email-masked]>
Date: Tue, 1 Mar 2011 20:36:52 -0500

> be regular keep-alive-like messages sent no matter what we do in the

And I am suggesting that applications, which as you agree must have them,
must implement them without use of the information you would propose to add
in the handshake because it is not exposed to them and wouldn't actually
trigger either keepalives or automatic timeouts of the connection.
 Therefore, providing the information in the handshake, which applications
then ignore and build their own, seems pointless.

Anyway, I think we have both said our points and I don't think we are
changing each others' mind.  My preference remains to leave it out of the
spec until it can be defined in such a way to actually be used for the
intended purpose, rather than just having it there just to be there.  If
others think it is useful to have it now, then fine.

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 2 Mar 2011 12:45:25 +1100

> trigger either keepalives or automatic timeouts of the connection.

Why is this information not available on the server?

There is no standardised API there that would need to be changed and
the current suggestion is to send this information as a simple HTTP
header, which will very likely be available to at least the framework.

> changing each others' mind.

However, I  do think we are understanding each other minds better and
also I think there are some  assumption (eg above) that may not be
valid and may influence your (and others) mind.

cheers

---

From: Salvatore Loreto <[email-masked]>
Date: Wed, 2 Mar 2011 15:41:33 +0200

On 3/2/11 1:02 AM, Brian McKelvey wrote:
> On Mar 1, 2011, at 2:51 PM, Greg Wilkins<[email-masked]>  wrote:
>
> _____________________________________________
(as individual)

last year Martin, Greg and I have written down the following (now 
expired) draft
as a possible improvement for cometd
(we presented it during the IETF meeting in Maastricht (in July 2010) 
during the APP area meeting)

http://tools.ietf.org/id/draft-loreto-http-timeout-00.txt

if there is enough interest we can refresh the draft (as an individual 
submission for the time being)
and discuss in this wg

cheers
/Sal

---

From: Salvatore Loreto <[email-masked]>
Date: Wed, 2 Mar 2011 15:47:43 +0200

(as individual)

I think the proposal has same value
and it is worth to be discussed in a separate thread

/Sal

-- 
Salvatore Loreto
www.sloreto.com

---

From: Gabriel Montenegro <[email-masked]>
Date: Wed, 2 Mar 2011 19:00:50 +0000

I think it's fine to pursue this on a separate draft. Let's not deal with it within websockets, both because it is not specific to websockets (as seen from the draft below) and to not add more stuff we need to converge on before 07 and Prague.

> Sent: Wednesday, March 02, 2011 05:42
> To: [email-masked]
> Subject: [hybi] http-timeout (was Re: Hello frames?)
> 
> (we presented it during the IETF meeting in Maastricht (in July 2010) during the
> APP area meeting)
> 
> if there is enough interest we can refresh the draft (as an individual submission
> for the time being) and discuss in this wg
> 
> cheers
> /Sal
>

---

From: Gabriel Montenegro <[email-masked]>
Date: Wed, 2 Mar 2011 19:01:38 +0000

Not sure I seem much need for HELLO, so I wouldn't call that a justification to reopen framing discussions.

> Sent: Wednesday, March 02, 2011 05:48
> To: [email-masked]
> Subject: [hybi] a single CONTROL opcode for all control frames (was Re: Hello
> frames?)
>

---

From: John Tamplin <[email-masked]>
Date: Wed, 2 Mar 2011 14:13:49 -0500

> Not sure I seem much need for HELLO, so I wouldn't call that a justification to reopen framing discussions.

My suggestion would be to define a CONTROL opcode, and then for that
opcode the first byte of the payload defines which control frame is
present.  The reasons are:

- it makes it easier to identify control frames, which are referenced
elsewhere in the spec (limitations on size, fragmenting, and the
ability to inject them in the middle of a fragmented data message.
When new control frames are added, the check for whether something is
a control frame or not remains accurate

- the number of available opcodes seems small, and over time it seems
certain we will need to define a way of extending them at some point,
similar to what I propose.  Control frames should be much less
frequent than data frames, so using an extra byte for control frames
instead of some future data frame type will be more efficient.

- by having a larger number of control opcodes available with
practically no additional cost, proposals for a new control frame that
is somewhat speculative won't see resistance solely about using scarce
opcode resources.

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 3 Mar 2011 07:44:40 +1100

> a control frame or not remains accurate

+1 on this reasoning

> instead of some future data frame type will be more efficient.

-1 on this.

There are spare opcodes and currently there does not appear to be great
demand for them.
If there was, then we would be free to create an opcode that means - look
for the real opcode in the first byte of payload.   This is essentially what
you are proposing, but we can defer until needed.

> is somewhat speculative won't see resistance solely about using scarce

+0   I think opcodes should always be well allocated, no matter how
plentiful.

So to repost my counter proposal.  We can partition the current opcodes into
normal and control:

cheers

---

From: John Tamplin <[email-masked]>
Date: Wed, 2 Mar 2011 15:57:02 -0500

On Wed, Mar 2, 2011 at 3:44 PM, Greg Wilkins <[email-masked]> wrote:
> There are spare opcodes and currently there does not appear to be great demand for them.
>
> If there was, then we would be free to create an opcode that means - look for the real opcode
> in the first byte of payload.   This is essentially what you are proposing, but we can defer until
> needed.

In the discussions of framing, there were many different proposals of
potential extensions, and there were two primary methods proposed for
them - extension metadata and extension opcodes.  To get a framing
spec done, we compromised on a middle-ground that allows both, but
also means there are fewer opcodes available.  If all of those
extensions being discussed every happened, we would already be out of
opcodes and that's with what we can think of today, much less the next
5 or 10 years.

> +0   I think opcodes should always be well allocated, no matter how plentiful.

As an example, I haven't seen anything suggested from a Hello frame
that would make me think the value of including it now outweighs the
cost of using a scarce opcode.  If instead it is using an extended
control opcode out of 256, then I am less inclined to object to using
one of the opcodes for something I don't personally see much value in
that others do.

> So to repost my counter proposal.  We can partition the current opcodes into normal and control:
>

My objection to this is that we still are defining now exactly how
many control opcodes there are and how many data opcodes there are,
and we don't know how many we will need of either.  Code is likely to
be written that defines isControl(opcode) as opcode & 8, so when we
need to create extension opcode space would that mean we need to have
two primary opcodes allocated, one for extended data opcodes and one
for extended control opcodes?

Also, what happens if we decide that we don't need as many reserved
bits for per-frame flags, but we do need more opcodes so we allocate
adjacent reserved bits to the opcode field?  Is control-ness of an
opcode still based on the 0x8 bit?

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 3 Mar 2011 08:25:25 +1100

John,

your proposal still sets a limit on the number of normal/control op-codes
as15/, while my proposal  is 8/8. If you exhaust the 15 normal codes, then
another primary opcode would have to be allocated, it is the same for both
proposals.

I think we are basically saying the same thing, the only real difference is
that you split primary normal/control opcodes as 15/1 and assume that we
need to immediately go for  0/256 in the payload.     I'm proposing we split
8/8 and go for 256/0, 0/256 or 256/256 in the payload only when we need to.

Eitherway, I'm supportive of the basic premise if a clear indication of what
are control frames.  Your proposal is totally workable - but I have a slight
preference for my variation.

cheers

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 3 Mar 2011 16:40:37 +0000

> Declaring that the protocol MUST send ping/pongs at that rate would be

First, if any of you find this mail too long, please skip to the last
paragraph, which is actually the most important.  Summary: the client
Javascript needs to know the timeout of its *own* WebSocket to do
client-initiated ping/pong, and your header puts the information in
the wrong place for that.

Back to keepalives.

Compared with two-way idle keepalives, ping/pongs are:

   (a) Wasteful of bandwidth (you don't need keepalive messages in
       both directions if one direction is actively moving data,
       but you still need keepalive in one direction - if the data
       sending side needs to detect a dead connection quickly.)

   (b) Overly sensitive to network latency variance (you have to
       subtract latency variance _twice_ from the timeouts to get the
       sending rate with ping/pongs).

   (c) Unnecessarily symmetric (the server may not care if the
       connection is gone for a few minutes, but the client probably
       wants to know ASAP, so it can launch a new connection.  That
       can approximately halve the number of keepalive messages.
       But in some applications, the server needs to know ASAP, so it
       can register that a client is offline on some status screen)

   (d) Not really simpler to implement than independent two-way idle
       keepalives, because you need a timeout mechanism at both ends
       in both cases anyway, and other factors as described below.

The only thing ping/pong has going for it, imho, is everyone thinks
they know what is intended by it.  But...?

That isn't actually clear: Does ping/pong mean send a ping every N
seconds, or only after N seconds of idleness?  If it means _every_ N
seconds, if the connection is saturated with real data, the ping or
pong may be delayed by the data, leading to "false timeout" even
though the connection is working perfectly.  If idleness, there a few
questions about what is idleness, when large messages and/or full
sending buffers happen.

And _who_ sends the ping - client or server?

Inside a browser it is even more exciting: the browser has to keep
reading and queuing messages for Javascript, even if the Javascript is
unable to keep up or busy with something else, and cannot bound the
memory used by simply pausing TCP, if pings must be read and responded
to quickly.  So there's a potential browser memory DOS side effect.

> a good first step towards a standard keep alive mechanism.

Only if the *meaning* of those timeouts is specified.

If a client says its timeout is 45s, but the acceptable network
latency variance is 5s, the server needs to transmit something every
40s to keep connected.  At the same time, if the server says its
timeout is 180s, the client needs to transmit something every 175s
seconds to keep the server happy.

But if it's PING/PONG, then the client needs to transmit PING every
35s to keep the connection going.

The difference between 175s and 35s is significant, yet both are
derived from the proposed timeout values - depending on the timeout
mechanism in use.

So not only the values matter - how they will be used matters.

> will just not work without them!

Yes.  Ian Hixie's tiny "tic-tac-toe" example suffers from this.  It
locks up on some networks if you don't make a move for a minute or two.

>   Sec-WebSocket-IdleTimeout: 200s

I think the more important thing is to define what that 200s actually
means.  Does the other end need to ensure it sends:

   (a) start of a message every 200s (minus network latency),
   (b) _any part_ of a message every 200s (minus network latency),
       (that matters when there are large messages, which may take a
       while to send),
   (c) an explicit "ping" every 200s (minus network latency twice for RTT),
   (d) an explicit "keepalive" every 200s (minus network latency),
   (e) an explicit TCP event every 200s (minus network latency),

> strategy at a realistic timeout).

I think if the applications are performing keepalive _itself_ - then
its up to the applications how they declare the keepalive parameters
_and_ what keepalive method they are using.

My applications *won't* use ping/pong for their keepalive, if it's
avoidable.

But the added wrinkle is we have application timeouts *and* WebSocket
implementation timeouts below the application.

It's quite a messy laying violation, but also unavoidable as long as
both the application and the WebSocket implementation below it both
have to implement timeout mechanisms - possibly different ones.

What I would really dislike is if an application has to keep sending
redundant empty messages over the net to keep the WebSocket
*implementation* at the other end happy, even though the application
at the other end knows the connection is working fine - just because
the application has no API to tell its own WebSocket that the
connection is still running fine.

Which brings me to something you haven't suggested:

The proposed Sec-WebSocket-IdleTimeout: tells the _other_ end what
timeout the generic WebSocket will use at this end.  But how does
_this_ end's application learn what its own WebSocket's timeout is?

That information is crucial: You can't PING/PONG from the client
Javascript unless you know the client-side WebSocket timeout, and your
Sec-WebSocket-IdleTimeout header *does not give that value to the code
which needs it*.  So real applications will have to receive it on the
server, and then send it in an application-specific control message
back to the client *anyway*.  The correct place for it is in API that
the local Javascript can read, and maybe change, but nobody dares add
to the API.

-- Jamie

---

From: John Tamplin <[email-masked]>
Date: Thu, 3 Mar 2011 11:52:13 -0500

When we first discussed Ping/Pong frames, my recollection was that we
were specifically not trying to decide on how
heartbeat/keep-alive/idle timeout/etc would be implemented, but rather
just providing a facility similar to ICMP ECHO at the WebSocket layer
for diagnostics and perhaps future use for such things.

My question is:  do we really need to decide on
heartbeat/keep-alive/idle-timeouts etc now, or can we ship the
protocol without it (knowing that means applications will build their
own facility on top of it) and then later add it to the protocol when
we have actual usage data to look at to help drive consensus?

My preference would be to keep the protocol simple now and add it
later, in the interest of getting something out soon.  However, if the
consensus is that the base protocol must include this functionality,
then we have a lot more work to do including discussions like this.

---

From: "Joonas Lehtolahti" <[email-masked]>
Date: Thu, 03 Mar 2011 19:04:19 +0200

> then we have a lot more work to do including discussions like this.

I agree with this. The protocol should IMO just state how to handle  
incoming Ping message (reply with Pong), but doesn't need any statement on  
actually using them. If a client or a server wants to use them, it can  
decide to send them when it sees it as proper and act as it wishes for not  
receiving pong message in whatever timeout value it desires to use.

This would also include the liberty of choosing whether to expose  
Ping/Pong to application from the websocket framework or not. For web  
browsers that question would of course be delegated to the Websockets API  
working group, for non-browser applications the exact implementation is a  
concern for only that application's developer and therefore they have free  
hands to use ping (or not use it) anyway they wish, unless limited in the  
websockets protocol specification. And I think the usage shouldn't be  
mandated.

Other might have (and do have) differing opinions...

---

From: David Endicott <[email-masked]>
Date: Thu, 3 Mar 2011 12:07:27 -0500

My understanding is that websocket is a "framed" transport protocol
sitting on top of TCP.   I am already uncomfortable with Ping/Pong
frames (not to mention "Hello") as I view that functionality as
properly belonging to a higher application layer.   I see no good
reason to introduce application layer mechanisms (time-outs,
heartbeats, etc.) into a transport layer protocol.

My vote is to exclude it as being outside the proper scope of this protocol.

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 03 Mar 2011 09:17:55 -0800

> then we have a lot more work to do including discussions like this.

Should the following be removed from 4.5.3 "Pong", following that line 
of thinking?

   A Pong is issued only in response to the most recent Ping.

Otherwise, the spec is making a decision against unidirectional keepalives.

There's also the option of pushing ping/pong to an extension if more 
time/experience is needed to get the definition right.

-- Scott

---

From: Bruce Atherton <[email-masked]>
Date: Thu, 03 Mar 2011 09:25:48 -0800

On 02/03/2011 11:13 AM, John Tamplin wrote:
> present.

+1. Simply from a design perspective it makes more sense. Text frame, 
binary frame, continuation frame, and control frame all feel like the 
same level of abstraction. Breaking control down into ping, pong, and 
close seems like an abstraction imbalance. And what do you gain by 
taking this deep-dive on the control frames? Saving a single byte from a 
frame that is already tiny?

Plus there are the other benefits you mention. I just think this makes 
sense.

---

From: Gabriel Montenegro <[email-masked]>
Date: Thu, 3 Mar 2011 17:26:34 +0000

> Should the following be removed from 4.5.3 "Pong", following that line of
> thinking?
> 

[gab] Unidirectional keepalives were not originally discussed, so I would list them as stuff to be done afterwards. I would much rather we keep the basic Ping/Pong as in the spec, with the above which simplifies the implementation a lot (and provides one element of a rudimentary but effective DoS protection).

> time/experience is needed to get the definition right.

[gab] Given that it's been there for a long time, in the interest of having a simple spec soon, I'd rather we kept it as is, but postpone further refinements, timeouts etc for  later.

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 03 Mar 2011 09:41:02 -0800

> [gab] Unidirectional keepalives were not originally discussed, so I would list them as stuff to be done afterwards. I would much rather we keep the basic Ping/Pong as in the spec, with the above wh…

That's not accurate, actually. Jamie Lokier raised similar issues about 
unidirectional keepalives well before we had any opcodes at all.

One of the reason Willy suggested the ping/pong in the first place, as I 
recall, was to allow for unsolicited pongs.

The sentence forbidding unsolicited pongs appeared without any 
discussion that I remember.

> [gab] Given that it's been there for a long time, in the interest of having a simple spec soon, I'd rather we kept it as is, but postpone further refinements, timeouts etc for  later.

There's a difference. If the spec defines a request-response keepalive 
but forbids unsolicited keepalive, and if that's the wrong answer, it's 
doing more harm than good.

That's not the same as restricting the spec to a simple definition.

-- Scott

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 3 Mar 2011 17:52:04 +0000

> then we have a lot more work to do including discussions like this.

I agree with all of you that it should be left to the application for
now (and maybe forever).

However, Greg has correctly identified there is one area where timeout
behaviour may already be occurring in WebSocket below the application,
even if it is not specified anywhere.

That is, when the WebSocket implementation times out a connection it
thinks is dead, even if the application above it continues to just sit
there waiting for data.

HTTP servers universally do this, even though it is not specified, and
I guess HTTP clients do it too.  Generic WebSocket server
implementations will probably tend to do it as well - just to avoid
building up unbounded dead connections.

Maybe that's just an implementation detail that applications (or
application-specific protocols, when 3rd parties talk to each other)
are expected to be aware of, and WebSocket implementations are
expected to be configured appropriately for the application running
over them; I'm not sure.

There is no reason I can see why browser WebSockets should implement a
timeout at all.  If the Javascript application keeps the WebSocket
object around without closing it, then let it stay open, so that
application timeout policy is always followed.  There is no need to
protect applications that don't timeout themselves, unlike on the
server where it is more useful, but it can remain something that
WebSocket does not need to mandate.

-- Jamie

---

From: Gabriel Montenegro <[email-masked]>
Date: Thu, 3 Mar 2011 18:52:15 +0000

> discussion that I remember.

[gab] The intent was to simply say that *if* responding to a Ping, only the last Ping need to be answered. I'm fine if folks want to see this clarified by adding "*if/when responding to a Ping*, only the last Ping is responded to." I thought that was implicit, so was not quite sure why you claimed it prohibited unsolicited Pongs, even if it was not clear to me that was already allowed in the draft.

---

From: Scott Ferguson <[email-masked]>
Date: Thu, 03 Mar 2011 11:31:51 -0800

> [gab] The intent was to simply say that *if* responding to a Ping, only the last Ping need to be answered. I'm fine if folks want to see this clarified by adding "*if/when responding to a Ping*, on…

Ah.

In that case the wording would be clearer as something like "MAY send a 
pong for only the most recently received ping" because the current "is 
issued only" sounds like a MUST (and is ambiguous as to the context.)

MUST would also be also because a client cannot rely on a server 
skipping pongs.

(which is why the MAY/MUST convention is useful instead of wording like 
"is issued only")

-- Scott

---

From: Andy Green <[email-masked]>
Date: Thu, 03 Mar 2011 20:38:19 +0000

> the local Javascript can read, and maybe change, but nobody dares add

FWIW the selected protocol itself is another potential source of timeout 
knowledge, ie, both sides can understand because it is "superchat-1.0" 
protocol that was negotiated for the connection the idle timeout for the 
channel is 15s or whatever.

-Andy

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 4 Mar 2011 08:35:33 +1100

With regard to removing ping/pong,  I've already found them very useful
testing interoperability between different servers and clients -
specifically jetty to libwebsockets.  It would be a pity to lose this easy
test vector.

I think unsolicited pongs should also be supported.

> even if it is not specified anywhere.

Exactly.   Either the infrastructure does not attempt to timeout the
connection OR it must divulge it's timeouts.  Either is workable.   I think
the only thing that is not workable is the application being in control of
keep alives, but having zero knowledge of what timeouts it has to meet.
That will just result in frequent empty messages or ping/pongs.

Note however, that because of intermediaries that will also be timing out
connections, I think that keep alives will be somewhat of a black art for
the the foreseeable future.  Even if the client has no timeout (good idea)
and the server knows it's own timeout,  there will still be unexpected
closes from NAT routers and the like.  It is likely that keep alive packets
will be needed and their frequency will need to be dynamically adjusted for
the path - perhaps with a cache of known frequencies for paths.        I
don't expect many applications will go to such lengths, so websockets usage
may just result in unreliable applications or overly frequent keep alives.
   Punting the hard problems to the application does not make them simpler.

cheers

---

From: John Tamplin <[email-masked]>
Date: Thu, 3 Mar 2011 16:52:32 -0500

>    Punting the hard problems to the application does not make them simpler.

No, but it does prevent us baking in a solution that doesn't actually
solve the problem at hand because we have no experience with how
WebSockets will actually be deployed.

---

From: [email-masked]
Date: Fri, 4 Mar 2011 02:28:09 -0800

> Note however, that because of intermediaries that will also be 
> timing out connections, I think that keep alives will be somewhat of
> a black art for the the foreseeable future.  Even if the client has 
> no timeout (good idea) and the server knows it's own timeout,  there
> will still be unexpected closes from NAT routers and the like.

Right. A while ago, I had to detail with deciding ping packet interval to 
keep TCP NAT binding 
alive, and ended up with sending keepalive packets every 60 seconds. The 
interval timeout 
was determined from my research with more than 50 routers, and minimum 
time I have seen was 
5 minutes, and from another similar research done by Saikat@Cornel Univ. 
seemed to have
indicated that TCP NAT binding kept alive for more than 4 minutes.
http://saikat.guha.cc/stunt-results.php? 

IETF behave WG attempted to define NAT behavior against TCP including 
binding lifetime:
http://tools.ietf.org/html/rfc5382 
The document requires TCP idle connection timeout MUST NOT be less than 2 
hours. 
However, this document came out not a long ago, so we cannot assume that 
all NATs
in the wild follow this spec.

As far as I know, when TCP NAT binding timeouts, the NAT does not send 
anything. (no FIN nor RST)
Therefore, when client or server sends a packet after that, the packet 
will be dropped
at the router as it no longer has route for the packet. If you are lucky, 
you may get ICMP
error that could trigger closing connection, but as you know you cannot 
assume ICMP 
packet to arrive all the time. Client would likely end up with 
retransmitting the packet for a while 
(usually 10 - 30 minutes - see tcp(7)) then finally the endpoints give up 
the connection.

To make application code simpler, sending ping/pong every 1 to 3 minutes 
for 
inactive websocket sounds like a good idea to me. Timeout for pong to come 
back
would also help detecting such broken pipe earlier.

- Yutaka

[email-masked] wrote on 03/03/2011 01:35:33 PM:

> 
> With regard to removing ping/pong,  I've already found them very 
> useful testing interoperability between different servers and 
> clients - specifically jetty to libwebsockets.  It would be a pity 
> to lose this easy test vector.
> 

> connection OR it must divulge it's timeouts.  Either is workable.   
> I think the only thing that is not workable is the application being
> in control of keep alives, but having zero knowledge of what 
> timeouts it has to meet.  That will just result in frequent empty 
> messages or ping/pongs.
> 
> Note however, that because of intermediaries that will also be 
> timing out connections, I think that keep alives will be somewhat of
> a black art for the the foreseeable future.  Even if the client has 
> no timeout (good idea) and the server knows it's own timeout,  there
> will still be unexpected closes from NAT routers and the like.  It 
> is likely that keep alive packets will be needed and their frequency
> will need to be dynamically adjusted for the path - perhaps with a 
> cache of known frequencies for paths.        I don't expect many 
> applications will go to such lengths, so websockets usage may just 
> result in unreliable applications or overly frequent keep alives.

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 4 Mar 2011 11:25:42 +0000

[email-masked] wrote:
>    Right. A while ago, I had to detail with deciding ping packet interval
>    to keep TCP NAT binding
>    alive, and ended up with sending keepalive packets every 60 seconds.
>    The interval timeout
>    5 minutes, and from another similar research done by Saikat@Cornel
>    Univ. seemed to have

I have often used 3G mobile routes that timed out in less than 1
minute consistently, and sometimes in about 30 seconds.
So for those I sent keepalive every 15 seconds - 3O seconds was
unreliable.  That was last checked about 3 years ago.

>    [1]http://saikat.guha.cc/stunt-results.php?
>    [2]http://tools.ietf.org/html/rfc5382
>    The document requires TCP idle connection timeout MUST NOT be less
>    than 2 hours.
>    However, this document came out not a long ago, so we cannot assume
>    that all NATs

That is absurdly unrealistic for routers with limited memory when a
large number of connections are made through them and those
connections don't terminate cleanly, thus keeping NAT state open.
When just one person is using certain P2P networks at home, I have
seen 30 or more TCP SYNs _per second_ going out - over a 2Mbit ADSL.
It's not immediately clear what kind of demand that places on NAT
table size with a timeout of 2 hours, not least because it depends how
many connections become established then get stuck, but I suspect it's
more than current home routers allow for.

>    anything. (no FIN nor RST)

Yes, that is my experience.  It just drops the NAT state, sometimes to
make room for another one.

>    at the router as it no longer has route for the packet. If you are
>    lucky, you may get ICMP
>    error that could trigger closing connection, but as you know you
>    cannot assume ICMP
>    (usually 10 - 30 minutes - see tcp(7)) then finally the endpoints give
>    up the connection.

Yes, that is my experience.

However even worse, if the client doesn't send anything, it's just
_assumes_ it receives notification messages from the server when
whatever it's subscribed to changes, then the client _never_ times out
at the TCP level.

That is why client applications have to timeout.

The same is true on the server, to avoid accumulating an infinite
number of dead sockets, and if it needs to know when subscribers have
left.

>    To make application code simpler, sending ping/pong every 1 to 3
>    minutes for inactive websocket sounds like a good idea to
>    me. Timeout for pong to come back would also help detecting such
>    broken pipe earlier.

Who sends the ping - the client or the server?

What if the connection is active in one direction and idle in the
other direction?  Does that count as idle or not?

Independent keepalives and timeouts are more efficient.

Ping/pong means you have to send packets more often for the same level of robustness.

And it's useless if your application needs to detect broken connection
more quickly, to start another connection, or connect to a backup
site, or notify the user that the link is down.

An online board game, or live Twitter display for example, or live
stock price display, needs to detect broken connection more quickly
than 1 to 3 minutes, so it can reconnect or notify the user of a problem.

For that you need to send keepalives quite often.  Then the
latency-doubling effect of the ping/pong pattern becomes more
significant.

-- Jamie

---

From: [email-masked]
Date: Fri, 4 Mar 2011 23:54:17 -0800

> unreliable.  That was last checked about 3 years ago.

Ouch.. I looked for any known spec for idle connection 
timeout with 3G but I could not come across any... Do you 
know any?

> connections don't terminate cleanly, thus keeping NAT state open.

Agree. The behave WG's scope includes carrier grade NAT, may not be 
practical for residential routers. The document also says "MAY" to
drop existing bindings for new connections, it does not help our 
issue anyway..

> That is why client applications have to timeout.

Agreed.

> Who sends the ping - the client or the server?

In SIP, specifically, RFC 5626("Client-Initialted Connections in SIP",
Section 4.4), UAC(User Agent Client) sends ping, not from a server,
but it allows the server to tell client "how long it can wait for ping
before it concludes that connection is dead" with an optional header,
'Flow-Timer' header in its REGISTER method transaction. If the header 
is present, client adjust its ping/pong timers reflecting what server 
says with Flow-Timer header.

The UAC waits for 'pong' after sending 'ping' up to 10 seconds before
it gives up the connection, which is described in section 4.4.1. with
some rationale.

In the section, ping/pong interval time is defined as 120 seconds
where battery power is not a concern, and 840 seconds where battery
power is a concern. In it's rationale paragraph, it is assumed that 
TCP NAT timeout it as low as 15 minites, which differs from what I have
seen in the past, and may not work for other reasons of connection
loss. (e.g. 3G as you pointed out)

Anyways, I just thought we may want to look into SIP's solution to
this issue as what they are trying to achieve looks exactly the same
as WebSocket's, and their method received consensus at least within SIP
community. 

- Yutaka

---

From: <[email-masked]>
Date: Mon, 7 Mar 2011 08:30:51 +0000

> From: ext John Tamplin [mailto:[email-masked]]
> Sent: 01 March, 2011 17:29
> To: Veikkolainen Simo (Nokia-MS/Helsinki)
> Cc: [email-masked]; [email-masked]; [email-masked];
> [email-masked]

Coming back to this... Using these kind of API requires some co-operation from the application as well, as it would typically need to define a tolerance range within which the heartbeat can be sent. Maybe not perfect but might save some power on a mobile.

Simo

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 9 Mar 2011 15:13:15 +1100

Ian,

What are your thoughts about this as editor?  there does not appear to
be any opposed and there is support for Johns original proposal.
It would be good to know soon if you think this will be in -07, as it
affects the server APIs somewhat.

cheers

On 4 March 2011 04:25, Bruce Atherton <[email-masked]> wrote:
> +1. Simply from a design perspective it makes more sense. Text frame, binary
> frame, continuation frame, and control frame all feel like the same level of
> abstraction. Breaking control down into ping, pong, and close seems like an
> abstraction imbalance. And what do you gain by taking this deep-dive on the
> control frames? Saving a single byte from a frame that is already tiny?
>

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Tue, 8 Mar 2011 20:54:14 -0800

I am rather ambivalent, however if we do it we should do it now. I do agree
with the following:

1. It would be nice to understand what is a "control" frame, especially if
an extension wants to define its own definition of a control frame that
should not be fragmented. If we had a single control frame opcode and the
first byte defined which control it was (with some number reserved for
extensions), this would be much clearer.

The other points about scarcity of opcodes or making it lower cost hold some
value to me, but frankly not nearly as much as the first.

I will make the change unless anyone objects strongly.

On Tue, Mar 8, 2011 at 8:13 PM, Greg Wilkins <[email-masked]> wrote:

> binary

---

From: Salvatore Loreto <[email-masked]>
Date: Wed, 9 Mar 2011 12:17:15 +0200

(as co-chair I want to bring your attention to this)

Hi there,

the following changes (proposed by John Tamplin) in 07:

    /to define a CONTROL opcode, and then for that opcode the first byte
    of the payload defines which control frame is present
    /

has been discussed during the last couple of weeks:
(see the thread 
http://www.ietf.org/mail-archive/web/hybi/current/msg06726.html )

the proposal has gather some consensus and so 07 will include this 
change unless anyone object strongly!

cheers
/Sal

-- 
Salvatore Loreto
www.sloreto.com

On 3/9/11 6:54 AM, Ian Fette (イアンフェッティ) wrote:
> I am rather ambivalent, however if we do it we should do it now. I do 
> agree with the following:
>
> 1. It would be nice to understand what is a "control" frame, 
> especially if an extension wants to define its own definition of a 
> control frame that should not be fragmented. If we had a single 
> control frame opcode and the first byte defined which control it was 
> (with some number reserved for extensions), this would be much clearer.
>
> The other points about scarcity of opcodes or making it lower cost 
> hold some value to me, but frankly not nearly as much as the first.
>
> On Tue, Mar 8, 2011 at 8:13 PM, Greg Wilkins <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Ian,
>
>     On 4 March 2011 04:25, Bruce Atherton <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     > [email-masked] <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>