Subject: [hybi] Keep-Alive analysis

From: Martin Thomson <[email-masked]>
Date: Thu, 29 Mar 2012 07:21:19 +0200

I think that the discussion on Keep-Alive yesterday was quite
productive.  I've had some time to consider this and I'd like to share
those thoughts.  It's a very long email, but I think that I've reached
some conclusions.

The arguments against a mechanism were all predicated on two things:
connection termination events are largely unpredictable, and that
keep-alives (ping/pong as opposed to the proposed header) are
expensive.  I should point out that from a formal logic perspective,
keep-alives are not implied by the draft, but it is certainly
something that this enables.

There are a few things that are missing from the argument, but there
is a compelling case in here.  Ultimately, any decision to use the
mechanism will be a subjective trade-off, as is the case for many such
engineering decisions.  So I'd also like to share some of the
reasoning that goes with my particular trade-off.

I'm not that much of a statistician, so after a little reading, it
became clear that I couldn't really apply anything from reliability
analysis (MTBF and those sorts of thins).  Over-analysis paralysis
almost caught me.  It's better (and simpler) to think of the problem
like so:

In the absence of a declared policy, the state of an idle connection
over time is largely determined by the probability of ...stuff
happening.  In the absence of a declared policy, ...stuff includes
termination by the remote end due to expiration of a timer (i.e. their
policy).  At the point that this probability reaches some non-trivial
proportion, then the state of the connection is no longer reliable.

Note that this probability only really covers the probability of
connection badness that might affect the next request that the client
attempts.  This is important.  This covers the probability that the
connection has gone away in the past without notification (for
example, because a TCP RST was suppressed or lost) and the probability
that the connection will fail between the time that the client makes
the request and the response is received.  Probability of failure
therefore increases with RTT.

You can imagine that an client with an understanding of these
probabilities might use this information to advise their behavior.
Given a sufficiently low probability of connection badness - such as
might be the case immediately after establishing the connection, or
after having received data - the implementation might simply use the
connection without precaution.  However, as the subjective view of the
probability of connection failure increases beyond a certain
threshold, the client might alter its behavior.  For instance, a
client might avoid the use of non-idempotent requests, or use some
sort of liveness test.

If a client does not alter behavior in response to its understanding
of the probability of connection loss, then there is no point in any
mechanism.  I'll get back to what I care about later.

Where policy is declared, the overall probability function can be
considered to have a step, where the probability of disconnection
jumps at the declared idle connection timeout.  Over the time period
allowed by the minimum declared policy, it is only the probability of
...stuff happening that influences whether a connection is usable,
declared policy is not a factor.

There are certainly cases where knowledge of policy does absolutely no
good.  If, as was indicated yesterday, a 45 second TCP keep-alive is
necessary to deal with some class of unreliable middleboxes that can't
be reliably detected, then something like this is not going to be very
much use.  (I happen to think that in this case, extreme behavior of
that sort shouldn't be triggered until something bad happens).
Certainly, if you don't actually implement a policy, or you don't have
one imposed on you, then it simply doesn't make sense to declare one.

But I think that in some cases, this knowledge can be enabling.

Modern operating systems are starting to constrain access to network
resources, primarily for power management reasons.  This is especially
true of mobile platforms where radio is very expensive, but Windows 8
is imposing some tight constraints on native applications that are
idle.  In Windows 8, an idle application is given one "lifeline"
connection.  Being able to rely upon that connection is really
important.  I'd like to think that the connection will be an HTTP/?
connection.

In practice, the probability of losing a connection due to random
events is quite low, despite some of the rather evocative examples to
the contrary.  More often than not, it's the server policy, or (in the
case of Windows Azure) the loadbalancer right in front of the server,
that drops the connection.  And I know that the Windows Azure load
balancer doesn't send a TCP RST when it cleans house, as much as I
might like it to do otherwise.

Because I care so strongly about this connection, I am prepared to pay
(in processing time, battery, etc...) to keep it.  I will send
keep-alive messages just to make sure that the connection is
available.  But I really want to pay as little as possible.  That's
where declarative timeout policy helps out.  If I have some idea what
these policies are, then I can make certain that I send keep-alives as
infrequently as possible.  That matters to me a lot, because the
difference between 45s keep alives and 10m keep alives is probably the
difference between needing a stupidly large backup battery for my
weekend trip and not.

In some sense, I recognize that this is saying: "people are going to
do bad things (i.e. waste effort on keep-alives), let us help make it
less bad", but I think that's still a valid argument.

One conclusion that I've reached is that sending a Keep-Alive header
from a client is actually more wasteful than helpful.  It's actually
the server policy that is most useful.  A client is usually
constrained more by the needs of the application than by any sort of
resource management during active operation.  The server, in a passive
role, really doesn't benefit from knowledge about client policies.
It's the client - in periods of idleness - that is most affected by
the indeterminate state of the connection.

--Martin

p.s. Chrome sends Connection: Keep-Alive without a Keep-Alive header.
I'd be interested to learn why those bytes are still being spent given
the (current) status of Keep-Alive.

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 29 Mar 2012 07:37:17 +0200

Hi Martin,

interesting reading, you gave me some ideas I'll probably try to
describe to you today. In the mean time I have a response to your
last comment below :

> the (current) status of Keep-Alive.

This is simply to benefit from persistent connections via HTTP/1.0 proxies :-)

Willy

---

From: Tobias Oberstein <[email-masked]>
Date: Thu, 29 Mar 2012 00:32:01 -0700

I'd like to add: there are use cases where it isn't only about keeping a connection _alive_, but keep it fast.

Mobiles and carrier networks will downgrade idle connections from 3.5G to 3G. That makes a difference
in RTT between 300-800ms (3G) to 80-120ms (3.5G).

I have seen that being done within seconds of idle connections. I have seen it being downgraded even with
a 1 sec heartbeat in place. Probably because the bandwidth consumed by the connection was deemed too
low to justify a 3.5G channel.

So if your prime goal is to keep up a very low latency connection, then you even might need very short
heartbeat (3-20s), with substantial payload.

---

From: Martin Thomson <[email-masked]>
Date: Thu, 29 Mar 2012 09:42:27 +0200

> heartbeat (3-20s), with substantial payload.

That's grossly wasteful.  But whatever you need.

---

From: Tobias Oberstein <[email-masked]>
Date: Thu, 29 Mar 2012 00:47:04 -0700

> > So if your prime goal is to keep up a very low latency connection,
> > then you even might need very short heartbeat (3-20s), with substantial
> payload.
> 

Yes it is.

It wouldn't be needed if there were 1) mechanisms for a mobile devices
to prohibit downgrading a 3.5G connection to 3G and 2) an API usable
from apps to control that.

As far as I know neither 1) nor 2) is there. If you know how to do that,
I'd be happy to learn!

---

From: Roberto Peon <[email-masked]>
Date: Thu, 29 Mar 2012 09:50:48 +0200

It is all tradeoffs. For some applications, perhaps the  increased
signaling cost, bandwidth cost, cpu cost, and battery-life cost is a
reasonable tradeoff. At least until the carrier attempts to ban the
application and/or charge you for it :)

-=R

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 29 Mar 2012 10:54:12 +0200

Hi,

> application and/or charge you for it :)

The problem is that in an ideal world we'd like the application to
communicate QoS preferences to the upper layers (including the OS).

There should be a lot of thinking done around mobile-specific usages
and tradeoffs I think, as these are not specific to websocket.

Willy

---

From: Roberto Peon <[email-masked]>
Date: Thu, 29 Mar 2012 02:29:46 -0700

I agree that the issues are not solely owned by the WG, but the
implementation is.

In an ideal world, the battery wouldn't be consumed so much by the radio
and the signaling protocols would be cheaper for all parties. Saving that,
a mechanism for synchronizing periodic updates so the radio had few/minimal
state changes would be good... that may be something we should talk about
finding or chartering a WG to discuss...
On Mar 29, 2012 1:54 AM, "Willy Tarreau" <[email-masked]> wrote:

> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>

---

From: Tobias Oberstein <[email-masked]>
Date: Thu, 29 Mar 2012 09:21:45 -0700

> The problem is that in an ideal world we'd like the application to communicate
> QoS preferences to the upper layers (including the OS).
> 
> There should be a lot of thinking done around mobile-specific usages and
> tradeoffs I think, as these are not specific to websocket.

I did some experiments.

On the HSPA network I use, I need 4kb/s traffic to keep the radio in fully active,
low-latency HSPA DCH state. Lower rates won't cut it. Payload is WS Ping/Pong.

http://lists.w3.org/Archives/Public/ietf-http-wg/2012JanMar/1083.html

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 30 Mar 2012 11:34:06 +1100

> the indeterminate state of the connection.

The way that I think about it, is that the client is the entity that
best knows if a connection should be persisted or not.  It knows if
the chat room is open or if the portfolio is displayed, and thus it
knows if the UI is still interested in receiving updates, not matter
how infrequent they may be.  The problem is that the client does not
know what it has to do to keep itself in the connected state and I
agree that the client sending a keep-alive;timeout=x header does not
help.   It does not tell the server if the client wants to keep the
connection open to stay in a connected state, or if it is simply
suggesting a time in which it might use the connection again.

In the HTTP scenario, I think a client sending keep-alive;max=y  can
be useful, as it can tell the server if the client is interested in
persisting the  connection or not, but I think connection:close on the
last request also provides that capability.  I can see how ;max can be
applied to WS.

So I agree that the real value is in the keep alive message from the
server to the client.  This tells the client something concrete that
it can take action about.  If the application wants to stay connected,
then it must at least send some data (message or ping) as frequently
as indicated in the servers keepalive.

Although, we must remember that keep-alive is hop by hop, so it is not
so much the servers keep-alive, but a keep-alives sent from the
direction of the server towards the client.

cheers

---

From: Martin Thomson <[email-masked]>
Date: Fri, 30 Mar 2012 09:54:22 +0200

On 30 March 2012 02:34, Greg Wilkins <[email-masked]> wrote:
> best knows if a connection should be persisted or not.

That's the key.  As we've seen, some clients do some borderline
lunatic things to meet their goals, and better information can help
them recognize when they can do less than completely bat-guano.

> I can see how ;max can be applied to WS.

Can you elaborate, or did you mean "can't"?

> direction of the server towards the client.

Yeah, good point.  To be perfectly clear, the word "server" as I have
used it in this discussion applies to the serving end of the
_connection_, not the request.

--Martin

---

From: Tobias Oberstein <[email-masked]>
Date: Fri, 30 Mar 2012 01:32:15 -0700

> The way that I think about it, is that the client is the entity that best knows if a
> connection should be persisted or not.  It knows if the chat room is open or if

Tend to agree. Keeping a connection alive, and in some use cases not only alive,
but in a low-latency state, is something the client should control.

Even when that does not only mean consuming resources on the client, but
also on the network and server side. In a user-centric world, the user should
be in control.

At least for HSPA connections, keeping a connection in low-latency state seems
to involve hacks, lack of optimized radio states and device APIs.

Imagine a mobile 2 player Memory-like game. Messaging volume is very small,
but if there is a message, it needs to be low-latency.

It's an example where the client not only wants to keep a connection alive, but
more so in a low-latency state.

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 30 Mar 2012 11:19:18 +0200

> _connection_, not the request.

That's my point too, so I think we're all in line :-)

Willy

---

From: Roberto Peon <[email-masked]>
Date: Fri, 30 Mar 2012 11:52:52 +0200

yup

---

From: Greg Wilkins <[email-masked]>
Date: Sat, 31 Mar 2012 01:26:54 +1100

> Can you elaborate, or did you mean "can't"?

opps - s/can/can't/