Subject: #131: Connection limits (proposal)

From: Mark Nottingham <[email-masked]>
Date: Mon, 19 Oct 2009 14:37:19 +1100

<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/131>

OLD p1 7.1.4:

>   Clients that use persistent connections SHOULD limit the number of
>   simultaneous connections that they maintain to a given server.  A
>   single-user client SHOULD NOT maintain more than 2 connections with
>   any server or proxy.  A proxy SHOULD use up to 2*N connections to
>   another server or proxy, where N is the number of simultaneously
>   active users.  These guidelines are intended to improve HTTP  
> response
>   times and avoid congestion.

NEW:

"""
Clients (including proxies) SHOULD limit the number of simultaneous  
connections that they maintain to a given server (including proxies).

Previous revisions of HTTP gave a specific number of connections as a  
ceiling, but this was found to be impractical for many applications.  
As a result, this specification does not mandate a particular maximum  
number of connections, but instead encourages clients to be  
conservative when opening multiple connections.

In particular, while using multiple connections avoids the "head-of- 
line blocking" problem (whereby a request that takes significant  
server-side processing and/or has a large payload can block subsequent  
requests on the same connection), each connection used consumes server  
resources (sometimes significantly), and furthermore using multiple  
connections can cause undesirable side effects in congested networks.
"""

--
Mark Nottingham     http://www.mnot.net/

---

From: "William A. Rowe, Jr." <[email-masked]>
Date: Sun, 18 Oct 2009 23:01:54 -0500

Mark Nottingham wrote:
> ceiling, but this was found to be impractical for many applications. As
> a result, this specification does not mandate a particular maximum
> number of connections, but instead encourages clients to be conservative
> when opening multiple connections.
> 
> In particular, while using multiple connections avoids the "head-of-line
> blocking" problem (whereby a request that takes significant server-side
> processing and/or has a large payload can block subsequent requests on
> the same connection), each connection used consumes server resources
> (sometimes significantly), and furthermore using multiple connections
> can cause undesirable side effects in congested networks.
> """

Is it worthwhile to add the caveat;

"""
Clients attempting to establish simultaneous connections SHOULD anticipate
the server to reject excessive attempts to establish additional connections,
and gracefully degrade to passing all requests through the successfully
established connection(s), rather than retrying.
"""

---

From: Adrien de Croy <[email-masked]>
Date: Mon, 19 Oct 2009 17:41:23 +1300

as discussed previously, I'm not convinced that concurrent connections
is the (only?) thing that should be limited, or that this should be the
responsibility of the client (rather than the server or an
intermediary).

Server resources are becoming cheaper every day (memory / disk / CPU).
So I believe the best place to restrict usage is at the server, where
the operator then has a choice about how much service will be
provided.  Putting a responsibility on the client takes away this
choice from the server operator.

If you made cellphones, and decided to restrict the number of calls it
would make in any 1 day, do you think you'd find a telco that would
sell it?

Furthermore, putting restrictions into the protocol makes it
independent of application.  For instance an in-house client server
system using HTTP may have no need or desire for any sort of limit.

I think therefore the best we can do is encourage implementors to act
responsibly, and consider other users of the network (where there are
any).

So, what about something like:

"Implementors of client applications SHOULD give consideration to
effects that a client's use of resources may have on the network (both
local and non-local), and design clients to act responsibly within any
network they participate in.  Some intermediaries and servers are known
to limit the number of concurrent connections, or rate of requests.  An
excessive number of connections has also been known to cause issues on
congested shared networks.  In the past HTTP has recommended a maximum
number of concurrent connections a client should make, however this
limit has also caused problems in some applications.  It is also
believed that any recommendation on number of concurrent connections
made now will not apply properly to all applications, and will become
obsolete with advances in technology."

this then potentially covers any resource that should be managed, e.g.
not just connections, but perhaps also bandwidth, cache space on an
intermediary etc etc etc

Regards

Adrien.

William A. Rowe, Jr. wrote:

Mark Nottingham wrote:

NEW:

"""

"""

--
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com

---

From: "William A. Rowe, Jr." <[email-masked]>
Date: Sun, 18 Oct 2009 23:55:46 -0500

Adrien de Croy wrote:
> the operator then has a choice about how much service will be provided. 
> Putting a responsibility on the client takes away this choice from the
> server operator.

Which says nothing about the client expectations, which was the point of my
initial response.  The responsibility has been assumed by servers and their
intermediate firewalls/load balancers to ferret out abusive traffic.  Any
poorly constructed client can and will fall into such traps.

> obsolete with advances in technology."

This does not address my specific concern, which is to beat into the implementors
heads not to aggressively retry parallel connections where none will be permitted;

>> established connection(s), rather than retrying.

We appear to be in agreement, but addressing two different aspects of the same.

---

From: Adrien de Croy <[email-masked]>
Date: Mon, 19 Oct 2009 18:54:34 +1300

Adrien de Croy wrote:

Some of the history of this was the 2 connection limit implemented in
IE7, which caused havoc with sites like facebook, especially through a
proxy.  So I guess we are seeing the background to this from different
angles.

What do you mean by client expectations?

Do we therefore need some wording on how a client should detect such
cases, and respond?  E.g. since there's no specific status code for a
rejection based on number of concurrent connections, it would at best
be an assumption that this is occuring.   I agree hammering away would
be problematic.

Should there be a new status code for this?  Otherwise we need some
sort of back-out algorithm to reduce demand client-side.  I think that
could cause other problems as well if there is no explicit way to
signal this particular problem.

Regards

Adrien

"""

---

From: "William A. Rowe, Jr." <[email-masked]>
Date: Mon, 19 Oct 2009 00:55:56 -0500

Adrien de Croy wrote:
> rejection based on number of concurrent connections, it would at best be
> an assumption that this is occuring.   I agree hammering away would be
> problematic.

The larger issue is that server implementors (rightfully) do proceed to blacklist the
relevant IP addresses as abusive, whether at the firewall/load balancer or http server
demarcation.  My comment was ment to illustrate that this behavior is correct, and
that the client behavior is invalid.

We can't implement a server response code to a dropped/refused connection; it is up
to the client to determine that a second/third/fourty-fifth connection is unwelcome
and stay at level or back off from consuming excessive services.

---

From: Adrien de Croy <[email-masked]>
Date: Mon, 19 Oct 2009 20:11:49 +1300

> that the client behavior is invalid.
but the client in this case - what is the chance of it being a 
human-operated browser?

I've seen cases where the server starts responding with a 503 (google 
maps does this if you scroll around for too long).

If the connection rate is deemed high enough to be a DOS attack, that is 
a different issue - one I'd be reluctant to try and address in HTTP.

If there will be no response to the client, I think all bets are off, 
but which clients are we talking about?  Custom attack clients?  They 
will ignore any requirement they like anyway.

Otherwise if a site blacklists an IP due to bona fide human activity, I 
would argue perhaps the threshold is too low.  People can create other 
problems by setting arbitrary limits, arbitrarily deciding what 
constitutes abuse vs acceptable use etc.

None of these I believe would be solved by a recommendation in HTTP though?

> and stay at level or back off from consuming excessive services.

I think this could be solved with a status code.  Disconnecting at a 
firewall gives an ambiguous message.

---

From: Jim Gettys <[email-masked]>
Date: Mon, 19 Oct 2009 07:22:51 -0400

Adrien de Croy wrote:
> responsibility of the client (rather than the server or an intermediary).
> 
> would make in any 1 day, do you think you'd find a telco that would sell it?

This analogy is badly flawed.

More connections does not result in higher performance.

And such a client can cause congestion, as TCP's congestion control does 
not detect multiple connections from the same client, degrading not only 
the user's client, but others as well.

It is much as if someone opened 50 cell phones all at once and dialed at 
the same time: you can be sure that your service (and others) were 
degraded badly).
                     - Jim

> 
> Furthermore, putting restrictions into the protocol makes it independent 
> of application.  For instance an in-house client server system using 
> HTTP may have no need or desire for any sort of limit.
> 
> responsibly, and consider other users of the network (where there are any).
>

---

From: "William A. Rowe, Jr." <[email-masked]>
Date: Mon, 19 Oct 2009 13:30:31 -0500

> when opening multiple connections.

It really seems like this is ripe for a Connection: max=# tag recommendation.
Wherein the application can recommend a number of parallel connections that
1) they support and 2) provide optimal user/application experience.  But this
would be out of scope of 2616bis :)

---

From: Jim Gettys <[email-masked]>
Date: Mon, 19 Oct 2009 14:45:37 -0400

> would be out of scope of 2616bis :)

The problem is that congestion can occur anywhere in the path from 
client to server, and therefore any information the server can give on 
number of connections is incorrect.  So this idea doesn't work.

(One of) the fundamental issues is that many/most implementations of 
HTTP (or other application protocols, for that matter) do not understand 
that in current implementations, it is extremely important to read the 
data from the operating system so that you don't delay acks: otherwise, 
TCP cannot  run at anything near full speed.

We showed in the HTTP/1.1 paper that additional parallel connections did 
not actually increase performance; fastest performance was achieved with 
a single TCP connection.  But for many bad implementations, doing so 
will, without the implementers having to actually think or restructure 
their code.

It may (or may not) be as pernicious (to others) than at the time 2616 
was first issued, as in that era, buffering in edge router equipment was 
very limited, and besides screwing yourself, you may be less evil than 
then.  But I've not seen any data (nor thought deeply) on how the 
Internet has changed since then.

Exactly what the right recommendations to implementers should be, and 
whether the spec should try to enforce such behavior is a different 
discussion.
				- Jim

---

From: Adrien de Croy <[email-masked]>
Date: Tue, 20 Oct 2009 09:46:34 +1300

Jim Gettys wrote:
> Adrien de Croy wrote:
>> as discussed previously, I'm not convinced that concurrent 
>> connections is the (only?) thing that should be limited, or that this 
>> should be the responsibility of the client (rather than the server or 
>> an intermediary).
>>
>> Server resources are becoming cheaper every day (memory / disk / 
>> CPU).  So I believe the best place to restrict usage is at the 
>> server, where the operator then has a choice about how much service 
>> will be provided.  Putting a responsibility on the client takes away 
>> this choice from the server operator.
>>
>> If you made cellphones, and decided to restrict the number of calls 
>> it would make in any 1 day, do you think you'd find a telco that 
>> would sell it?
>
I disagree.  We're discussing implementing voluntary restrictions in a 
client to ease loads on a service.

Anything done on the client side is voluntary.  Only the 
server/intermediary can enforce anything.  Basic security principles 
state you should never _rely_ on the client to behave.

My point was that if we decide an arbitrary (which it would necessarily 
be) limit on connections made by clients, and get co-operating vendors 
to implement it, then we take away choice from service providers for 
what level of service they are then able to provide a client.  If I 
wrote a service that relied on a client opening many connections, I'd be 
hosed.  I'd also be stupid for relying on it, but in the past we set 
arbitrary limits, and it caused problems.  Any number we pick now will 
likely just be a problem for later.  The limit of 2 has been in the spec 
for 13 years, but wasn't widely implemented AFAIK for quite a while (and 
when it was, it caused problems).  Can you guarantee any number you pick 
will still be valid in 13 years from now?

> More connections does not result in higher performance.
Agreed, and therefore I believe this provides sufficient incentive for 
bona-fide client vendors to not abuse concurrency.  If you sell a client 
and all it does is create problems for your users etc, then you won't 
sell much.

>
> And such a client can cause congestion, as TCP's congestion control 
> does not detect multiple connections from the same client, degrading 
> not only the user's client, but others as well.
is this on the backbone, or just a single stack?  Or are you talking 
about backbone elevated priority of SYN packets etc?

>
> It is much as if someone opened 50 cell phones all at once and dialed 
> at the same time: you can be sure that your service (and others) were 
> degraded badly).
As a telco, I'd be happy to charge the user for that, and if demand 
outstripped supply, I would have a choice either to extend capacity, and 
earn more, or treat it as a DOS attack.  I believe the choice should be 
the telcos.

So in the HTTP world, the servers have an existing need to be able to 
limit supply, if only to cater for many individual clients.  Since such 
a capability for limiting is required anyway, then that can also cope 
with individual clients doing things the server operator decides are not 
to be allowed.  But putting an arbitrary limit in the client takes the 
choice away from the server operator, which takes away their opportunity 
to provide more service should they choose to, which could limit their 
opportunities for commercial growth.

Adrien

>> effects that a client's use of resources may have on the network 
>> (both local and non-local), and design clients to act responsibly 
>> within any network they participate in.  Some intermediaries and 
>> servers are known to limit the number of concurrent connections, or 
>> rate of requests.  An excessive number of connections has also been 
>> known to cause issues on congested shared networks.  In the past HTTP 
>> has recommended a maximum number of concurrent connections a client 
>> should make, however this limit has also caused problems in some 
>> applications.  It is also believed that any recommendation on number 
>> of concurrent connections made now will not apply properly to all 
>> applications, and will become obsolete with advances in technology."
>>
>> this then potentially covers any resource that should be managed, 
>> e.g. not just connections, but perhaps also bandwidth, cache space on 
>> an intermediary etc etc etc
>>
>> Regards
>>
>> Adrien.
>>
>>

---

From: Adrien de Croy <[email-masked]>
Date: Tue, 20 Oct 2009 09:50:35 +1300

> would be out of scope of 2616bis :)
I think that could create problems, especially with intermediaries.  
E.g. intermediaries and servers may have different limits.  Would you 
treat the max= as an indication of desired number of connections, or 
undesired number of connections etc.

I think until we can signal a client that service is being refused 
because of number of connections, we are going to have trouble dealing 
with this problem.

Regards

Adrien

---

From: Adrien de Croy <[email-masked]>
Date: Tue, 20 Oct 2009 09:59:57 +1300

Jim Gettys wrote:
>
> We showed in the HTTP/1.1 paper that additional parallel connections 
> did not actually increase performance; fastest performance was 
> achieved with a single TCP connection.  But for many bad 
> implementations, doing so will, without the implementers having to 
> actually think or restructure their code.
>
>
depends on how you define performance.  Agreed for the case where you 
are making a single HTTP request to download a large resource.

But for a site with a large number of embedded images or parts (common 
nowadays for a home-page to result in well over 100 requests), then 
serializing requests + latency results in a poor user experience.  
Opening multiple connections and making concurrent requests greatly 
improves user experience.   That's why browsers do it.

I don't think browsers do it to increase throughput due to poorly 
structured code.  Or are you talking about download accelerators?

Adrien

---

From: "William A. Rowe, Jr." <[email-masked]>
Date: Mon, 19 Oct 2009 16:15:15 -0500

> undesired number of connections etc.

Connection: is a hop-by-hop header.  Yes, it would be stripped by some
intermediaries, yes, it is advisory only.  But if something was adopted along
these lines, intermediaries would be able to

> because of number of connections, we are going to have trouble dealing

It is not a problem.  If you are declined 503 or disconnected, throttle back,
don't blast ahead.

Jim Gettys also replied:
> [...]

Yes, but the implication of max= was the -apparent- performance, not the net
total performance.  User agents are throttled on multiple resources, and in
many cases are not aware from the document structure which of the resources
will be bottlenecks.  Adrien explained this well in his follow-up.

What would be nice in this case, is to inform the browser of max=1 where it
is known that the resources are not dynamic, all quickly served from cache,
with no display delays.  Or max=4 to suggest that the page is very complex
and contains a number of nested, dynamic resources which take time to compute.

> whether the spec should try to enforce such behavior is a different

And thus, I labeled this Off-Topic, but it's an interesting discussion to have :)
This is probably as far as the http-wg wants to see this discussion move at the
moment, until the work on 2616bis is complete.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 20 Oct 2009 09:23:43 +1100

I've been thinking the same thing for a while now, and would be  
willing to both work on a spec and get it into implementations.

I was thinking of using a new Keep-Alive token, e.g.,

   Keep-Alive: max-conns=...

but that's just syntax (mostly).

It's true that doing so is not in the current charter of HTTPbis, but  
there's no reason we can't work on a draft.

If you've been following the hybi list, there are some other things  
that may be interesting to work on too; e.g., the end-to-end max of  
various, more fine-grained timeouts than is currently available with  
Keep-Alive: timeout=...

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 20 Oct 2009 09:32:48 +1100

Adrien proposed:

> known to cause issues on congested shared networks.  In the past  
> HTTP has recommended a maximum number of concurrent connections a  
> client should make, however this limit has also caused problems in  
> some applications.  It is also believed that any recommendation on  
> number of concurrent connections made now will not apply properly to  
> all applications, and will become obsolete with advances in  
> technology."

>> established connection(s), rather than retrying.

My .02 - both of these proposed requirements are quite vague and can't  
be tested. I'd put forth that client authors know full well whether  
they're being abusive, and a few sentences in HTTP isn't going to  
convince them not to be. At the most, we might add something like

"""
Note that servers might reject traffic that they deem abusive,  
including an excessive number of connections from a client.
""

(IMHO)

---

From: Julian Reschke <[email-masked]>
Date: Tue, 20 Oct 2009 10:55:00 +0200

Mark Nottingham wrote:
> ...
> Note that servers might reject traffic that they deem abusive, including 
> an excessive number of connections from a client.
> ""
> ...

OK, proposed patch: 
<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/131/131.diff>

Full text including context:

7.1.4.  Practical Considerations

    Servers will usually have some time-out value beyond which they will
    no longer maintain an inactive connection.  Proxy servers might make
    this a higher value since it is likely that the client will be making
    more connections through the same server.  The use of persistent
    connections places no requirements on the length (or existence) of
    this time-out for either the client or the server.

    When a client or server wishes to time-out it SHOULD issue a graceful
    close on the transport connection.  Clients and servers SHOULD both
    constantly watch for the other side of the transport close, and
    respond to it as appropriate.  If a client or server does not detect
    the other side's close promptly it could cause unnecessary resource
    drain on the network.

    A client, server, or proxy MAY close the transport connection at any
    time.  For example, a client might have started to send a new request
    at the same time that the server has decided to close the "idle"
    connection.  From the server's point of view, the connection is being
    closed while it was idle, but from the client's point of view, a
    request is in progress.

    This means that clients, servers, and proxies MUST be able to recover
    from asynchronous close events.  Client software SHOULD reopen the
    transport connection and retransmit the aborted sequence of requests
    without user interaction so long as the request sequence is
    idempotent (see Section 7.1.2 of [Part2]).  Non-idempotent methods or
    sequences MUST NOT be automatically retried, although user agents MAY
    offer a human operator the choice of retrying the request(s).
    Confirmation by user-agent software with semantic understanding of
    the application MAY substitute for user confirmation.  The automatic
    retry SHOULD NOT be repeated if the second sequence of requests
    fails.

    Servers SHOULD always respond to at least one request per connection,
    if at all possible.  Servers SHOULD NOT close a connection in the
    middle of transmitting a response, unless a network or client failure
    is suspected.

    In particular, while using multiple connections avoids the "head-of-
    line blocking" problem (whereby a request that takes significant
    server-side processing and/or has a large payload can block
    subsequent requests on the same connection), each connection used
    consumes server resources (sometimes significantly), and furthermore
    using multiple connections can cause undesirable side effects in
    congested networks.

Also, in the Changes Section:

    Remove hard limit of two connections per server.  (Section 7.1.4)

Best regards,

Julian

---

From: Jim Gettys <[email-masked]>
Date: Tue, 20 Oct 2009 07:42:21 -0400

> moment, until the work on 2616bis is complete.

Note TCP congestion is usually a transient phenomena in the network, not 
necessarily anywhere near an origin server or proxy; I think this idea 
is doomed to fail.

In any case, it would be an addition to HTTP, and should be dealt with 
as a draft on its own, that would progress on its own, and off topic for 
updating 2616.
                      - Jim

---

From: Julian Reschke <[email-masked]>
Date: Wed, 21 Oct 2009 17:55:29 +0200

> <http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/131/131.diff>

Applied with <http://trac.tools.ietf.org/wg/httpbis/trac/changeset/715>.

BR, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 22 Oct 2009 01:57:51 +0100

> improves user experience.   That's why browsers do it.

This is precisely why I keep advocating multiplexing over on the hybi
list, if anyone is subscribed to both...

Multiple connections reduces aggregate throughput (mostly), but
improves the user experience because of the concurrent arrivals.
Resources taking a long time to generate on the server, and large
resources, don't block the arrival of other resources so much.

Good multiplexing can provide a combination of both - maximal
throughput using few or one TCP connection, at the same time as
concurrency.

Which is why HTTP-over-SCTP is being looked at (it is multiplexing
done better than TCP can).  But even over TCP it has potential.

-- Jamie