Subject: Suggestion for NEW Issue: Pipelining problems

From: "Yngve N. Pettersen (Developer Opera Software ASA)" <[email-masked]>
Date: Tue, 17 Jul 2007 13:04:14 +0200

Hello all,

I thought I should mention this particular issue I am seeing.

There are a number of servers that does not handle pipelining at all well.

Problems that have been observed:

   - Handle the first m of n requests in order, then skip to request n  
(IIRC I saw this on IIS/5)
   - Handle the first request in a packet, ignore the rest, then process  
the first in the next, and so on. I have seen this problem in WebSeal 5  
(IBM) where it was fixed, and then recently broke in WebSeal 6, and is  
apparently fixed again recently. The problem is particularly visible for  
HTTPS in Opera 9, but is not limited to HTTPS.
   - Handle one or more requests, then stall
   - Handle a request, then stall when the next arrives.
   - Not quite pipelining: Server sending Content-Length of original  
content, then apply Content-Encoding without removing or updating the  
Content-Length header. I have seen this happen with PHP using global  
compression when the script sets the Content-Length header.

Opera has had to implement quite a lot of heuristics, as well as having  
blacklisted a number of servers, in order to handle or recover (partially)  
 from such problems.

-- 
Sincerely,
Yngve N. Pettersen
 
********************************************************************
Senior Developer                     Email: [email-masked]
Opera Software ASA                   http://www.opera.com/
Phone:  +47 24 16 42 60              Fax:    +47 24 16 40 01
********************************************************************

---

From: Travis Snoozy <[email-masked]>
Date: Tue, 17 Jul 2007 07:22:15 -0700

On Tue, 17 Jul 2007 13:04:14 +0200, "Yngve N. Pettersen (Developer

> There are a number of servers that does not handle pipelining at all
<snip>

... and?

More important than pointing out a bunch of bad implementations is
to point out what's wrong with the RFC that led to the bad
implementations. However, pipelining is pretty cut-and-dried -- I don't
know that the RFC can make it much clearer than it already is. If you
see ambiguities in the RFC that lead to these differing server
behaviors, though, please do identify them.

-- 
Travis

---

From: Eric Lawrence <[email-masked]>
Date: Tue, 17 Jul 2007 08:31:14 -0700

I think Yngve is primarily pointing out that a significant body of the deployed base of HTTP servers do not support pipelining.  Unfortunately, there's no reliable method, a priori, to determine whether or not the remote server is going to handle pipelining correctly, or whether it will either crash or deliver incorrect results.

I don't know that ambiguities or inaccuracies in the RFC are to blame-- I think the problem is that most servers never bothered to try to handle pipelining correctly.  Today, there is a chicken and egg problem: without broader client support for pipelining, servers have little real incentive to do the work to support pipelining, but the large number of broken servers means that the cost to implement pipelining in the client skyrockets.  As the only popular browser that implements and enables pipelining by default, Opera is leading the way from the browser space.

Proposed modifications I've seen to help reduce the need for client heuristics include:

        1. The server provides some form of indication that it intends to support pipelining
        2. Extending #1, the server's response in some way indicates which request it matches (e.g. client request header and matching server response header)

I'm sure there are many more approaches.

Eric Lawrence
Program Manager
Internet Explorer
Want to view and tamper with HTTP(S) traffic?  Try Microsoft Fiddler
Install IE7 - http://install7.com

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Travis Snoozy
Sent: Tuesday, July 17, 2007 7:22 AM
To: Yngve N. Pettersen (Developer Opera Software ASA)
Cc: [email-masked]
Subject: Re: Suggestion for NEW Issue: Pipelining problems

<snip>

... and?

--
Travis

---

From: Travis Snoozy <[email-masked]>
Date: Tue, 17 Jul 2007 08:54:29 -0700

On Tue, 17 Jul 2007 08:31:14 -0700, Eric Lawrence
<[email-masked]> wrote:

> I think Yngve is primarily pointing out that a significant body of
> the deployed base of HTTP servers do not support pipelining.
> Unfortunately, there's no reliable method, a priori, to determine
> whether or not the remote server is going to handle pipelining
> correctly, or whether it will either crash or deliver incorrect
> results.
> 
> I don't know that ambiguities or inaccuracies in the RFC are to
> blame-- I think the problem is that most servers never bothered to
> try to handle pipelining correctly.

<snip>

<snip>

Marking a message "HTTP/1.1" should do that -- clearly, it hasn't
stopped folks from doing an abysmal job at implementing this
and other features. Likewise, nothing prevents this exact same
problem from re-occurring, with badly-behaving servers claiming to
support pipelining via any new identification mechanism proposed[1].

Really, I think that the best solution for this (and many other
HTTP-related problems) is to have a certification process, and
(ideally) a reference implementation. For the server validation at
least, it could even be an automated service (like the W3C (X)HTML
validator).

-- 
Travis

[1] The suggestion does, however, open up the interesting notion of
modularizing HTTP -- which would be nifty, but probably something
more suited for a 2.0 revision.

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 17 Jul 2007 16:17:22 +0100

> behaviors, though, please do identify them.

I think it's important to document practical known problems which
exist on a substantial scale, and suggested workarounds.  There are
RFCs which do this.  RFC 2616 does a little bit, e.g. suggesting that
implementations accept LF or CR LF as a line ending while being clear
that LF alone is not conforming.

If that sort of thing isn't for RFC2616bis, I suggest that there ought
to be _some_ centralised place for it?  

Dealing with broken implementations is an important part of most HTTP
implementations, even though such workarounds ought not to be
standardised, but perhaps suggested for a limited time.

-- Jamie

---

From: Eric Lawrence <[email-masked]>
Date: Tue, 17 Jul 2007 09:06:22 -0700

<< Marking a message "HTTP/1.1" should do that >>

Of course it should, but clearly the practice of lumping multiple capabilities into a single capabilities identifier has not worked out in practice.

I don't think server developers are generally malicious, so I don't think it's likely that those NOT attempting to support pipelining would go out of their way to report that they support Pipelining when they've made no attempt to do so.

While you're right that there's nothing stopping a server from trying to support Pipelining, reporting it does, and having bugs in such implementation, ~that~ is a much more constrained problem than Opera and other clients face today.

I'm not sure how a certification process would help with the deployed servers problem.  If there were some way to make certification mandatory (seems unlikely) then you could prevent deployment of additional broken servers, but it will be MANY years before the existing base of broken servers will be retired.

Eric Lawrence
Program Manager
Internet Explorer

-----Original Message-----
From: Travis Snoozy [mailto:[email-masked]]
Sent: Tuesday, July 17, 2007 8:54 AM
To: Eric Lawrence
Cc: Yngve N. Pettersen (Developer Opera Software ASA); [email-masked]
Subject: Re: Suggestion for NEW Issue: Pipelining problems

<snip>

<snip>

--
Travis

---

From: "Yngve Nysaeter Pettersen" <[email-masked]>
Date: Wed, 18 Jul 2007 01:05:11 +0200

> behaviors, though, please do identify them.

What we have here is, indeed, incorrect implementations.

However, it is something that I think this group should be aware of, and  
perhaps address in some fashion, if possible.

There is quite some variation in the problems encountered, and their  
severity, from the less serious cut-or-hang-the-connection variations, to  
skip-requests to the variations I forgot to mention in my initial mail:  
the ones folding two responses into each other, and the ones that performs  
a response card shuffle. The most serious of the latter include the  
transparent proxies of major internet security suites (the ones I am aware  
of are Panda and Symantec, although I have not yet investigated fully what  
they are doing).

The question is if something can be done with the existing system, for  
example by adding some headers, or should a new system be created?

The two pipeline capable clients I am aware of at present is Opera and the  
Mozilla engine, although Mozilla's is disabled by default. I don't know  
what capabilities IE7 and KDE have in this area.

Serverside, both Apache and IIS 6 are AFAIK fully pipeline capable.

The problems I have been seeing the past few years have appeared to be  
caused by intermediate servers or frontends, which may work as  
loadbalancers or reverse proxies. The problem with these is that there is  
no way to detect them.

---

From: Henrik Nordstrom <[email-masked]>
Date: Wed, 18 Jul 2007 01:37:13 +0200

> loadbalancers or reverse proxies. The problem with these is that there is  

And additionally, if the protocol is extended by a new header allowing
the server to signal it supports pipelining, these intermediary devices
is likely to still screw things up just as badly.

As I see it is the same as Jamie, the only path out of this is a common
HTTP validation test suite, allowing server (and intermediary) authors
to verify their implementation. Co-Advisor is one important step on that
path.

Things like the pipelining problems easily crop up during development,
even on a minor revision, and is often just as easily fixed once the
developer is aware of the problem.

The invalid content-length problem is tricker, as it's root cause is
often more application level than server or individual server
components.. but it's easy to test for, and should be easy to convince
the ones responsible that it's a fault in their implementation..

but i'd also say it should be expected from most servers that they close
the connection if they see an server application (or plugin module)
generating a response with an incorrect content-length, with an error
log entry about the application malfunction. However, I also don't think
many servers do this today..

Regards
Henrik

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 17 Jul 2007 17:14:33 -0700

On Jul 17, 2007, at 4:05 PM, Yngve Nysaeter Pettersen wrote:
> However, it is something that I think this group should be aware  
> of, and perhaps address in some fashion, if possible.
>
> There is quite some variation in the problems encountered, and  
> their severity, from the less serious cut-or-hang-the-connection  
> variations, to skip-requests to the variations I forgot to mention  
> in my initial mail: the ones folding two responses into each other,  
> and the ones that performs a response card shuffle. The most  
> serious of the latter include the transparent proxies of major  
> internet security suites (the ones I am aware of are Panda and  
> Symantec, although I have not yet investigated fully what they are  
> doing).
>
> The question is if something can be done with the existing system,  
> for example by adding some headers, or should a new system be created?

No.  The same broken implementations will just break the new system,
and then we have twice as many broken ways to do the same thing.

The solution is trivial.  When you encounter a broken server, display
an error message (not a stupid modal dialog that stops everything and
prevents further progress until a user presses the OK button, just a
side box that indicates the server is broken because ...).  Convince
the other major browser developers to do the same and all of those
problems will quickly disappear on their own.

....Roy

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 18 Jul 2007 11:06:25 +0100

> problems will quickly disappear on their own.

I don't think "major browsers" will be keen to appear broken to their
users, for the sake of a minor potential performance tweak.  If a site
works fine on other browsers, and has worked fine for years on all of
them, to a user it's the new browser which is broken.

However, a "This site has a faulty HTTP implementation" advisory which
can expand to more detail, like those warning bars which sometime
appear at the top of a page in Firefox, would allow sites to continue
to be used, while motivating developers to remove the unsightly warning.

That said, pipelining is quite a poor way to improve performance, and
even correct implementations need guesswork on the client side, to
guess whether a response will block subsequent responses.

Out of order multiplexing would be _much_ better.

I suspect pipelining was added to HTTP because it's such a trivial and
natural change - in fact no change to many servers - that it was
thought it might be easy to deploy.  And there is no excuse really for
getting it wrong, it really is quite trivial to implement.

But in practice, it's not being deployed much on the internet because
of it's unreliability with some servers and proxies - and worse,
there's no way (at the HTTP level) to detect when it goes wrong.

Given that it's not so easy to deploy, and the silly requirement of
client guesswork even with good servers, I wonder if it would be
better to simply deprecate pipelining, and instead develop a reliably
advertised and broken-proxy-safe flavour of out of order multiplexing.

-- Jamie

---

From: Henrik Nordstrom <[email-masked]>
Date: Wed, 18 Jul 2007 14:28:41 +0200

> there's no way (at the HTTP level) to detect when it goes wrong.

It's deployed actively in quite many batch jobs such as software
upgrades etc, where throughput and TCP niceness is more important than
low latency.

And as the servers these tools are using is verified, all they need to
worry about is proxies..

Regards
Henrik

---

From: Jeffrey Mogul <[email-masked]>
Date: Thu, 19 Jul 2007 10:45:19 -0700

This is obviously out-of-scope for the RFC2616-bis effort, but
it might be worth exploring a way to support out-of-order responses.

In fact, I did that, about 6 years ago: see

   http://www.watersprings.org/pub/id/draft-mogul-http-ooo-00.txt

which proposed a "simple, compatible, and optional extension to HTTP to
allow a server to issue responses out of order".  Note that I chose
to make this a hop-by-hop design; the I-D explains the rationale.
Using the "Connection" header thus satisfies (I think) your desire
to make this "reliably advertised."  That I-D died at the time,
for lack of interest, but I have no objections if someone wants
to try again.

While I share your desire to make this "broken-proxy-safe",
I think that's probably impossible.  The I-D has a detailed
section on what could go wrong.  I think if we tried to insist
on all HTTP features being "broken-proxy-safe", we would
be unable to make any progress.

What I don't understand is why you think that support for
out-of-order responses would motivate us to drop support
for (deprecate) pipelining.  I would think that O-O-O support
would make pipelining even more useful, and vice versa.
Or more precisely, I don't see how it makes sense to reorder
responses unless the requests are pipelined.

-Jeff

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Thu, 19 Jul 2007 19:01:04 -0700

> getting it wrong, it really is quite trivial to implement.

actually pipelining (as far as I remember the debates - it is many years 
ago now) was added in an effort to get HTTP to be a better-behaved network 
citizen.

One interesting feature of one-TCP-connection-per-request is that TCP 
congestion control really doesn't kick in until about packet 6 or so, after 
the slow-start phase is over - and when most TCP connections are shorter 
than 6 packets, it means that a huge percentage of the Internet traffic 
really is travelling the Internet with no congestion at all - leaving the 
Internet unprotected from another round of congestion collapse.

It seems, however, that the Internet has survived the situation - once the 
majority of traffic is (congestion-controlled) filesharing, perhaps the 
not-so-nice HTTP sessions don't count for that much after all....

                   Harald

---

From: "Roy T. Fielding" <[email-masked]>
Date: Fri, 20 Jul 2007 15:55:13 -0700

On Jul 19, 2007, at 7:01 PM, Harald Tveit Alvestrand wrote:
> actually pipelining (as far as I remember the debates - it is many  
> years ago now) was added in an effort to get HTTP to be a better- 
> behaved network citizen.

That was persistent connections.  Pipelining is keeping the outgoing
half of the connection in use while waiting for responses.  Persistent
connections have other benefits beyond congestion control, but that
was its primary reason for introduction.

> One interesting feature of one-TCP-connection-per-request is that  
> TCP congestion control really doesn't kick in until about packet 6  
> or so, after the slow-start phase is over - and when most TCP  
> connections are shorter than 6 packets, it means that a huge  
> percentage of the Internet traffic really is travelling the  
> Internet with no congestion at all - leaving the Internet  
> unprotected from another round of congestion collapse.

Hmm, I haven't seen that.  HTTP requests are typically smaller than the
3000 bytes you are likely to get in even the most congestion-controlled
window.  Pipelined requests actually increase congestion because any
messages left unsatisfied have to be sent again on a new connection.
Receiving several responses in parallel across multiple connections
does cause congestion, but that is a bug in TCP.  TCP is vulnerable
to trivial denial of service attacks because it doesn't share congestion
control across connections to the same host.  The fact that this was
exposed by the popularity of HTTP does not make is solvable by HTTP.

> It seems, however, that the Internet has survived the situation -  
> once the majority of traffic is (congestion-controlled)  
> filesharing, perhaps the not-so-nice HTTP sessions don't count for  
> that much after all....

They never did.  I think you are remembering discussions from 1994-95,
in which the north atlantic lines were maxed out on traffic.  That
problem was solved by the increase in caching with conditional GET
and 304 responses (about 35% of all responses by the end of 1995),
and by the gradual replacement of Mosaic's ridiculous implementation
of Accept headers.

....Roy

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Sat, 21 Jul 2007 11:49:50 -0700

> was its primary reason for introduction.

You're right, I was mixing the two concepts.
Persistence is necessary for pipelining, but it's possible to implement 
persistence without pipelining (and from the conversation, it seems that 
that's the common case).

---

From: Jeffrey Mogul <[email-masked]>
Date: Mon, 23 Jul 2007 14:02:50 -0700

Roy writes:
    
    Pipelined requests actually increase congestion because any
    messages left unsatisfied have to be sent again on a new connection.

Just curious: can you point us to the experimental evidence for this?
That is, evidence that shows that the effect you described outweighs
the congestion that might be avoided when successful pipelining, for
example, reduces the burstiness of non-pipelined TCP connections.

Thanks
-Jeff

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 23 Jul 2007 16:57:21 -0700

> Just curious: can you point us to the experimental evidence for this?

No, just ad hoc observation while viewing systems though tcpdump.

> example, reduces the burstiness of non-pipelined TCP connections.

It depends on what kind of connections you are talking about.
If the connection is so heavily used that the persistence remains
open (either by accident or through special configuration) then
there is no question that congestion will be reduced on average
versus multiple TCP connections.  We can just sum the bits for that.
However, the burstiness of real HTTP traffic is because the
applications have bursty needs.  I don't see how pipelining can
change that without being an artificial request profile or an
unusual application (like Google's spider).

What I've seen in traces is that pipelining does have substantial
benefits up until the point of reaching an application steady-state
(for a browser, that means a web page with all inlined resource
requests complete).  If the client closes the connection at that
point, instead of waiting for the next request to find out if the
connection was kept open by the server, then it minimizes its use
of the network.  Likewise, pipelining has huge benefits for
specialized HTTP services, such as Subversion (when it uses it)
and content management systems.

If the connections are used infrequently and are susceptible to
timeouts occurring while the next request is in transit, then the
data sent across the network far exceeds any average
*congestion-control* benefit obtained by avoiding the separate
connections. A new connection will be required anyway and the
request message is sent twice (though whether that message hit
the network or not seems to depend on the TCP implementation).

That should be testable to find the actual trade-off point where
a client should close the connection, but I have no experimental
evidence and no time to set up an experiment.  In any case,
pipelining is still worthwhile for many reasons, particularly
when we have control over both sides of the connection, so we need
it defined in HTTP regardless of any single client's request profile.

BTW, do you know of any good (experiment-quality) evaluations
of HTTP over SCTP?

....Roy

---

From: David Morris <[email-masked]>
Date: Mon, 23 Jul 2007 17:56:57 -0700 (PDT)

Just a footnote ... with content from the likes of cnn, I've measured 150+
objects retrieved to compose a single page. Not all from the same server,
but a significant opportunity for pipelining/persistent connections, etc.

---

From: Henrik Nordstrom <[email-masked]>
Date: Tue, 24 Jul 2007 15:59:36 +0200

On mån, 2007-07-23 at 16:57 -0700, Roy T. Fielding wrote:

> connections.

Indeed. And also causes a fair bit of other headaches.

There is also the related case of a server forgetting to do a graceful
close when it needs to close the connection after a response, and this
causing even already sent responses in the TCP pipe to get flushed.

>  A new connection will be required anyway and the

And timing details..

> evidence and no time to set up an experiment.

It's very dependent on the connection timeout used by the server, and
there is no standard or even recommendations on what timeouts to use.

It's one of those shortcomings of TCP, missing an advisory close
operation allowing either endpoint to gracefully ask if it's ok to close
the connection.

> In any case,

Yes.

> BTW, do you know of any good (experiment-quality) evaluations

Saw some when I looked into this. But unfortunately I don't have
references..  The conclusions from what I can remember was that it
currently performs rather poorly in comparision to pipelined HTTP over
TCP, part due to the inability to batch fragments from different
messages in the same IP packet to utilize the MTU fully, part from SCTP
implementations not performing as well as they could. But also that SCTP
is fairly young and implementations likely to improve over time likely
making it perform noticeably better.

I don't think there is any question that HTTP over SCTP has a great
potential in reducing the average perceived latency of viewing complex
pages.

Regards
Henrik

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 25 Jul 2007 11:49:14 -0700

> potential in reducing the average perceived latency of viewing complex

I've been wondering about this for a little while, and would very  
much like to see more work in this area. One of the interesting  
questions is how you'd layer HTTP-over-SCTP onto the existing Web;  
Upgrade doesn't quite meet the requirement (it's a whole new  
connection). Seems like one more piece of site-wide metadata to  
advertise...

--
Mark Nottingham       [email-masked]

---

From: "Mark Baker" <[email-masked]>
Date: Wed, 25 Jul 2007 16:00:24 -0400

> connection). Seems like one more piece of site-wide metadata to

I expect a new URI scheme would be required.

Mark.
-- 
Mark Baker.  Ottawa, Ontario, CANADA.         http://www.markbaker.ca
Coactus; Web-inspired integration strategies  http://www.coactus.com

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 25 Jul 2007 13:03:17 -0700

Really? That doesn't seem to help caching or linking very much.  
Although the semantics of the HTTP scheme currently are "that the  
identified resource is located at the server listening for TCP  
connections on that port of that host."

---

From: Stefanos Harhalakis <[email-masked]>
Date: Wed, 25 Jul 2007 23:14:11 +0300

> connection). Seems like one more piece of site-wide metadata to

  Why one would want to implement "HTTP over SCTP"? AFAIK SCTP is not meant 
for this kind of usage. HTTP connections are short lived and have a strict 
requirement for ordered data delivery. They also seem to better match 
the 'stream' instead of the 'message' notion. AFAICS, SCTP should be used for 
long (or at least longer than HTTP) lived sessions that communicate using 
messages instead of a stream and possibly don't require ordered data 
delivery.

---

From: David Morris <[email-masked]>
Date: Wed, 25 Jul 2007 14:42:03 -0700 (PDT)

> connection). Seems like one more piece of site-wide metadata to

Could easily be used between clients and pre-configured proxy servers. For
example mobile clients where the whole execution space is pretty tightly
controlled. This might vastly improve the mobile user's experience while
not requiring a wide roll out to have significant impact.

Might also use a different port so client discovery via a simple probe
would be sufficient. Not overly burdensom on any modern client platform to
remember whether a given server supports SCTP ... at least over the next
hour or two.

Dave Morris

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 25 Jul 2007 15:26:24 -0700

Because of the head-of-line problems which pipelining attempts to  
solve. HTTP connections being short-lived is largely a side effect of  
implementation techniques that more and more servers are overcoming.

---

From: Stefanos Harhalakis <[email-masked]>
Date: Thu, 26 Jul 2007 10:46:16 +0300

> implementation techniques that more and more servers are overcoming.

  Then it is the multiplexing that is needed. What about something like this: 
http://www.w3.org/TR/WD-mux. Isn't it simpler to implement and more network 
friendly? Perhaps by removing all the bandwidth related functionality...

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 27 Jul 2007 13:14:40 +0100

> be unable to make any progress.

At least, we should enumerate the known problems with deployed proxies
- visible and hidden ones - and the extent of the problem, so the
implementors can decide whether to implement particular features, or
if the problems outweigh the benefits for them.

I suspect we can make more advanced HTTP broken-proxy-safe but only by
running it through an overlay network which looks like HTTP/1.0
messages to old proxies.  Something to wait a few more years before
standardising, I think.  (There are implementations already, running
over the internet).

Perhaps the problem of HTTP proxies intercepting traffic, or being the
only available connectivity, will be less prevalent in a few years anyway.

> responses unless the requests are pipelined.

I was suggesting deprecating pipelining-in-order as it is described in
RFC2616, because of the problems clients have of guessing when it's
safe, and when it isn't, due to the responses blocking other
responses.

Of course, overlapping requests is essential for performance, and I
was suggesting that in-order pipelining be deprecated, while replacing
it with a mechanism for pipelined requests with out-of-order responses
- the same as you have in mind.

For a private application I'm working on, which uses a protocol like
HTTP but not quite the same, I do something like that and it works well.

-- Jamie