Subject: Fwd: I-D Action:draft-nottingham-http-pipeline-00.txt

From: Mark Nottingham <[email-masked]>
Date: Tue, 10 Aug 2010 11:39:56 +1000

FYI. I see this as the start of a discussion more than anything else.

Cheers,

Begin forwarded message:

> From: [email-masked]
> Date: 10 August 2010 11:30:02 AM AEST
> To: [email-masked]
> Subject: I-D Action:draft-nottingham-http-pipeline-00.txt 
> Reply-To: [email-masked]
> 
> A New Internet-Draft is available from the on-line Internet-Drafts directories.
> 
> 	Title           : Making HTTP Pipelining Usable on the Open Web
> 	Author(s)       : M. Nottingham
> 	Filename        : draft-nottingham-http-pipeline-00.txt
> 	Pages           : 9
> 	Date            : 2010-08-09
> 
> Pipelining was added to HTTP/1.1 as a means of improving the
> performance of persistent connections in common cases.  While it is
> deployed in some limited circumstances, it is not widely used by
> clients on the open Internet.  This memo suggests some measures
> designed to make it more possible for clients to reliably and safely
> use HTTP pipelining in these situations.
> 
> This memo should be discussed on the [email-masked] mailing
> list, although it is not a work item of the HTTPbis WG.
> 
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-nottingham-http-pipeline-00.txt
> 
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
> 
> Below is the data which will enable a MIME compliant mail reader
> implementation to automatically retrieve the ASCII version of the
> Internet-Draft.
> _______________________________________________
> I-D-Announce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i-d-announce
> Internet-Draft directories: http://www.ietf.org/shadow.html
> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

--
Mark Nottingham     http://www.mnot.net/

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 10 Aug 2010 07:35:08 +0200

Hi Mark,

> FYI. I see this as the start of a discussion more than anything else.
(...)
> http://www.ietf.org/internet-drafts/draft-nottingham-http-pipeline-00.txt

I found interesting things in this memo (I never realized that 1xx
responses were causing difficulties for pipelined responses). I'm also
realizing that it may be the beginning for the possibility for out-of-order
responses. But I think that copying the full request into a response header
will have some drawbacks which will still limit its adoption :

  - it will noticeably increase response length for some static servers
    which usually return small objects, or even many 304 responses.

  - it forces the server to keep a copy of the URI during the processing.
    While with most application servers this is not a problem, this can
    be one on static servers which otherwise don't have to keep a buffer
    allocated once they start to respond. Also, depending on the URI
    length, performing the copy itself might add a few more CPU cycles to
    process the response (similar to large cookies in fact).

  - on servers, this will have to be done unconditionally eventhough at
    the beginning a very few number of clients will consider it. This
    further delays adoption because there is no perceived added value on
    the server side, and adding some processing cost for a little minority
    of clients means that it will be long before we find it everywhere.

  - it's common for some gateways (caches, etc...) to rewrite parts of
    URLs when forwarding requests to servers. Sometimes, a path prefix
    is changed, sometimes they just perform some normalization by
    resolving unneeded %-encoding, etc... In that case, the server will
    return a response that does not match the client's request. Until
    the gateways make the rewrite rules configurable by the admin, this
    will simply break clients which rely on this behaviour and it may
    result in this option being disabled by default in browsers.

I think that a request identifier could be a lot better (I noticed the
point about it at the end but don't agree :-)).

In my opinion, what is important is an identifier related to the
connection, since the problem we're having is that we need to distinguish
several requests/responses on a same connection. So basically a client
would send a request counter with each request over the same connection,
that the server would simply echo. The advantages I see with this method :

  - smaller for servers, and they don't have to emit a response if they
    don't see the request header ;

  - it is immediately deployable, client adoption rate will be perceived
    from the servers, which will incite them to adopt it too ;

  - does not break with existing intermediaries : if they don't support
    it, it's simply just as before, and we rely on their good job at
    switching the right request to the right server and send back the
    corresponding response. Of course there is a small risk that a client
    receives a response with its ID for a request from someone else, but
    when this happens, it means the intermediary is already broken and was
    doing that from the beginning. Also, the client will be able to detect
    that at least *some* responses do not match, and emit an error message
    indicating the user that there is something broken with the server on
    that site.

  - helps intermediaries participate : those which perform multiplexing
    will be able to easily translate request/response numbers from client-
    side and server-side connections, maintaining the correct matching for
    the client whatever rewrite has been performed ;

  - generalizing this will also for the first time bring the ability for a
    client to detect that a response is not the one it had expected. It
    still happens from time to time to get an unwanted response from broken
    multiplexers even without pipelining, and this request ID can fix that.

Since the counter is related to the connection, it should be announced in
the Connection header so that each intermediate can apply its own counter
on the other side. It will also make it easy for clients to detect that
their proxy is still not compatible and automatically disable pipelining
when the header is not present in responses.

So, to sum up :

  client to server :
     GET /foo.css HTTP/1.1
     Host: www
     Request-Id: 1
     Connection: Request-Id
  
     GET /foo.png HTTP/1.1
     Host: www
     Request-Id: 2
     Connection: Request-Id
  
     GET /foo.js HTTP/1.1
     Host: www
     Request-Id: 3
     Connection: Request-Id
  
  server to client :
     HTTP/1.1 200 OK
     Request-Id: 1
     Connection: Request-Id
  
     HTTP/1.1 200 OK
     Request-Id: 2
     Connection: Request-Id
  
     HTTP/1.1 200 OK
     Request-Id: 3
     Connection: Request-Id

And if connection multiplexing intermediaries are present, we can get this :

  client to proxy :
     GET http://example.com/foo.css HTTP/1.1
     Host: example.com
     Request-Id: 1
     Connection: Request-Id
  
     GET http://example.com/foo.png HTTP/1.1
     Host: example.com
     Request-Id: 2
     Connection: Request-Id
  
     GET http://example.com/foo.js HTTP/1.1
     Host: example.com
     Request-Id: 3
     Connection: Request-Id
  
  proxy to server:
     GET /foo.css HTTP/1.1
     Host: example.com
     Request-Id: 11
     Connection: Request-Id
  
     GET /foo-client2.png HTTP/1.1    <- from another client
     Host: example.com
     Request-Id: 12
     Connection: Request-Id
  
     GET /foo.png HTTP/1.1
     Host: example.com
     Request-Id: 13
     Connection: Request-Id
  
     GET /foo.js HTTP/1.1
     Host: example.com
     Request-Id: 14
     Connection: Request-Id
  
  server to proxy:
     HTTP/1.1 200 OK
     Request-Id: 11
     Connection: Request-Id
     Content-length: 10
     ...
  
     HTTP/1.1 200 OK
     Request-Id: 12
     Connection: Request-Id
     Content-length: 20
     ...
  
     HTTP/1.1 200 OK
     Request-Id: 13
     Connection: Request-Id
     Content-length: 30
     ...
  
     HTTP/1.1 200 OK
     Request-Id: 14
     Connection: Request-Id
     Content-length: 40
     ...
  
  proxy to client :
     HTTP/1.1 200 OK
     Request-Id: 1
     Connection: Request-Id
     Content-length: 10
  
     HTTP/1.1 200 OK
     Request-Id: 2
     Connection: Request-Id
     Content-length: 20
  
     HTTP/1.1 200 OK
     Request-Id: 3
     Connection: Request-Id
     Content-length: 40
  
Any opinion ?

Thanks,
Willy

---

From: "Thomson, Martin" <[email-masked]>
Date: Tue, 10 Aug 2010 14:00:34 +0800

Hi Willy,

> Any opinion ?

There's a big difference between what you are proposing and what I understand Mark is trying to achieve.

Anything that requires action from any entity is inherently not going to be necessary if all you are looking to do is make pipelining work.  After all, if the implementer has the sophistication to deploy these sorts of mechanisms, then they can equally be told to stop mucking with pipelining requests.  End of story; no new protocol mechanisms required.

In part, this draft looks at ways for a server and client to collude in detecting bad intermediaries.  There's not a lot that a document like this can do for bad servers aside from documenting some detection techniques - there certainly aren't any protocol mechanisms that you could deploy.

My first reaction was to use a request identifier, but then that messes with caches that don't understand the new header.  Going hop-by-hop, as you suggest runs afoul of the principle above.

Personally, I wonder whether the judicious application of Content-Location and Content-MD5 would not be enough.  It's close to what is suggested, but less definitive. 

If you want to talk about out of order responses, then we're in a whole 'nother ballpark.  That's where discussion of SCTP, SPDY, HTTP/2.0, et. al. come in.

--Martin

---

From: Adrien de Croy <[email-masked]>
Date: Tue, 10 Aug 2010 19:15:28 +1200

Just playing devil's advocate here... does pipelining really deserve all 
this (large) effort?

To me it seems multiple TCP connections is better for several reasons:

a) orders of magnitude more simple to implement
b) get true multiplexing without protocol changes to HTTP
c) better supported with existing infrastructure (scalability issues 
apart, which is another issue)
d) potentially better response wrt latency if you can request and start 
receiving an image or CSS before you've received all the HTML.

Most browsers seem to have already taken this path judging by the 
numbers of connections I see our proxy clients using.  They are doing 
this successfully now, whereas achieving a significant deployment of 
agents with these proposed changes is a long time away.

Pipelining is up against a fairly big chicken and egg problem, as well 
as a non-trivial implementation complexity problem (esp for proxies with 
plug-in filters trying to detect if an upstream server supports 
pipelining or not).

I also find it hard to favour a protocol change to cope with buggy 
servers and intermediaries (e.g. interleaved responses, dropped 
responses etc).  They should just be fixed.  Adding an effective request 
URI to every response is a significant traffic overhead (at least please 
make it MD5(URI))  URIs can be very very long (often several kB).

This also is just to find broken intermediaries, since a new server 
employing this would presumably fix its pipelining bugs first?  
Therefore why burden every response for this task?

A new administrative or content provider burden relating to maintaining 
information about likely benefits of pipelining seems to me well down on 
the list of things people want to worry about, and fraught with issues 
relating to authority and access to information. How can a server or 
human really know if once some content is deployed a pipelined request 
will truly provide advantage or not - it could be dependent on many 
unknowable factors, such as other server load etc.  Does a hosting site 
really want users fighting over who gets to put what meta tag in to try 
and get better responsiveness for their users?

There are also some legitimate cases where content back needs to be 
generated by an intermediary, or diverted / requests re-written.  E.g. 
reverse proxies, payment gateways (e.g. hotels), corporate use policy 
challenge pages etc.  The server generating the response may never have 
seen the actual request made by the client.

I just think it's already been put in the too-hard basket by many 
implementors, and  they are just working around the perceived 
performance issues, so the opportunity for pipelining to provide real 
benefits is diminishing, compounded by cost of development.

Cheers

Adrien

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 10 Aug 2010 09:52:04 +0200

Hi Martin,

> Anything that requires action from any entity is inherently not going to be necessary if all you are looking to do is make pipelining work.  After all, if the implementer has the sophistication to…

I agree that in theory this should be enough. But if Mark posted this
proposal, it's precisely because we have several difficulties in the
path.

> In part, this draft looks at ways for a server and client to collude in detecting bad intermediaries.  There's not a lot that a document like this can do for bad servers aside from documenting some…

I agree with your point concerning bad servers. But my main point is
to detect bad intermediaries without too high a cost for servers.

> My first reaction was to use a request identifier, but then that messes with caches that don't understand the new header.  Going hop-by-hop, as you suggest runs afoul of the principle above.

In fact in my opinion it would be the right solution. With a response
header from the server, we'll have to wait for servers to adopt that
and given the cost for some servers, we'll not see that happen soon.

With just a per-connection request counter, we can have that progressively
deployed where pipelining matters the most. Right now, as Mark said,
browsers disable it by default. The request identifier would allow a
smooth deployment which would work like this :

  1) browsers maintain a default disabled for pipelining

  2) browsers can start emitting the request identifier, and accept to
     automatically switch pipelining ON for the origin server (in case
     of direct access), or for the proxy (when passing through a proxy).
     That means that in a first time, nothing will change, we'll keep the
     current default setting which is disabled.

  3) proxies that support pipelining will be able to quickly adopt the
     principle and start responding with a request ID in responses, thus
     immediately allowing their local users to pipeline requests. This
     will even be valuable add-on for proxy vendors so this should be
     quickly adopted.

  4) proxies that are modified to respond to clients should also start
     emitting request IDs in the requests they send to servers.

  5) at this point, browsers and proxies are able to cooperate and will
     speed up adoption, as was done with the messy proxy-connection header.

  6) servers will get many requests with an identifier, either from the
     wide number of clients or from some progressively upgraded proxies.
     Servers will then be incited into putting that ID in the response.
     Since it's cheap to do, it should not take much time to see this
     appear.

  7) bad intermediaries will sometimes be identified (same goal as with
     the Assoc-Req hedaer). Those will have to be fixed, either by
     disabling pipelining (config option) or by making it work (takes
     longer but given support around the component, there will be more
     incentive).

> Personally, I wonder whether the judicious application of Content-Location and Content-MD5 would not be enough.  It's close to what is suggested, but less definitive. 

I'm not sure it would be an efficient alternative, nor that controls
will be correclty performed on that. Also, the pipelining issue really
is a hop-by-hop issue and not an end-to-end. So in my opinion, by trying
to get it fixed end-to-end, we won't succeed because every actor in the
chain has no motivation for trying to improve the situation since all
other actors around are known to be bad.

That's the current situation BTW : browsers support it but disable it,
so intermediates have no reason to work hard on making it work for a
small minority of users, considering that they won't even know if servers
will correctly process it.

> If you want to talk about out of order responses, then we're in a whole 'nother ballpark.  That's where discussion of SCTP, SPDY, HTTP/2.0, et. al. come in.

Yes, you're right, this is something different.

Regards,
Willy

---

From: "Thomson, Martin" <[email-masked]>
Date: Wed, 11 Aug 2010 09:27:28 +0800

Hi Willy,

To me, hinting is the most important part.  You need to know that a) the server and intermediaries that you are traversing wont break, and b) that there are no other problems with specific resources, like potential head-of-line blocking.

As for the request ID, I think that Mark's succinct description works best:

"
For example, if client A makes a request to a proxy cache with:
  Req-ID: abc
and the proxy goes to the origin server and gets a response with:
  Req-ID: abc
  Cache-Control: max-age=300
...and then client B makes a request to the proxy cache with:
  Req-ID: def
and gets the same response back from the cache (remembering that it hasn't been modified to support this), it'll get an error.
"

If you go to the trouble of fixing the proxy to support pipelining (again, most do), then all you really need is some indication that it's safe to proceed, not a per-request header.

The big challenge is with bad intermediaries and your plan seems to require that intermediaries change.  The bad ones wont.

--Martin

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 11 Aug 2010 14:07:48 +1000

Exactly.

The properties that I'm looking for are ones that will make it safe from a browser vendor's perspective; i.e., they can deploy pipelining and not break existing sites (which they inevitably get blamed for). 

By making pipelining effectively opt-in through hinting, the decision is in the hands of the server administrator, not the browser. It's not an optimal situation, but it does give us the opportunity to realistically make the world better. Expecting intermediaries to suddenly get better is IME not realistic.

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 11 Aug 2010 14:16:20 +1000

On 10/08/2010, at 5:15 PM, Adrien de Croy wrote:

> Just playing devil's advocate here... does pipelining really deserve all this (large) effort?
> 
> c) better supported with existing infrastructure (scalability issues apart, which is another issue)
> d) potentially better response wrt latency if you can request and start receiving an image or CSS before you've received all the HTML.

Yes; using multiple TCP connections isn't realistic when you're downloading fifty images across the pacific ocean. They don't share connection state, they all have to go through slow start, etc. This is all well-covered in the background material on SPDY.

> Most browsers seem to have already taken this path judging by the numbers of connections I see our proxy clients using.  They are doing this successfully now, whereas achieving a significant deployment of agents with these proposed changes is a long time away.

Define "long time." One browser vendor implementing measures like these can change things pretty quickly, at least for their users.

> Pipelining is up against a fairly big chicken and egg problem, as well as a non-trivial implementation complexity problem (esp for proxies with plug-in filters trying to detect if an upstream server supports pipelining or not).

Agreed; I've never liked using heuristics for this sort of thing.

> I also find it hard to favour a protocol change to cope with buggy servers and intermediaries (e.g. interleaved responses, dropped responses etc).  They should just be fixed.  

"They should just be fixed" hasn't worked for the past ten years. I'd like to try something else.

> Adding an effective request URI to every response is a significant traffic overhead (at least please make it MD5(URI))  URIs can be very very long (often several kB).

They can, but usually aren't for things that you're interested in pipelining (e.g., images, JS libraries, stylesheets).

> This also is just to find broken intermediaries, since a new server employing this would presumably fix its pipelining bugs first?  Therefore why burden every response for this task?

Fair question. I agree that the ability to opt in has the potential for solving most server-side problems, if the server admin is willing to debug and check their server. 

Are you suggesting that user agents basically don't pipeline at all when they discover a proxy (whether intercepting or configured)?

> A new administrative or content provider burden relating to maintaining information about likely benefits of pipelining seems to me well down on the list of things people want to worry about, and fraught with issues relating to authority and access to information. How can a server or human really know if once some content is deployed a pipelined request will truly provide advantage or not - it could be dependent on many unknowable factors, such as other server load etc.  Does a hosting site really want users fighting over who gets to put what meta tag in to try and get better responsiveness for their users?

Some people will certainly not be interested in this, but I know many (including, I suspect, my employer) who would be quite avidly interested in this.

> There are also some legitimate cases where content back needs to be generated by an intermediary, or diverted / requests re-written.  E.g. reverse proxies, payment gateways (e.g. hotels), corporate use policy challenge pages etc.  The server generating the response may never have seen the actual request made by the client.

Not sure where you're going here.

> I just think it's already been put in the too-hard basket by many implementors, and  they are just working around the perceived performance issues, so the opportunity for pipelining to provide real benefits is diminishing, compounded by cost of development.

I disagree. Many developers are keenly interested in deploying new approaches (e.g., HTTP-over-SCTP, SPDY) to work around the same problem set; if we can see benefits in the common case (downloading page assets) by tweaking/hinting HTTP and not making such fundamental changes, I think it's worth investigation. 

Cheers,

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 11 Aug 2010 07:02:09 +0200

> and gets the same response back from the cache (remembering that it hasn't been modified to support this), it'll get an error.

That's precisely why I was suggesting that the header should be marked
in Connection.

> If you go to the trouble of fixing the proxy to support pipelining (again, most do), then all you really need is some indication that it's safe to proceed, not a per-request header.

I see your point, but given the multiple possible paths to reach a same
point and the complexity involved by pipelining, seeing it succeed once
does not mean it will succeed twice even for the same site.

Ideally we could make use of OPTIONS requests as a first check that the
local proxy seems to support it. But that's very limited.

> The big challenge is with bad intermediaries and your plan seems to require that intermediaries change.  The bad ones wont.

I agree the bad ones won't change but at least the client gets an
indication that they don't work, exactly as with the Assoc-Req header.
In fact in my opinion it becomes a way for an intermediary to explicitly
declare support for pipelining. The currently broken ones won't announce
it, that's fine, let's not pipeline with them.

Regards,
Willy

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 11 Aug 2010 07:08:41 +0200

> By making pipelining effectively opt-in through hinting, the decision is in the hands of the server administrator, not the browser. It's not an optimal situation, but it does give us the opportunit…

That's not what I was saying, Mark. Instead it goes exactly in the
direction you suggest above. By hinting the browser knows it can use
pipelining, so that new browsers can ship with pipelining enabled by
default. Then, pipelining will have to be enabled on the server sides
in order to be usable. And from the browser's perspective, "the server"
means the local proxy, as all intermediaries, as well as the origin
server.

I'm not expecting the intermediaries to suddenly get better, and even
I think that many of them won't improve, but the mechanism at least
makes it clear which ones have explicit support for pipelining and which
ones don't. Once again, my goal is that browsers can ship with pipelining
enabled by default.

Even if today there are a few intermediaries that already support pipelining
well, not making use of their ability until they're upgraded will not be
a big waste, considering the extremely low deployment rate of pipelining
today.

Regards,
Willy

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 11 Aug 2010 07:17:33 +0200

> Not sure where you're going here.

I think Adrien was talking about the same example I gave, where the server
which generates the header in the response gets a request that has been
transformed by multiple intermediaries, to the point that the generated
header won't have any relation with the client's request and that
intermediaries that have changed it won't be able to correctly rewrite it
with a single regex. It's basically the same problem as with the Location
header which is often wrong when emitted by a server behind several reverse
proxies which rewrite the Host or the URI. Most often, the responses end up
being forced into the server or forced by intermediaries because doing the
transformation back is not always simple. The Assoc-Req header here will
have to experience similarly bad transformations to try to match the request
URI.

Regards,
Willy

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 11 Aug 2010 15:19:15 +1000

Right. So generate Assoc-Req at the outermost reverse proxy; it's the device that's acting as the origin server in that interchange, after all.

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 11 Aug 2010 08:16:21 +0200

> Right. So generate Assoc-Req at the outermost reverse proxy; it's the device that's acting as the origin server in that interchange, after all.

Well, in this case, it would be better to only transport an MD5 of the
request. That would be a lot more scalable, a lot lighter for servers
and reverse-proxies and cache friendly :

Client:
    GET http://example.com/foo1.css HTTP/1.1
    Host: example.com
    Req-MD5: 939caaf0a676b57a89429d5759834d21        (MD5 of "GET /foo1.css")

Server:
    HTTP/1.1 200 OK
    Req-MD5: 939caaf0a676b57a89429d5759834d21        (copy of client's MD5)

The advantage is that caches can cache this header just the same way
as it would with Assoc-Req, but servers don't have to keep a copy of
the full URI (which is variable-length) but instead just have to save
128 bits.  Also, it works well through reverse-proxies that rewrite
requests because the client sends the MD5 of the request before it is
mangled by reverse proxies.

Also, servers don't have to emit the header if the clients don't emit
it either.

And I find it light in terms of bandwidth too.

What do you think ?

Regards,
Willy

---

From: "Thomson, Martin" <[email-masked]>
Date: Wed, 11 Aug 2010 14:56:51 +0800

> it, that's fine, let's not pipeline with them.

The only sort of indication that a client is likely to get for a bad proxy/server is the absence of any mechanism that is defined.

So, if all we are looking to achieve is to indicate that the proxy (read: next hop) expressly supports pipelining, all you need to do is add a header:

  GET http://example.com/foo HTTP/1.1
  Pipeline: yes          <-- read: please
  Connection: pipeline
  ...

  HTTP/1.1 200 OK
  Pipeline: yes          <-- read: OK
  Connection: pipeline

(Actually, you don't need the Pipeline header, just the connection token - the header contains no extra information.)

Subsequent requests to that host can omit the token - you already know that pipelining is safe.

That does the same as what you proposed with less mechanism.  It doesn't address the bad intermediary problem, but it helps a client identify if pipelining is safe to attempt.

--Martin

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 11 Aug 2010 09:07:35 +0200

> That does the same as what you proposed with less mechanism.  It doesn't address the bad intermediary problem, but it helps a client identify if pipelining is safe to attempt.

I completely agree with you, Martin. My point was to try to automatically
eliminate bad intermediaries (as what the original proposal does), but
your proposal is much cleaner though it also relies on updates in
intermediaries (as did mine).

After all, it's how the support for keep-alive was added. And the
advantage is that it remains a connection oriented check and as you
pointed, can be omitted in subsequent requests as the connection's
status won't change.

Regards,
Willy

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 11 Aug 2010 17:29:32 +1000

It's interesting, but it would require browsers to spew Req-MD5 headers into requests unconditionally... something that I can't imagine they're likely to want to do (especially at first, when adoption on the server side is low).

OTOH just doing the MD5 in the response is potentially interesting, although having the URI available in the response may be useful for other purposes (e.g., verifying that we're using the right URL for a base URI, etc.). As Martin pointed out, Content-Location could serve that purpose as well.

Cheers,

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 11 Aug 2010 10:53:59 +0200

> It's interesting, but it would require browsers to spew Req-MD5 headers into requests unconditionally... something that I can't imagine they're likely to want to do (especially at first, when adopt…

Well, it's cheap and scalable. Also, it's hard to imagine that browsers
will both want to have pipelining to work and do nothing for that. The
"connection: pipeline" method suggested by Martin looks perfect to me
in this regards, but it does not address the point you want to address
which is to try to detect bad intermediaries without having to upgrade
them.

> OTOH just doing the MD5 in the response is potentially interesting,

if the server does the MD5 itself, we're back to the problem I emitted
about reverse proxies having to do the job themselves if they rewrite
requests. This will remain broken for a very long time, until those
reverse-proxies know how to write the header. Also, the problem of which
one has to do it depending on the request path still exists. In practice,
a browser may receive 4-5 different values for a same response header
because multiple pipiline-capable reverse-proxies will have put theirs.

That's why I really think that having the server report the information
it got is nice. That way we can ensure that even in case of multiple
actors, the reported values are the correct ones.

That said, for the long run, my preference still goes for the
"Connection: pipeline" method, which would take longer to deploy but
which is more in line with the real needs :-)

> although having the URI available in the response may be useful for other purposes (e.g., verifying that we're using the right URL for a base URI, etc.). As Martin pointed out, Content-Location could serve that purpose as well.

yes, though using it to detect a perfect match is still quite hard due to
possible rewrites (eg: url prefixing, double-slash normalization, the '+'
that becomes '%20', along the way, etc...).

Regards,
Willy

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 11 Aug 2010 19:17:05 +1000

This has been discussed a few times. Any indication of "I support pipelining" -- especially by intermediaries -- is going to lose its value as soon as somebody gets it wrong. Also, from what I've seen you can't rely on intermediaries to strip headers listed in Connection (unfortunately).  

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 11 Aug 2010 19:17:09 +1000

On 11/08/2010, at 6:53 PM, Willy Tarreau wrote:

> will both want to have pipelining to work and do nothing for that.

That's a false choice; they don't mind doing something, but they do mind putting extra bytes on the wire for every HTTP request on the planet if it can be avoided.

> The

I think we're going to have to disagree, but I'd encourage you to look at this from the perspective of a browser vendor.

> actors, the reported values are the correct ones.

I'm not concerned about reverse proxy deployments because they're under the control of the server, and looking at implementations I'm familiar with (Squid, Traffic Server, various others), it's pretty trivial to add a configuration option to take charge of this header. 

> which is more in line with the real needs :-)

When I think "longer to deploy", I think about techniques like SPDY, because they'll be more rewarding, given the effort required. 

Cheers,

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 11 Aug 2010 11:34:13 +0200

> That's a false choice; they don't mind doing something, but they do mind putting extra bytes on the wire for every HTTP request on the planet if it can be avoided.

OK, but a "Req-MD5: <hash>" is not *that* long. And we can even think of a
lot smaller strings, we don't need a cryptographically signed header, we
just need a hint to detect that a response does not match the request.

> I think we're going to have to disagree, but I'd encourage you to look at this from the perspective of a browser vendor.

Well, I don't see why. If I were a browser vendor, I would like it a lot
to be able to simply emit that header once with the first request on the
connection and be able to rely on the response instead of having to perform
complex heuristics. If this is in order to avoid sending that with every
request, at least that will only be sent with the first request of each
connection on which we plan to send more than one request. This is the
cheapest thing we can do in terms of network traffic.

> I'm not concerned about reverse proxy deployments because they're under the control of the server, and looking at implementations I'm familiar with (Squid, Traffic Server, various others), it's pre…

But doing it conditionally in complex infrastructures is quite complicated,
and will often result in getting multiple responses depending what path the
request takes, or no response at all. If this is identified by server-side
admins as complex to configure and expensive in terms of bandwidth, it will
probably not be much adopted. Also, the servers where it's the most important
are static file servers, especially the ones that can handle tens of thousands
of requests per second and want to optimize every single bit of processing.
Having them memcpy() the request+uri into a response header will have a small
but noticeable performance impact. This is even more true for the cases where
only a 304 is returned. And if clients notice that they regularly get mixed
responses (eg: header which does not exactly match, or multiple headers with
one or more which matches), they will probably abandon it too.

Don't get me wrong, I'm not saying it's a bad solution, I think it tries to
address a specific issue of servers or intermediaries, at a high cost for
legitimate servers which already correctly handle the behaviour, and at a
higher cost for admins of complex infrastructures, with finally not much
better reliability from the client side.

Regards,
Willy

---

From: Patrick McManus <[email-masked]>
Date: Mon, 23 Aug 2010 16:31:33 -0400

Hi Mark, and ietf-wg folk,

First, thanks for undertaking this. As RTTs go up it is more relevant
every day.

Re: draft-nottingham-http-pipeline-00.txt ... the four anecdotal
pipelining problems you enumerate nicely sum up the reasons I have heard
and encountered over the years myself. In some ways I think it is a
mistake just to mix them all together as they have such different
characteristics.

To give them handy reference names:
 1] unprocessed pipeline
 2] broken response order
 3] corrupted responses
 4] hol blockers

I think it is helpful to break #1 down into two categories (and maybe
you meant this and my reading of the document would just benefit from
clarification):

  - 1a] server closes connection without processing (some portion of)
pipeline. This is a minor performance nit when the request could have
been served on a parallel connection instead of being pipelined. But the
loose constraints about when servers can close connections make it
difficult to determine whether this is a pipelining failure or just a
semi-random and expected event.

  - 1b] server silently discards pipelined requests but remains open for
requests on persistent connection. That's really ugly because it either
turns into a big timeout or #2 if another request is sent that the
server perceives as non-pipelined. (and of course what the ua perceives
as pipelined may not look that way to the server).
 
While not terribly important, I think if we're going to undertake a
document on pipelining deployment problems it is worth noting that some
deployments operate correctly but still don't pipeline end to end due to
intermediaries enforcing lock-step forwarding (perhaps unintentionally).

My first question is do we have any information on the relative
importance of each? I must say that I have not personally witnessed an
instance of #1b, #2 or #3 in several years (and I say that as anecdote,
not as an assertion that they aren't running wild in someone else's view
of the Internet - so the question is sincere).. while #1a and #4 have
been various-levels of disasters for me when I have tried to introduce
pipelining into an intermediary.

As previously noted, categories 1b, 2 and 3 are implementation bugs -
mechanisms to detect and work around the bugs are helpful but any kind
of feature negotiation that requires an upgrade might as just well be
replaced with bug fixed code, right? In that case I'm on board with the
folks that are skeptical that an unverifiable promise such as
"Connection: pipelining" really promises anything useful.. A verifiable
MD-5 or echoed ID token such as your draft suggests is a much more
useful thing to provide - so I think that is the right approach. But I'm
a bigger fan of md5.

I do share the expressed concern about making the correlation ID be the
same URL used by the UA. I appreciate the fact that it can just be
stored as part of cached headers, but too much rewriting, routing, et al
goes on in today's Web to expect that to be consistently matched (or
even translated to be the same easily). More importantly, I fear that we
would accidentally be introducing false-positives into the system as
each endpoint viewed the naming differently and through errors even when
pipelining was working. It's not clear to me assoc-req is less fragile
than anonymous pipelining which it is trying to help.

In contrast, the checksum scheme is also cache friendly, isn't dependent
on naming, seems to accomplish all of what assoc-req does(*), and has
the advantage of already being well defined. There is a minor
computational cost, to be sure, but I think its lack of deployment is
more related to the lack of perceived benefit than anything else. If
developers buy into a benefit for assoc-req then why wouldn't they be
convinced of the same logic supporting md-5? And md-5 doesn't need a new
standards document.

Also along those lines, md-5 plays well with proxies. Assoc-req makes my
head hurt - the draft says proxies should never generate assoc-req (and
I get why), but pipelining is certainly not necessarily an end-to-end
property in practice.. some hops might want to initiate it by
multiplexing different tcp sessions together (crazy in today's
conditions, but if they had reason to believe it was safe to do so it
makes a lot of sense for something like a server side content
accelerator to do).. and some hops commonly undo pipelines without
breaking interop simply by behaving in a store and forward fashion...
even if a hop didn't want to initiate a pipeline it might want to deploy
a "safe pipeline" spec in order to allow the safe enablement of
pipelining in legacy browsers as a service to its users. 

(*)assoc-req could of course also be used in an out-of-order scheme but
we seem to be in agreement that is out of scope for http/1.*

Conditions 1a and 4 suffer from an information gap that hinting could
help with. 

Especially #4 - a primary case is simply the filtering out of things
like comet long polls as well as large and slow documents. To a
reasonable extent content (and javascript) authors can figure some of
that stuff out and label it as you suggest, but I am troubled by the gap
between the protocol and whatever markup the attribute lives in. I don't
have a suggestion at my fingertips but it seems that if the hint lived
within the protocol somehow it would be much more versatile and helpful,
right?  The timeout of 1b could also be shortened significantly in the
presence of some kind of hinting.

I'm pessimistic that attribute level meta data can help much with
scenario #1a.. #1a seems more driven by the implementation and
configuration of the server itself rather than the application data that
it serves. The latter often doesn't know much about the former. The
server could hint at the protocol layer about its implementation, but
that doesn't portend to be very satisfying. I wish I was making a more
constructive statement here. In some ways this is the least significant
but most intractable problem.

HTH at least a little.

-Patrick

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 31 Aug 2010 15:53:26 +1000

Hi Patrick,

Thanks for the feedback. A few (delayed) responses inline:

> as pipelined may not look that way to the server).

Yes. If you'd like to co-author, BTW, I'm sure that can be arranged...

> pipelining into an intermediary.

Good question. While we could run some studies against top100 Web sites, etc., it would IMO be more useful if we could get some stats from implementations that do use pipelining; namely, Opera and Mozilla (when enabled). 

I wonder how feasible it would be to have them collect stats (e.g., # of reqs successfully pipelined, average pipeline length, pipeline errors broken down by type) and have the user opt into them being sent to a server.

> than anonymous pipelining which it is trying to help.

False positives are a concern, but at worst pipelining wouldn't work in those cases; i.e., the Web site still would, so it would be a "soft" failure.

What I'm trying to do is create some aids to pipelining deployment that might even be temporary (i.e., only in use for a few years); they'll help get us to a place where pipelining is more common. As such, false positives aren't as bad as the current situation, IMO.

> convinced of the same logic supporting md-5? And md-5 doesn't need a new

When you say 'the checksum scheme' do you mean Content-MD5 and similar approaches? 

If so, I don't think it provides everything that assoc-req does; it doesn't tie the response to its associated request.

If not, could you please explain? 

> pipelining in legacy browsers as a service to its users. 

Why does Assoc-Req make that hard?

> we seem to be in agreement that is out of scope for http/1.*

Absolutely. 

> presence of some kind of hinting.

Yes. One of the things I want to add to a future revision are ways for XmlHttpRequest authors to hint pipelining use. 

> but most intractable problem.

Agreed. The hinting is really for #4.

Cheers,

---

From: Patrick McManus <[email-masked]>
Date: Tue, 31 Aug 2010 10:33:58 -0400

On Tue, 2010-08-31 at 15:53 +1000, Mark Nottingham wrote:

> Good question. While we could run some studies against top100 Web
> sites, etc., it would IMO be more useful if we could get some stats
> from implementations that do use pipelining; namely, Opera and Mozilla
> (when enabled). 
> 
> I wonder how feasible it would be to have them collect stats (e.g., #
> of reqs successfully pipelined, average pipeline length, pipeline
> errors broken down by type) and have the user opt into them being sent
> to a server.
> 

its a little off topic, but mozilla has been having a discussion on
their platform list regarding stats collection and the value of
characterizing the web in general. fyi:

http://groups.google.com/group/mozilla.dev.platform/browse_thread/thread/a9745463f8dd01f9#

> False positives are a concern, but at worst pipelining wouldn't work
> in those cases; i.e., the Web site still would, so it would be a
> "soft" failure.

well, the transaction presumably has to be redone in a non pipelined
context so there is a real cost to the false positive. But yes, it
doesn't really break anything so the only important question is how
often would it happen? My gut says it would happen a whole lot more
often than actual pipelining errors happen - especially because server
operators wouldn't have good visibility into it even occurring and it
seems the conditions for the origin-server and the user-agent to have
different views of the URI are common enough.

> What I'm trying to do is create some aids to pipelining deployment that might even be temporary 

+1 !

(re md5)
> If so, I don't think it provides everything that assoc-req does; it doesn't tie the response to its associated request.

you're right. For a moment, I had my head wrapped around headers and
bodies being mismatched rather than requests and responses.

> Why does Assoc-Req make that hard?

A week's mulling has mellowed me on this point.. but I'm reacting to
assoc-req being only generated on the origin server, but pipelines
potentially being added and removed at various hops. Origin is optimal,
sure, but isn't it still valuable if a proxy adds the response header as
long as it doesn't already exist - it is matching the request and
response as close to the origin as something that supports it is capable
of doing...