Subject: Backwards compatibility

From: Mark Watson <[email-masked]>
Date: Fri, 30 Mar 2012 16:13:24 +0000

All,

I'd like to make a plea/request/suggestion that wherever possible new features be added incrementally to HTTP1.1, in a backwards compatible way, in preference to a "new protocol" approach. A "new protocol" is required only if it is not technically possible (or especially awkward) to add the feature in a backwards compatible way.

The object should be to enable incremental implementation and deployment on a feature by feature basis, rather than all-or-nothing. HTTP1.1 has been rather successful and there is an immense quantity of code and systems - including intermediaries of various sorts - that work well with HTTP1.1. It should be possible to add features to that code and those systems without forklifting substantial amounts of it. It is better if intermediaries that do not support the new features cause fallback to HTTP1.1 vs the alternative of just blocking the new protocol. In particular, it should not cost a round trip to fall back to HTTP1.1. It is often lamented that the Internet is now the "port-80 network", but at least it is that.

Many of the features contemplated as solutions to the problems of HTTP1.1 can be implemented this way: avoiding head-of-line blocking of responses just requires a request id that is dropped by intermediaries that don't support it and echoed on responses. Request and response header compression can be negotiated - again with a request flag that is just dropped by unsupporting intermediaries. Pipelined requests could be canceled with a new method. These things are responsible for most of the speed improvements of SPDY, I believe.

Interleaving within responses does require some kind of framing layer, but I'd like to learn why anything more complex than interleaving the existing chunked-transfer chunks is needed (this is also especially easy to undo).

Putting my question another way, what is the desired new feature that really *requires* that we break backwards compatibility with the extremely successful HTTP1.1 ?

…Mark

---

From: William Chan (陈智昌) <[email-masked]>
Date: Fri, 30 Mar 2012 18:29:55 +0200

On Fri, Mar 30, 2012 at 6:13 PM, Mark Watson <[email-masked]> wrote:

> All,
>
> I'd like to make a plea/request/suggestion that wherever possible new
> features be added incrementally to HTTP1.1, in a backwards compatible way,
> in preference to a "new protocol" approach. A "new protocol" is required
> only if it is not technically possible (or especially awkward) to add the
> feature in a backwards compatible way.
>
> The object should be to enable incremental implementation and deployment
> on a feature by feature basis, rather than all-or-nothing. HTTP1.1 has been
> rather successful and there is an immense quantity of code and systems -
> including intermediaries of various sorts - that work well with HTTP1.1. It
> should be possible to add features to that code and those systems without
> forklifting substantial amounts of it. It is better if intermediaries that
> do not support the new features cause fallback to HTTP1.1 vs the
> alternative of just blocking the new protocol. In particular, it should not
> cost a round trip to fall back to HTTP1.1. It is often lamented that the
> Internet is now the "port-80 network", but at least it is that.
>

Don't forget port 443. And I agree, it should not cost a round trip to
fallback to HTTP/1.1.

>
> Many of the features contemplated as solutions to the problems of HTTP1.1
> can be implemented this way: avoiding head-of-line blocking of responses
> just requires a request id that is dropped by intermediaries that don't
> support it and echoed on responses. Request and response header compression
> can be negotiated - again with a request flag that is just dropped by
> unsupporting intermediaries. Pipelined requests could be canceled with a
> new method. These things are responsible for most of the speed improvements
> of SPDY, I believe.
>

It's unclear to me how this would work. Are you suggesting waiting a HTTP
request/response pair to figure out if the id gets echoed, before trying to
multiplex requests? Or would you rely on HTTP pipelining as a fallback if
the ids don't get echoed? The former incurs a large latency cost. The
latter depends very much on how deployable you view pipelining on the
overall internet. I am skeptical it is sufficiently deployable and we on
Chromium are gathering numbers to answer this question (
http://crbug.com/110794). Also, pipelining is clearly inferior to
multiplexing.

> Interleaving within responses does require some kind of framing layer, but
> I'd like to learn why anything more complex than interleaving the existing
> chunked-transfer chunks is needed (this is also especially easy to undo).
>

Sorry, I'm not sure I understand what you mean by interleaving existing
chunked-transfer chunks. Are these being interleaved across different
responses (that requires framing, right?).

>
> Putting my question another way, what is the desired new feature that
> really *requires* that we break backwards compatibility with the extremely
> successful HTTP1.1 ?
>

Multiplexing, header compression, prioritization.

---

From: Mark Watson <[email-masked]>
Date: Fri, 30 Mar 2012 16:53:49 +0000

On Mar 30, 2012, at 9:29 AM, William Chan (陈智昌) wrote:

On Fri, Mar 30, 2012 at 6:13 PM, Mark Watson <[email-masked]<mailto:[email-masked]>> wrote:
All,

Don't forget port 443. And I agree, it should not cost a round trip to fallback to HTTP/1.1.

It's unclear to me how this would work. Are you suggesting waiting a HTTP request/response pair to figure out if the id gets echoed, before trying to multiplex requests? Or would you rely on HTTP pipelining as a fallback if the ids don't get echoed?

Send the requests (yes, pipelined). If they come back without ids, then they are coming back in the order they were sent. If they come back with ids, then that tells you which response is which.

The former incurs a large latency cost. The latter depends very much on how deployable you view pipelining on the overall internet.

It's certainly widely deployed in servers and non-transparent proxies. Non-supporting non-transparent proxies are easily detected. Yes, broken transparent proxies are a (small) problem, but you can also detect these.

I am skeptical it is sufficiently deployable and we on Chromium are gathering numbers to answer this question (http://crbug.com/110794).

Our internal figures suggest that more than 95% of users can successfully use pipelining. That's an average. On some ISPs the figure is much lower.

Also, pipelining is clearly inferior to multiplexing.

Yes, but perhaps in practice not by much. To render a page you need all the objects, so from a time-to-page-load perspective it makes no difference how you multiplex them, as long as the link remains fully utilized. To see some difference you need some notion of object importance and some metric for 'page loaded except for the unimportant bits'. You send the most important requests first. Even then it's not clear that multiplexing within objects will perform significantly better than object by object sending.

Sorry, I'm not sure I understand what you mean by interleaving existing chunked-transfer chunks. Are these being interleaved across different responses (that requires framing, right?).

Interleaving data from multiple responses requires some kind of framing, yes. Chunked transfer encoding is a kind of framing that is already supported by HTTP. Allowing chunks to be associated with different responses would be a simple change. Maybe it feels like a hack ? That was my question: why isn't a small enhancement to the existing framing sufficient ?

Multiplexing,

See my question above

header compression,

Easily negotiated: an indicator in the first request indicates that the client supports it. If that indicator survives to the server, the server can start compressing response headers right away. If the client receives a compressed response it can start compressing future requests on that connection. It's important that this indicator be one which is dropped by intermediaries that don't support compression.

prioritization.

I think you mean "re-priortization". I can send requests in priority order - what I can't do is change that order to response to user actions. How big a deal is this, vs closing the connection and re-issuing outstanding requests in the new order ?

…Mark

…Mark

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 30 Mar 2012 19:04:49 +0200

Hi William,

> Don't forget port 443. And I agree, it should not cost a round trip to

That's why I think we should make new measurements on Upgrade. Concerning
the 67% success that was reported two years ago for WebSocket, a very large
part of the failures was due to the server not getting the Upgrade header
and thus not upgrading the connection. For HTTP this simply means an implicit
fallback to 1.1 (which is in our proposal).

I also predict that with WebSocket being released now, vendors will be more
careful about better supporting, and particularly fix their bugs related to
the Upgrade+101 processing.

But at the moment I have no numbers.

> http://crbug.com/110794). Also, pipelining is clearly inferior to

In our proposal, we have request headers to indicate which header fields
are to be kept for next requests, and a header to push several requests
into the pipeline if the Upgrade is accepted. This way we can have pipelining
from the first RTT when the Upgrade succeeds, and otherwise we have normal and
transparent 1.1 fallback if the Upgrade is not accepted.

Best regards,
Willy

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Fri, 30 Mar 2012 22:41:56 +0000

------ Original Message ------
From: "Mark Watson" [email-masked]
>
>Send the requests (yes, pipelined). If they come back without ids, 
>then they are coming back in the order they were sent. If they come 
>back with ids, then that tells you which response is which.
  
there could be pathological cases where some come back with IDs and 
some without.
  
>
>>The former incurs a large latency cost. The latter depends very much 
>>on how deployable you view pipelining on the overall internet. 
>
>It's certainly widely deployed in servers and non-transparent proxies. 
>Non-supporting non-transparent proxies are easily detected. Yes, 
>broken transparent proxies are a (small) problem, but you can also 
>detect these.
>
>>I am skeptical it is sufficiently deployable and we on Chromium are 
>>gathering numbers to answer this question (http://crbug.com/110794). 
>
>Our internal figures suggest that more than 95% of users can 
>successfully use pipelining. That's an average. On some ISPs the 
>figure is much lower.
  
Do you keep stats of how many of those 95% are not going through a 
proxy of any (detectable) kind?  I'd imagine the proportion (of 
directly-connected users) to be quite high.
>>Interleaving data from multiple responses requires some kind of 
>>framing, yes. Chunked transfer encoding is a kind of framing that is 
>>already supported by HTTP. Allowing chunks to be associated with 
>>different responses would be a simple change. Maybe it feels like a 
>>hack ? That was my question: why isn't a small enhancement to the 
>>existing framing sufficient ?
I think there would be interop issues.
  
>> 
>> Putting my question another way, what is the desired new feature 
>> that really *requires* that we break backwards compatibility with 
>> the extremely successful HTTP1.1 ?
> 
> Multiplexing,
 
 See my question above
 
 >header compression, 
>
>Easily negotiated: an indicator in the first request indicates that 
>the client supports it. If that indicator survives to the server, the 
>server can start compressing response headers right away. If the 
>client receives a compressed response it can start compressing future 
>requests on that connection. It's important that this indicator be one 
>which is dropped by intermediaries that don't support compression.
>
>>prioritization.
>
>I think you mean "re-priortization". I can send requests in priority 
>order - what I can't do is change that order to response to user 
>actions. How big a deal is this, vs closing the connection and 
>re-issuing outstanding requests in the new order ?
I'd like to add
  
support for new additional semantics.  Such as aren't possible if 
there's a 1.1 hop in the chain, but otherwise possible.
  
An example is some sort of subscribed notification, where you can send 
a single request, and get any number of responses with entities, as and 
when the server feels is right to send.
  
Think Facebook new message notifications, or online shopping card 
transaction status.
  
Adrien

---

From: Mark Watson <[email-masked]>
Date: Fri, 30 Mar 2012 23:11:12 +0000

On Mar 30, 2012, at 3:41 PM, Adrien W. de Croy wrote:

------ Original Message ------
From: "Mark Watson" [email-masked]<mailto:[email-masked]>

there could be pathological cases where some come back with IDs and some without.

I don't see how that could be the case if every intermediary on the path has indicated that it supports the extension. But I was not presenting a detailed protocol design, just an illustration of the type of backwards compatible design approach I was advocating. Work would certainly be required to design it.

I am skeptical it is sufficiently deployable and we on Chromium are gathering numbers to answer this question (<http://crbug.com/110794>http://crbug.com/110794).

Do you keep stats of how many of those 95% are not going through a proxy of any (detectable) kind?  I'd imagine the proportion (of directly-connected users) to be quite high.

No, we don't have that information.

Can you elaborate ?

Multiplexing,

See my question above

header compression,

prioritization.

support for new additional semantics.  Such as aren't possible if there's a 1.1 hop in the chain, but otherwise possible.

An example is some sort of subscribed notification, where you can send a single request, and get any number of responses with entities, as and when the server feels is right to send.

Think Facebook new message notifications, or online shopping card transaction status.

That indeed would be a new protocol, if you can make the case for providing that functionality at the HTTP layer, compared to the application layer where it lives today.

Adrien

…Mark

…Mark

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Fri, 30 Mar 2012 23:18:02 +0000

------ Original Message ------
>I don't see how that could be the case if every intermediary on the 
>path has indicated that it supports the extension. But I was not 
>presenting a detailed protocol design, just an illustration of the 
>type of backwards compatible design approach I was advocating. Work 
>would certainly be required to design it.
I was just thinking of a LB that talks to 2.0 and 1.1 servers on the 
back end from an incoming 2.0 stream of requests.  If any back end 
server was only 1.1, then the mux couldn't send any IDs.... it might 
not know that until too late though.  A better option for it may be to 
assign its own ID though.
  
>
>> 
>>>
>>>>The former incurs a large latency cost. The latter depends very 
>>>>much on how deployable you view pipelining on the overall internet.
>>>
>>>It's certainly widely deployed in servers and non-transparent 
>>>proxies. Non-supporting non-transparent proxies are easily detected. 
>>>Yes, broken transparent proxies are a (small) problem, but you can 
>>>also detect these.
>>>
>>>>framing, yes. Chunked transfer encoding is a kind of framing that 
>>>>is already supported by HTTP. Allowing chunks to be associated with 
  
intercepting intermediaries that think they know what they are looking 
at.
  
These are legion, at least in NZ ISPs, and many corporates.

> >the client supports it. If that indicator survives to the server, 
> >the server can start compressing response headers right away. If the 
> >client receives a compressed response it can start compressing 
> >future requests on that connection. It's important that this 
> >indicator be one which is dropped by intermediaries that don't 
> >support compression.
> >
> >>prioritization.
 >>
>>An example is some sort of subscribed notification, where you can 
>>send a single request, and get any number of responses with entities, 
>>as and when the server feels is right to send.
>> 
>That indeed would be a new protocol, if you can make the case for 
>providing that functionality at the HTTP layer, compared to the 
>application layer where it lives today.
  
where it lives today is on top of HTTP, which creates problems for the 
underlying HTTP infrastructure.  Things like building effectively TCP 
over 2xHTTP connections over TCP.  It's at best very inefficient.  It 
works by and large though.  Do we still want to be doing it that way in 
20, or 100 years?  
  
Adrien

---

From: Mike Belshe <[email-masked]>
Date: Sat, 31 Mar 2012 01:46:15 +0200

On Fri, Mar 30, 2012 at 6:53 PM, Mark Watson <[email-masked]> wrote:

>  It's unclear to me how this would work. Are you suggesting waiting a
> HTTP request/response pair to figure out if the id gets echoed, before
> trying to multiplex requests? Or would you rely on HTTP pipelining as a
> fallback if the ids don't get echoed?
>
>
>  Send the requests (yes, pipelined). If they come back without ids, then
> they are coming back in the order they were sent. If they come back with
> ids, then that tells you which response is which.
>

You can't do this until you've got confirmation that the server is going to
give you a HTTP/1.1 response.  It could come back HTTP/1.0.

So do we first have to do a 1.1 request successfully (with 1.1 response)
before we can ever attempt to do a pipelined upgrade?

>
>  The former incurs a large latency cost. The latter depends very much on
> how deployable you view pipelining on the overall internet.
>
>
> Non-supporting non-transparent proxies are easily detected. Yes, broken
> transparent proxies are a (small) problem, but you can also detect these.
>
> successfully use pipelining. That's an average. On some ISPs the figure is
> much lower.
>

Do you a research result to cite here?  Sounds interesting.  5% failures is
pretty high.

>  Yes, but perhaps in practice not by much. To render a page you need all
> the objects, so from a time-to-page-load perspective it makes no difference
> how you multiplex them, as long as the link remains fully utilized. To see
> some difference you need some notion of object importance and some metric
> for 'page loaded except for the unimportant bits'. You send the most
> important requests first. Even then it's not clear that multiplexing within
> objects will perform significantly better than object by object sending.
>

Don't forget that pipelining does *not* apply to all resources.  Even when
pipelining works end-to-end, browsers need to take great care not to
accidentally pipeline a critical resource behind a slow one (like a hanging
GET).  This leads to browsers doing tricks like "only pipeline images
together" or other subsets of pipelining.

But when we consider pipelining a fallback for SPDY, this all falls apart.
 SPDY did not have these restrictions.  So now, SPDY would need to run in
some sort of degraded mode for what types of requests are pipelined just so
fallback to a HTTP/1.1 protocol that the server might not support (because
it could be HTTP/1.0) or which the user might not support because he's one
of the unlucky 5% (according to Mark's data) where pipelining just breaks
altogether.

All in all, we've now compounded 3 unique restrictions on the initial set
of requests in order to work around past bugs in order to support use of
the Upgrade header.

Realistically, you're going to get one request on the upgrade, and you'll
have to wait to open up the parallel requests.  This is a significant
restriction of the Upgrade process - it requires a round trip to kick into
the real protocol at full gear.

This is highly annoying, but for initial web page loads, probably won't be
a significant burden because the browser initially only has one URL.  For
page reloads, or validations, or subsequent pages on reconnect, it will be
a performance hit.

>
>
>
>> Interleaving within responses does require some kind of framing layer,
>> but I'd like to learn why anything more complex than interleaving the
>> existing chunked-transfer chunks is needed (this is also especially easy to
>> undo).
>>
>
>  Interleaving data from multiple responses requires some kind of framing,
> yes. Chunked transfer encoding is a kind of framing that is already
> supported by HTTP. Allowing chunks to be associated with different
> responses would be a simple change. Maybe it feels like a hack ? That was
> my question: why isn't a small enhancement to the existing framing
> sufficient ?
>
>
Even if you could hack it into a chunk, thats a real jumbled mess.  Why do
you want to do this?  It doesn't give you backward compatibility in any way
(existing browsers won't know what to do with these nonstandard chunks
anyway), its just a mess for the sake of a mess?

>  Easily negotiated: an indicator in the first request indicates that the
> client supports it. If that indicator survives to the server, the server
> can start compressing response headers right away. If the client receives a
> compressed response it can start compressing future requests on that
> connection. It's important that this indicator be one which is dropped by
> intermediaries that don't support compression.
>
>  prioritization.
>
>
> order - what I can't do is change that order to response to user actions.
> How big a deal is this, vs closing the connection and re-issuing
> outstanding requests in the new order ?
>

Its the difference between web pages rendering faster or slower.    Load up
100 image requests on your twitter page, and then fetch the images before
the JS.  The page loads slower unless you lower the priority of the images.
 But you still don't want to add serialization delays that HTTP has.

BTW - the effects of priorities has been measured, and you're welcome to
use the existing benchmarking harness to verify yourself that these things
are true in real code rather than just theory.  (see dev.chromium.org/spdy).
 I wish I had published the tests when I did this long ago - spent a lot of
time on it.

Mike

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sat, 31 Mar 2012 02:57:44 +0200

On Sat, Mar 31, 2012 at 1:46 AM, Mike Belshe <[email-masked]> wrote:

>>> Many of the features contemplated as solutions to the problems of
>>> HTTP1.1 can be implemented this way: avoiding head-of-line blocking of
>>> responses just requires a request id that is dropped by intermediaries that
>>> don't support it and echoed on responses. Request and response header
>>> compression can be negotiated - again with a request flag that is just
>>> dropped by unsupporting intermediaries. Pipelined requests could be
>>> canceled with a new method. These things are responsible for most of the
>>> speed improvements of SPDY, I believe.
>>>
>>
> You can't do this until you've got confirmation that the server is going
> to give you a HTTP/1.1 response.  It could come back HTTP/1.0.
>
Can you? Are you referring to
http://tools.ietf.org/html/draft-nottingham-http-pipeline-01#section-5?
What about mobile clients which change networks frequently? How often are
we going to issue these test probes to discover faulty transparent proxies?

> Do you a research result to cite here?  Sounds interesting.  5% failures
> is pretty high.
>

I'd love to hear more about how these numbers were derived.

>> objects will perform significantly better than object by object sending.
Mike's already covered many of the downsides of pipelining. I'd like to
reiterate that the head of line blocking problem is huge. Also, there is a
question of how deep a pipeline can successfully go, since there are
constraints on the amount of buffering intermediaries can do. Furthermore,
pipelining has ill-defined failure modes. If it turns out that a HTTP/1.1
server doesn't support pipelining, how do you tell (see
http://tools.ietf.org/html/draft-nottingham-http-pipeline-01#section-3)?
And if the underlying TCP connection dies (http://crbug.com/27400), how do
you discover this? SPDY has PING frames to address this.

>> already supported by HTTP. Allowing chunks to be associated with different
So we have to wait for the first response to come back before we can see if
we can use header compression? That's quite unfortunate, especially for the
mobile case where the client more aggressively closes connections to
conserve power, and also since the mobile case especially benefits from
header compression. I'm also not a fan of allowing this to be negotiated,
for reasons Mike already covered in his talk. Note that only 2/3 of
compressible content on a page is actually compressed (
https://developers.google.com/speed/articles/web-metrics) :(

> Its the difference between web pages rendering faster or slower.    Load
> up 100 image requests on your twitter page, and then fetch the images
> before the JS.  The page loads slower unless you lower the priority of the
> images.  But you still don't want to add serialization delays that HTTP has.
>

---

From: Mark Watson <[email-masked]>
Date: Sat, 31 Mar 2012 01:03:16 +0000

On Mar 30, 2012, at 4:46 PM, Mike Belshe wrote:

On Fri, Mar 30, 2012 at 6:53 PM, Mark Watson <[email-masked]<mailto:[email-masked]>> wrote:

You can't do this until you've got confirmation that the server is going to give you a HTTP/1.1 response.  It could come back HTTP/1.0.

So do we first have to do a 1.1 request successfully (with 1.1 response) before we can ever attempt to do a pipelined upgrade?

For each server, yes. Servers don't often get downgraded from 1.1 to 1.0, so you could cache that result for quite a while.

Do you a research result to cite here?  Sounds interesting.  5% failures is pretty high.

No, these are just internal figures right now. Yes, it does seem high, I've a feeling many of those are false negatives where we avoid pipelining unnecessarily.

Don't forget that pipelining does *not* apply to all resources.  Even when pipelining works end-to-end, browsers need to take great care not to accidentally pipeline a critical resource behind a slow one (like a hanging GET).  This leads to browsers doing tricks like "only pipeline images together" or other subsets of pipelining.

I was assuming you could avoid the head-of-line blocking with an extension that allows out-of-order responses.

But when we consider pipelining a fallback for SPDY, this all falls apart.  SPDY did not have these restrictions.  So now, SPDY would need to run in some sort of degraded mode for what types of requests are pipelined just so fallback to a HTTP/1.1 protocol that the server might not support (because it could be HTTP/1.0) or which the user might not support because he's one of the unlucky 5% (according to Mark's data) where pipelining just breaks altogether.

All in all, we've now compounded 3 unique restrictions on the initial set of requests in order to work around past bugs in order to support use of the Upgrade header.

Realistically, you're going to get one request on the upgrade, and you'll have to wait to open up the parallel requests.  This is a significant restriction of the Upgrade process - it requires a round trip to kick into the real protocol at full gear.

This is highly annoying, but for initial web page loads, probably won't be a significant burden because the browser initially only has one URL.  For page reloads, or validations, or subsequent pages on reconnect, it will be a performance hit.

Even if you could hack it into a chunk, thats a real jumbled mess.  Why do you want to do this?  It doesn't give you backward compatibility in any way (existing browsers won't know what to do with these nonstandard chunks anyway), its just a mess for the sake of a mess?

So, your answer to my question is fairly clear ;-)

It doesn't feel like such a 'mess' to me - we're talking about negotiating use of new protocol elements. They're only used if both ends support them so, yes, the only kind of backwards compatibility is that the use of framing is negotiated, rather than assumed from the start. My point was that you don't need a whole shim layer to do this, because HTTP already has framing. Perhaps it makes little difference, but it means you can develop and deploy functionality incrementally, rather than all-or-nothing.

Multiplexing,

See my question above

header compression,

prioritization.

Its the difference between web pages rendering faster or slower.    Load up 100 image requests on your twitter page, and then fetch the images before the JS.  The page loads slower unless you lower the priority of the images.  But you still don't want to add serialization delays that HTTP has.

BTW - the effects of priorities has been measured, and you're welcome to use the existing benchmarking harness to verify yourself that these things are true in real code rather than just theory.  (see dev.chromium.org/spdy<http://dev.chromium.org/spdy>).  I wish I had published the tests when I did this long ago - spent a lot of time on it.

Again, I don't think you need anything more than the basic possibility to return responses out-of-order to get most of the gains. Send the requests in priority order and have the server return them in priority order, unless a response is not available in which case other responses can push ahead. The absence of interleaving within responses just reduces the granularity. Request the JS first, then the 100 images. With interleaving, if the JS is available half way through sending image 3, we can start sending the JS right there. Without interleaving you have to wait until the end of image 3.

What you don't have is, as I said, "re-prioritization", where the client can change its mind about the priority order after sending the requests - you'd have to close the connection and send the requests again.

Not perfect, but I feel you could get a good chunk of the gains, with out-of-order responses and negotiated compression.

Set aside that the significant advantages of small incremental changes to a well-understood, widely deployed, very successful protocol vs invention and all-at-once deployment of a new one.

…Mark

Mike

…Mark

…Mark

---

From: Roberto Peon <[email-masked]>
Date: Sat, 31 Mar 2012 03:22:20 +0200

On Sat, Mar 31, 2012 at 3:03 AM, Mark Watson <[email-masked]> wrote:

>  For each server, yes. Servers don't often get downgraded from 1.1 to
> 1.0, so you could cache that result for quite a while.
>
>
>
>>
>  No, these are just internal figures right now. Yes, it does seem high,
> I've a feeling many of those are false negatives where we avoid pipelining
> unnecessarily.
>
>
>
>
>>
>  Don't forget that pipelining does *not* apply to all resources.  Even
> when pipelining works end-to-end, browsers need to take great care not to
>  I was assuming you could avoid the head-of-line blocking with an
> extension that allows out-of-order responses.
>
>
>  But when we consider pipelining a fallback for SPDY, this all falls
> apart.  SPDY did not have these restrictions.  So now, SPDY would need to
> run in some sort of degraded mode for what types of requests are pipelined
> just so fallback to a HTTP/1.1 protocol that the server might not support
> (because it could be HTTP/1.0) or which the user might not support because
> he's one of the unlucky 5% (according to Mark's data) where pipelining just
> breaks altogether.
>
>  All in all, we've now compounded 3 unique restrictions on the initial
> set of requests in order to work around past bugs in order to support use
> of the Upgrade header.
>
>  Realistically, you're going to get one request on the upgrade, and
> you'll have to wait to open up the parallel requests.  This is a
> significant restriction of the Upgrade process - it requires a round trip
> to kick into the real protocol at full gear.
>
>  This is highly annoying, but for initial web page loads, probably won't
> be a significant burden because the browser initially only has one URL.
>  For page reloads, or validations, or subsequent pages on reconnect, it
> will be a performance hit.
>
>
>
>
>>
>>
>>
>  Even if you could hack it into a chunk, thats a real jumbled mess.  Why
> do you want to do this?  It doesn't give you backward compatibility in any
> way (existing browsers won't know what to do with these nonstandard chunks
>  It doesn't feel like such a 'mess' to me - we're talking about
> negotiating use of new protocol elements. They're only used if both ends
> support them so, yes, the only kind of backwards compatibility is that the
> use of framing is negotiated, rather than assumed from the start. My point
> was that you don't need a whole shim layer to do this, because HTTP already
> has framing. Perhaps it makes little difference, but it means you can
> develop and deploy functionality incrementally, rather than all-or-nothing.
>
>
>>
>>>
>  Again, I don't think you need anything more than the basic possibility
> to return responses out-of-order to get most of the gains. Send the
> requests in priority order and have the server return them in priority
> order, unless a response is not available in which case other responses can
> push ahead. The absence of interleaving within responses just reduces the
> granularity. Request the JS first, then the 100 images. With interleaving,
> if the JS is available half way through sending image 3, we can start
> sending the JS right there. Without interleaving you have to wait until the
> end of image 3.
>
>  What you don't have is, as I said, "re-prioritization", where the client
> can change its mind about the priority order after sending the requests -
> you'd have to close the connection and send the requests again.
>
>  Not perfect, but I feel you could get a good chunk of the gains, with
> out-of-order responses and negotiated compression.
>
>  Set aside that the significant advantages of small incremental changes
> to a well-understood, widely deployed, very successful protocol vs
> invention and all-at-once deployment of a new one.
>

HTTP is widely deployed, but most certainly not well understood :/

In any case... a number of clients and servers have implemented SPDY, and
haven't had any problems with deployment, nor big problems with
implementation.

What I understand that you're proposing is to build off HTTP/1.1's current
framing, but bolting on multiplexing (via response IDs), interleaving, and
prioritization. Note that you'll have to also do interleaving on the
request path to avoid HOL blocking there, e.g. when someone decides to POST
a video. I don't believe that this is trivial to make backwards
compatible... in fact, I think that the assumption that "building off of
current HTTP framing semantics is easier" is just plain not true. Hell, one
of the reasons I started SPDY with Mike was my more-than-slightly-strong
distaste for HTTP's framing after having to write a production-quality
framer for an intermediary that sees a lot of traffic (from billions of
people). It is the details that get you-- those niggly little pesky
annoying things.

SPDY doesn't have that problem with framing. The frames are well defined
and you don't have to dip into the application layer at any time to figure
out how to frame the dang frames. I hope that HTTP/2 is similarly easy to
implement with few (if any) corner cases for the framing.

Framing is just the wrong place to spend complexity... it makes far more
sense to spend it on features that improve performance, latency, or
security, at least in my opinion.

If you think I'm wrong about the complexity, hack it up :)
-=R

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 31 Mar 2012 09:13:33 +0200

Hi Roberto,

> security, at least in my opinion.

+1

Willy

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 31 Mar 2012 09:24:39 +0200

> before we can ever attempt to do a pipelined upgrade?

When you get some time to check our draft, you'll see that by using
header compression we can actually achieve this in a different way :

  http://tools.ietf.org/id/draft-tarreau-httpbis-network-friendly-00.html#optim-rt

Using a low request overhead for HTTP/1.x-only implementations, we can
make an opportunistic pipelining attempt without pushing any data past
the first message. Requests ending on 1.1 gateways or servers will simply
not be pipelined, but when end-to-end is 2.0 then pipelining is OK. This
is a nice fast fallback in my opinion.

For having worked with a mobile operator and observed how RTTs are wasted
on each connection by not pipelining the first request, I would really not
want anything that still requires one RTT before pipelining, it simply
doubles the page load time.

Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 31 Mar 2012 07:30:12 +0000

>> security, at least in my opinion.

Framing done badly hurts performance.

What we're looking for is the highest-performance HTTP protocol
we can imagine.  HTTP scaled from 10Mbit/s to 10Gbit/s so far,
HTTP/2.0 will have to do at least 1Tbit/s in its lifetime.

If we imagine the perfectly optimal behaviour from the network
stack, and perfectly optimal HTTP message from the other end, the
perfect protocol scenario looks like this:

        [length of head]
        [head]
        [length of body]
        [body]

Under utterly perfect circumstanses, just three socket reads will
get you the head and body into memory chosen, sized, aligned &
allocated perfectly for the purpose:

        READ(length header) ->len buffer
        (allocate workspace)
        READV(head + next length header) -> (workspace, len buffer)
        (allocate bodyspace)
        READ(body) -> bodyspace

Any protocol which by design requires more work to move the bits from
the TCP connection, through the socket API and into the applications
memory, is not a high-performance protocol, worthy of HTTP/2.0
consideration.

Notice that just doing:
        READ(1GB)
might get you the same data into memory, but you can not optimally
place it in memory without memcpy'ing it around.

Notice that at this point we have not talked about compression,
TLS, integrity or anything else, we are only talking about how we
pull a byte stream out of the TCP socket API, into memory of our
desire.

Likewise, adding other overhead to the [length] header, proto-id,
compression indicators, sequence numbers, session ids and so on,
may or may not make sense, but does not affect the analysis.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 31 Mar 2012 10:14:06 +0200

> HTTP/2.0 will have to do at least 1Tbit/s in its lifetime.

That's very true. And we can only assume that RTTs will become an
issue even for the local network and for NIC-to-memory and CPU-to-
memory communications, so streaming must be possible most of the
time to fill each pipe.

> perfect protocol scenario looks like this:

I'm still having a problem with this scheme, it is most of the requests
don't have a body, and having a fixed-size length for each request is
an overhead. We might however have a fixed-size length for a known number
of subsquent heads and compressed size for each message. Or we could also
decide that a head cannot be more than 64kB, and encode the length on 2
bytes. I don't know.

>         READV(head + next length header) -> (workspace, len buffer)

No, under perfect situations, a single readv() would give you all the
parts you need with fixed sizes, because each syscall is a big waste
of time. BTW, I'm sure that when running varnish at full throttle, the
CPU time is largely dominated by the system, as I already observe in
haproxy where the common figure is around 85% system and 15% user. That
said, we cannot obviously have fixed sizes for everything.

> memory, is not a high-performance protocol, worthy of HTTP/2.0

I agree that we must avoid memory copies as much as possible, but a read()
is a system-assisted memory copy. Depending on implementations, performing
one large read + one small copy will be preferable, and other implementations
will prefer several small reads. The protocol must cleanly address these
various needs.

> place it in memory without memcpy'ing it around.

Hopefully we'll not see a 1GB header soon !

> pull a byte stream out of the TCP socket API, into memory of our

Which is the point in favor of making it easy to consume just the data
we need.

Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 31 Mar 2012 08:28:52 +0000

>don't have a body,

If there is no body, you can (should be able to!) see that out of
the head, and you obviously will not try to read the body which is
not there.

>parts you need with fixed sizes,

There is no way that can work, because you dont know how many headers
the message has, and therefore you cannot know how many bytes to read
beforehand, unless the other end tells you.

If you read too many, you get part of the body also, and have to move
that to its proper memory space with a memcpy().

The important words in my description above are "three", "chosen",
"sized", "aligned" and "allocated", and they are just about equally
important once we get above 1Gbit/sec.

>is a system-assisted memory copy.

A system-assisted *unavoidable* memory copy, so we should make the most
of it, rather than have to move stuff again.

>Hopefully we'll not see a 1GB header soon !

My example above reads header & body.  4.7GB objects surprisingly
common.

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 31 Mar 2012 12:05:55 +0200

Hi Mark,

> I was assuming you could avoid the head-of-line blocking with an extension that allows out-of-order responses.

That's been tried in the past; see:
  http://tools.ietf.org/id/draft-mogul-http-ooo-00.txt

Regards,

--
Mark Nottingham
http://www.mnot.net/

---

From: Amos Jeffries <[email-masked]>
Date: Sat, 31 Mar 2012 23:45:10 +1300

On 31/03/2012 11:41 a.m., Adrien W. de Croy wrote:
> From: "Mark Watson" [email-masked] <mailto:[email-masked]>
>>

Such as hops going over any of the existing infrastructure. That 
infrastructure will be performing HTTP/1.x requirement of *ignore* 
unknown headers.
It would seem your feature requires the ID being a new Expect: option.

AYJ

---

From: Mark Watson <[email-masked]>
Date: Sat, 31 Mar 2012 16:31:17 +0000

Interesting - I knew this was not a new idea, but did not know about this draft.

Any idea why it did not catch on back then ? I could imagine at that time pipelining was not widely enough supported to make this interesting.

...Mark

Sent from my iPhone

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 31 Mar 2012 19:03:25 +0200

> Any idea why it did not catch on back then ? I could imagine at that time pipelining was not widely enough supported to make this interesting.

I didn't know about the draft either and find the idea of having a barrier
on requests without ID very good. Simple, efficient and avoids the risk of
reordering untagged traffic.

Willy