Subject: HTTP 2.0 and a Faster, more Mobile-friendly web

From: Henrik Frystyk Nielsen <[email-masked]>
Date: Sun, 29 Jul 2012 20:14:02 +0000

Dear All,

We remain committed to the HTTP/2.0 standards process and look forward to seeing many of you this week at the IETF meeting in Vancouver to continue the discussion.  In the spirit of open discussion, we wanted to share some observations in advance of the meeting and share the latest progress from prototyping and testing.

There are currently three different proposals that the group is working through:

   * SPDY (http://tools.ietf.org/html/draft-mbelshe-httpbis-spdy),

   * HTTP Speed+Mobility (http://tools.ietf.org/html/draft-montenegro-httpbis-speed-mobility),

   * Network-Friendly HTTP Upgrade (http://tools.ietf.org/html/draft-tarreau-httpbis-network-friendly).

The good news is that everyone involved wants to make the Web faster, more scalable, more secure, and more mobile-friendly, and each proposal has benefits in different areas that the discussion can choose from.

--- A Genuinely Faster Web ---

The SPDY proposal has been great for raising awareness of Web performance. It takes a "clean slate" approach to improving HTTP.

To compare the performance of SPDY with HTTP/1.1 we have run tests comparing download times of several public web sites using a controlled tested study. The test uses publically available software run with mostly default configurations while applying all the currently available optimizations to HTTP/1.1. You can find a preliminary report on the test results here: http://research.microsoft.com/apps/pubs/?id=170059. The results mirror other data (http://www.guypo.com/technical/not-as-spdy-as-you-thought) that indicate mixed results with SPDY performance.

Our results indicate almost equal performance between SPDY and HTTP/1.1 when one applies all the known optimizations to HTTP/1.1. SPDY's performance improvements are not consistent and significant. We will continue our testing, and we welcome others to publish their results so that HTTP/2.0 can choose the best changes and deliver the best possible performance and scalability improvements compared to HTTP/1.1.

--- Taking the Best from Each ---

Speed is one of several areas of improvement. Currently, there's no clear consensus that any one of the proposals is the clear choice or even starting point for HTTP/2.0 (based on our reading the Expressions of Interest and discussions on this mailing list. A good example of this is the vigorous discussion around mandating TLS encryption (http://tools.ietf.org/html/rfc5246) for HTTP/2.0.

We think a good approach for HTTP/2.0 is to take the best solution for each of these areas from each of the proposals.  This approach helps us focus the discussion for each area of the protocol. Of course, this approach would still allow the standard to benefit from the extensive knowledge gained from implementing existing proposals.

We believe that the group can converge on consensus in the following areas, based on our reading of the Expressions of Interest, by starting from the different proposals.

------------------|------------------

Area              | Opinion that

                  | seems to prevail

------------------|------------------

1. Compression    | SPDY or Friendly

------------------|------------------

2. Multiplexing   | SPDY

------------------|------------------

3. Mandatory TLS  | Speed+Mobility

------------------|------------------

4. Negotiation    | Friendly or

                  |   Speed+Mobility

------------------|------------------

5. Client Pull/   | Speed+Mobility

      Server Push |

------------------|------------------

6. Flow Control   | SPDY

------------------|------------------

7. WebSockets     | Speed+Mobility

------------------|------------------

Below, we discuss each HTTP/2.0 element and the current consensus that appears to be forming within the Working Group.

1. Compression

Compression is simple to conceptualize and implement, and it is important. Proxies and other boxes in the middle on today's Web often face problems with it. The HTTP/2.0 discussion has been rich but with little consensus.

Though some studies suggest that SPDY's header compression approach shows promise, other studies show this compression to be prohibitively onerous for intermediary devices. More information here would help us make sure we're making the Web faster and better.

Also, an entire segment of implementers are not interested in compression as defined in SPDY.  That's a challenge because the latest strawman for the working group charter (http://lists.w3.org/Archives/Public/ietf-http-wg/2012JulSep/0784.html) states that the "resulting specification(s) are expected to be meet these goals for common existing deployments of HTTP; in particular, ... intermediation (by proxies, corporate firewalls, 'reverse' proxies and Content Delivery Networks)."

We think the SPDY or Friendly proposals is a good starting point for progress.

2. Multiplexing

All three proposals define similar multiplexing models. We haven't had substantial discussion on the differences. This lack of discussion suggests that there is rough consensus around the SPDY framing for multiplexing.

We think that the SPDY proposal is a good starting point here and best captures the current consensus.

3. Mandating Always On TLS

There is definitely no consensus to mandate TLS for all Web communication, but some major implementers have stated they will not to adopt HTTP/2.0 unless the working group supports a "TLS is mandatory" position. A very preliminary note from the chair (http://lists.w3.org/Archives/Public/ietf-http-wg/2012JulSep/0601.html) states that there is a lack of consensus for mandating TLS.

We think the Speed+Mobility proposal is a good starting point here as it provides options to turn TLS on (or not).

4. Negotiation

Only two of the proposals actually discuss how different endpoints agree to use HTTP/2.0.

(The SPDY proposal does not specify a negotiation method. Current prototype implementations use the TLS-NPN (http://tools.ietf.org/html/draft-agl-tls-nextprotoneg) extension.  While the other proposals use HTTP Upgrade to negotiate HTTP/2.0, some parties have expressed non-support for this method as well.)

We think either of the Friendly or Speed+Mobility proposals is a good starting point because they are the only ones that have any language in this respect.

5. Client Pull and Server Push

There are tradeoffs between a server push model and a client pull model. The main question is how to improve performance while respecting bandwidth and client caches.

Server Push has not had the same level of implementation and experimentation as the other features in SPDY. More information here would help us make sure we're making the Web faster and better.

We think the Speed+Mobility proposal is a good starting point here, suggesting that this issue may be better served in a separate document rather than tied to the core HTTP/2.0 protocol.

6. Flow Control

There has only been limited discussion in the HTTPbis working group on flow control. Flow Control offers a lot of opportunity make the Web faster as well as to break it; for example, implementations need to figure out how to optimize for opposing goals (like throughput and responsiveness) at the same time.

The current version of the SPDY proposal specifies a flow control message with many settings are that are not well-defined. The Speed+Mobilty proposal has a simplified flow control model based on certain assumptions. More experimentation and information here would help us make sure we're making the Web faster and better.

We think that the SPDY proposal is a good starting point here.

7. WebSockets

We see support  for aligning HTTP/2.0 with a future version of WebSockets, as suggested in the introduction of the Speed+Mobility proposal.

--- Moving forward ---

We're excited for the Web to get faster, more stable, and more capable, and HTTP/2.0 is an important part of that.

We believe that bringing together the best elements of the current SPDY, HTTP Speed+Mobility, and Network-Friendly HTTP Upgrade proposals is the best approach to make that happen.

Based on the discussions on the HTTPbis mailing list, we've suggested which proposals make the most sense to start from for each of the areas that HTTP/2.0 is addressing. Each of these areas needs more prototyping and experimentation and data. We're looking forward to the discussion this week.

Sincerely,

Henrik Frystyk Nielsen

Principal Architect, Microsoft Open Technologies, Inc.

Gabriel Montenegro

Principal Software Development Engineer, Microsoft Corporation

Rob Trace

Senior Program Manager Lead, Microsoft Corporation

Adalberto Foresti

Senior Program Manager, Microsoft Open Technologies, Inc.

---

From: "Martin Nilsson" <[email-masked]>
Date: Mon, 30 Jul 2012 00:16:39 +0200

On Sun, 29 Jul 2012 22:14:02 +0200, Henrik Frystyk Nielsen  
<[email-masked]> wrote:

>
> To compare the performance of SPDY with HTTP/1.1 we have run tests  
> comparing download times of several public web sites using a controlled  
> tested study. The test uses publically available software run with  
> mostly default configurations while applying all the currently available  
> optimizations to HTTP/1.1. You can find a preliminary report on the test  
> results here: http://research.microsoft.com/apps/pubs/?id=170059.

This is an interesting demonstration of a use case HTTP was made for, and  
where HTTP works well. The issues we are facing are non-working pipelining  
in HTTP/1, increase in HTTP header size from things like cookies and  
ua-prof-diff, and streaming of video. It doesn't appear that you are  
testing these issues, so it makes sense that the performance difference  
isn't that great. It would be preferable if HTTP/2 would always win over  
HTTP/1 though.

Since your minify-tests performs non-reversible content modifications, I  
would be very careful to use those results.

/Martin Nilsson

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

---

From: Roberto Peon <[email-masked]>
Date: Sun, 29 Jul 2012 15:15:49 -0700

Interesting data! Questions about test methodology:

Was the client/server speaking SPDY/2 or SPDY/3?
A percentage of the time, Chrome will not use SPDY (it decides this at
startup). Did you verify that it was always speaking SPDY? (I'd guess yes!)
Was the server using the CWND set sent in the SETTINGS frame, and was it
sending it to the client?
What was the server's initcwnd?
Was slow-start-after-idle turned off on the kernel?
How long were the various connections in slow start, and did they ever exit
slow-start?
Was the SSL cert chain the same size and using the same algorithms?
Was the dummynet (or equivalent) configured to have appropriate buffer size
(else it does packet drops silently)?
Were these all cold-connection, cold-cache first-page load timings? The use
of keep-alive in figure 3 would seem to indicate otherwise, but I'm not
sure which are which...
Did you put Chromium into 'replay' mode (where date and random always
return in the same manner, regardless of actual date)?
Any particular reason that you turned off entity-body compression in
Apache? Were the resources saved verbatim from WinHTTrack (in which case
it'd make sense)?
Was the server emulating the original server's think times? This can have a
very, very large impact on pipelined HTTP performance.

We've found that the various automatic page-is-done signals are often less
useful than something which measures the above-the-fold page load time.
This generally involves cameras pointed at screens and a human looking at
the video to determine at which frame the page is useful, and designating
that frame as 'done-for-above-the-fold'

I'd love to see if we could get a common suite of test data over which we
could all run experiments/benchmarks.
Is anyone else interested in doing so?

-=R

On Sun, Jul 29, 2012 at 1:14 PM, Henrik Frystyk Nielsen <
[email-masked]> wrote:

>  Dear All,****
>
> ** **
>
> We remain committed to the HTTP/2.0 standards process and look forward to
> seeing many of you this week at the IETF meeting in Vancouver to continue
> the discussion.  In the spirit of open discussion, we wanted to share some
> observations in advance of the meeting and share the latest progress from
> prototyping and testing. ****
>
> ** **
>
> There are currently three different proposals that the group is working
> through:****
>
> ** **
>
>    * SPDY (http://tools.ietf.org/html/draft-mbelshe-httpbis-spdy),****
>
>    * HTTP Speed+Mobility (
> http://tools.ietf.org/html/draft-montenegro-httpbis-speed-mobility),****
>
>    * Network-Friendly HTTP Upgrade (
> http://tools.ietf.org/html/draft-tarreau-httpbis-network-friendly). ****
>
> ** **
>
> The good news is that everyone involved wants to make the Web faster, more
> scalable, more secure, and more mobile-friendly, and each proposal has
> benefits in different areas that the discussion can choose from.****
>
> ** **
>
> --- A Genuinely Faster Web ---****
>
> ** **
>
> The SPDY proposal has been great for raising awareness of Web performance.
> It takes a “clean slate” approach to improving HTTP.****
>
> ** **
>
> tested study. The test uses publically available software run with mostly
> default configurations while applying all the currently available
> results here: http://research.microsoft.com/apps/pubs/?id=170059. The
> results mirror other data (
> http://www.guypo.com/technical/not-as-spdy-as-you-thought) that indicate
> mixed results with SPDY performance.****
>
> ** **
>
> Our results indicate almost equal performance between SPDY and HTTP/1.1
> when one applies all the known optimizations to HTTP/1.1. SPDY’s
> performance improvements are not consistent and significant. We will
> continue our testing, and we welcome others to publish their results so
> that HTTP/2.0 can choose the best changes and deliver the best possible
> performance and scalability improvements compared to HTTP/1.1.****
>
> ** **
>
> ** **
>
> --- Taking the Best from Each ---****
>
> ** **
>
> Speed is one of several areas of improvement. Currently, there’s no clear
> consensus that any one of the proposals is the clear choice or even
> starting point for HTTP/2.0 (based on our reading the Expressions of
> Interest and discussions on this mailing list. A good example of this is
> the vigorous discussion around mandating TLS encryption (
> http://tools.ietf.org/html/rfc5246) for HTTP/2.0. ****
>
> ** **
>
> We think a good approach for HTTP/2.0 is to take the best solution for
> each of these areas from each of the proposals.  This approach helps us
> focus the discussion for each area of the protocol. Of course, this
> approach would still allow the standard to benefit from the extensive
> knowledge gained from implementing existing proposals.  ****
>
> ** **
>
> We believe that the group can converge on consensus in the following
> areas, based on our reading of the Expressions of Interest, by starting
> from the different proposals.****
>
> ** **
>
> ------------------|------------------****
>
> Area              | Opinion that ****
>
>                   | seems to prevail****
>
> ------------------|------------------****
>
> 1. Compression    | SPDY or Friendly****
>
> ------------------|------------------****
>
> 2. Multiplexing   | SPDY****
>
> ------------------|------------------****
>
> 3. Mandatory TLS  | Speed+Mobility****
>
> ------------------|------------------****
>
> 4. Negotiation    | Friendly or****
>
>                   |   Speed+Mobility****
>
> ------------------|------------------****
>
> 5. Client Pull/   | Speed+Mobility****
>
>       Server Push | ****
>
> ------------------|------------------****
>
> 6. Flow Control   | SPDY****
>
> ------------------|------------------****
>
> 7. WebSockets     | Speed+Mobility****
>
> ------------------|------------------****
>
> ** **
>
> Below, we discuss each HTTP/2.0 element and the current consensus that
> appears to be forming within the Working Group.****
>
> ** **
>
> 1. Compression****
>
> ** **
>
> Compression is simple to conceptualize and implement, and it is important.
> Proxies and other boxes in the middle on today’s Web often face problems
> with it. The HTTP/2.0 discussion has been rich but with little consensus.*
> ***
>
> ****
>
> Though some studies suggest that SPDY’s header compression approach shows
> promise, other studies show this compression to be prohibitively onerous
> for intermediary devices. More information here would help us make sure
> we’re making the Web faster and better. ****
>
> ** **
>
> Also, an entire segment of implementers are not interested in compression
> as defined in SPDY.  That’s a challenge because the latest strawman for the
> working group charter (
> http://lists.w3.org/Archives/Public/ietf-http-wg/2012JulSep/0784.html)
> states that the “resulting specification(s) are expected to be meet these
> goals for common existing deployments of HTTP; in particular, …
> intermediation (by proxies, corporate firewalls, ‘reverse’ proxies and
> Content Delivery Networks).” ****
>
> ** **
>
> We think the SPDY or Friendly proposals is a good starting point for
> progress.****
>
> ** **
>
> 2. Multiplexing****
>
> ** **
>
> All three proposals define similar multiplexing models. We haven’t had
> substantial discussion on the differences. This lack of discussion suggests
> that there is rough consensus around the SPDY framing for multiplexing. **
> **
>
> ** **
>
> We think that the SPDY proposal is a good starting point here and best
> captures the current consensus.****
>
> ** **
>
> 3. Mandating Always On TLS ****
>
> ** **
>
> There is definitely no consensus to mandate TLS for all Web communication,
> but some major implementers have stated they will not to adopt HTTP/2.0
> unless the working group supports a “TLS is mandatory” position. A very
> preliminary note from the chair (
> http://lists.w3.org/Archives/Public/ietf-http-wg/2012JulSep/0601.html)
> states that there is a lack of consensus for mandating TLS. ****
>
> ** **
>
> We think the Speed+Mobility proposal is a good starting point here as it
> provides options to turn TLS on (or not).****
>
> ** **
>
> 4. Negotiation****
>
> ** **
>
> Only two of the proposals actually discuss how different endpoints agree
> to use HTTP/2.0. ****
>
> ** **
>
> (The SPDY proposal does not specify a negotiation method. Current
> prototype implementations use the TLS-NPN (
> http://tools.ietf.org/html/draft-agl-tls-nextprotoneg) extension.  While
> the other proposals use HTTP Upgrade to negotiate HTTP/2.0, some parties
> have expressed non-support for this method as well.) ****
>
> ** **
>
> We think either of the Friendly or Speed+Mobility proposals is a good
> starting point because they are the only ones that have any language in
> this respect.  ****
>
> ** **
>
> 5. Client Pull and Server Push****
>
> ** **
>
> There are tradeoffs between a server push model and a client pull model.
> The main question is how to improve performance while respecting bandwidth
> and client caches. ****
>
> ** **
>
> Server Push has not had the same level of implementation and
> experimentation as the other features in SPDY. More information here would
> help us make sure we’re making the Web faster and better. ****
>
> ** **
>
> We think the Speed+Mobility proposal is a good starting point here,
> suggesting that this issue may be better served in a separate document
> rather than tied to the core HTTP/2.0 protocol. ****
>
> ** **
>
> 6. Flow Control****
>
> ** **
>
> There has only been limited discussion in the HTTPbis working group on
> flow control. Flow Control offers a lot of opportunity make the Web faster
> as well as to break it; for example, implementations need to figure out how
> to optimize for opposing goals (like throughput and responsiveness) at the
> same time. ****
>
> ** **
>
> The current version of the SPDY proposal specifies a flow control message
> with many settings are that are not well-defined. The Speed+Mobilty
> proposal has a simplified flow control model based on certain assumptions.
> More experimentation and information here would help us make sure we’re
> making the Web faster and better. ****
>
> ** **
>
> We think that the SPDY proposal is a good starting point here. ****
>
> ** **
>
> 7. WebSockets****
>
> ** **
>
> We see support  for aligning HTTP/2.0 with a future version of WebSockets,
> as suggested in the introduction of the Speed+Mobility proposal.****
>
> ** **
>
> ** **
>
> --- Moving forward ---****
>
> ** **
>
> We’re excited for the Web to get faster, more stable, and more capable,
> and HTTP/2.0 is an important part of that. ****
>
> ** **
>
> We believe that bringing together the best elements of the current SPDY,
> HTTP Speed+Mobility, and Network-Friendly HTTP Upgrade proposals is the
> best approach to make that happen. ****
>
> ** **
>
> Based on the discussions on the HTTPbis mailing list, we’ve suggested
> which proposals make the most sense to start from for each of the areas
> that HTTP/2.0 is addressing. Each of these areas needs more prototyping and
> experimentation and data. We’re looking forward to the discussion this week.
> ****
>
> ** **
>
> Sincerely,****
>
> ** **
>
> Henrik Frystyk Nielsen****
>
> Principal Architect, Microsoft Open Technologies, Inc.****
>
> ** **
>
> Gabriel Montenegro****
>
> Principal Software Development Engineer, Microsoft Corporation****
>
> ** **
>
> Rob Trace****
>
> Senior Program Manager Lead, Microsoft Corporation****
>
> ** **
>
> Adalberto Foresti****
>
> Senior Program Manager, Microsoft Open Technologies, Inc.****
>
> ** **
>
> ** **
>

---

From: Amos Jeffries <[email-masked]>
Date: Mon, 30 Jul 2012 10:18:26 +1200

On 30.07.2012 08:14, Henrik Frystyk Nielsen wrote:
> Dear All,
>
>
>
> We remain committed to the HTTP/2.0 standards process and look
> forward to seeing many of you this week at the IETF meeting in
> Vancouver to continue the discussion.  In the spirit of open
> discussion, we wanted to share some observations in advance of the
> meeting and share the latest progress from prototyping and testing.
>
>
>
>    * HTTP Speed+Mobility
> (http://tools.ietf.org/html/draft-montenegro-httpbis-speed-mobility),
>
>    * Network-Friendly HTTP Upgrade
> (http://tools.ietf.org/html/draft-tarreau-httpbis-network-friendly).
>
>
>
> The good news is that everyone involved wants to make the Web faster,
> more scalable, more secure, and more mobile-friendly, and each
> proposal has benefits in different areas that the discussion can
> choose from.
>
>
>
> The SPDY proposal has been great for raising awareness of Web
> performance. It takes a "clean slate" approach to improving HTTP.
>
>
>
> comparing download times of several public web sites using a
> controlled tested study. The test uses publically available software
> run with mostly default configurations while applying all the
> currently available optimizations to HTTP/1.1. You can find a
> preliminary report on the test results here:
> http://research.microsoft.com/apps/pubs/?id=170059. The results 
> mirror
> other data 
> (http://www.guypo.com/technical/not-as-spdy-as-you-thought)
> that indicate mixed results with SPDY performance.
>
>
>
> Our results indicate almost equal performance between SPDY and
> HTTP/1.1 when one applies all the known optimizations to HTTP/1.1.
> SPDY's performance improvements are not consistent and significant. 
> We
> will continue our testing, and we welcome others to publish their
> results so that HTTP/2.0 can choose the best changes and deliver the
> best possible performance and scalability improvements compared to
> HTTP/1.1.
>
>
>
>
>
> Speed is one of several areas of improvement. Currently, there's no
> clear consensus that any one of the proposals is the clear choice or
> even starting point for HTTP/2.0 (based on our reading the 
> Expressions
> of Interest and discussions on this mailing list. A good example of
> this is the vigorous discussion around mandating TLS encryption
> (http://tools.ietf.org/html/rfc5246) for HTTP/2.0.
>
>
>
> We think a good approach for HTTP/2.0 is to take the best solution
> for each of these areas from each of the proposals.  This approach
> helps us focus the discussion for each area of the protocol. Of
> course, this approach would still allow the standard to benefit from
> the extensive knowledge gained from implementing existing proposals.
>
>
>
> areas, based on our reading of the Expressions of Interest, by
> starting from the different proposals.
>
>
>
> Below, we discuss each HTTP/2.0 element and the current consensus
> that appears to be forming within the Working Group.
>

I question why SPDY is the only content from multiplexing? 
network-friendly was designed based on the SPDY multiplexing model with 
improvements for intermediary flow control in mind.

In light of the WG discussions since publishing network-friendly-00 we 
have alterations to the frames which should place it at as a stronger 
contender for:
  2 - better flow identification / control options.
  5 - client pull initiated with optional multiple responses (aka 
optional/negotiated server push),
  7 - WebSockets as extension frames. (also SPDY as extension frames).

>
> 1. Compression
>
>
>
> Compression is simple to conceptualize and implement, and it is
> important. Proxies and other boxes in the middle on today's Web often
> face problems with it. The HTTP/2.0 discussion has been rich but with
> little consensus.
>
> Though some studies suggest that SPDY's header compression approach
> shows promise, other studies show this compression to be 
> prohibitively
> onerous for intermediary devices. More information here would help us
> make sure we're making the Web faster and better.
>
>
>
> Also, an entire segment of implementers are not interested in
> compression as defined in SPDY.  That's a challenge because the 
> latest
> strawman for the working group charter
> 
> (http://lists.w3.org/Archives/Public/ietf-http-wg/2012JulSep/0784.html)
> states that the "resulting specification(s) are expected to be meet
> these goals for common existing deployments of HTTP; in particular,
> ... intermediation (by proxies, corporate firewalls, 'reverse' 
> proxies
> and Content Delivery Networks)."
>
>
>
> progress.
> All three proposals define similar multiplexing models. We haven't
> had substantial discussion on the differences. This lack of 
> discussion
> suggests that there is rough consensus around the SPDY framing for
> multiplexing.
>
>
> There is definitely no consensus to mandate TLS for all Web
> communication, but some major implementers have stated they will not
> to adopt HTTP/2.0 unless the working group supports a "TLS is
> mandatory" position. A very preliminary note from the chair
> 
> (http://lists.w3.org/Archives/Public/ietf-http-wg/2012JulSep/0601.html)
> states that there is a lack of consensus for mandating TLS.
>
>
>
> We think the Speed+Mobility proposal is a good starting point here as
> it provides options to turn TLS on (or not).
>

As does network-friendly, which has the ability to negotiate any 
feature in either client request frames or mid-stream interleaved 
control frames.

> prototype implementations use the TLS-NPN
> (http://tools.ietf.org/html/draft-agl-tls-nextprotoneg) extension.
> While the other proposals use HTTP Upgrade to negotiate HTTP/2.0, 
> some
> parties have expressed non-support for this method as well.)
>
>
>
> starting point because they are the only ones that have any language
> in this respect.
>
>
>
> There are tradeoffs between a server push model and a client pull
> model. The main question is how to improve performance while
> respecting bandwidth and client caches.
>
>
>
> experimentation as the other features in SPDY. More information here
> would help us make sure we're making the Web faster and better.
>
>
>
> rather than tied to the core HTTP/2.0 protocol.
>
>

Indeed. The point being that the framing model needs to be able to be 
used by a reasonable server-push even if the details of push flow 
control are omitted from the initial HTTP/2 spec.

>
> 6. Flow Control
>
>
>
> There has only been limited discussion in the HTTPbis working group
> on flow control. Flow Control offers a lot of opportunity make the 
> Web
> faster as well as to break it; for example, implementations need to
> figure out how to optimize for opposing goals (like throughput and
> responsiveness) at the same time.
>
>
>
> The current version of the SPDY proposal specifies a flow control
> message with many settings are that are not well-defined. The
> Speed+Mobilty proposal has a simplified flow control model based on
> certain assumptions. More experimentation and information here would
> help us make sure we're making the Web faster and better.
>
>
>
> We see support  for aligning HTTP/2.0 with a future version of
> WebSockets, as suggested in the introduction of the Speed+Mobility
> proposal.
>
>
>
> --- Moving forward ---
>
>
>
> We're excited for the Web to get faster, more stable, and more
> capable, and HTTP/2.0 is an important part of that.
>
>
>
> We believe that bringing together the best elements of the current
> SPDY, HTTP Speed+Mobility, and Network-Friendly HTTP Upgrade 
> proposals
> is the best approach to make that happen.
>
>

+1.

AYJ

---

From: Figroc Chen <[email-masked]>
Date: Mon, 30 Jul 2012 14:22:40 +0800

Hi,

What's the usecase of HTTP? And what kind of target does the HTTP/2.0
want to achieve?

The main performance issue of HTTP/1.0 is that one connection for one
request. Since, pipelining has been introduced in HTTP/1.1, without
introducing new header. People argue that pipelining only works
partially because of the interference of network intermediate. Why
don't we just change the default TCP port, or the URL schema? As all
those three protocols break current HTTP-compliant deployment.
Everything has to upgrade to stay in operation.

Considering the networking characteristic of mobile devices, the TCP
connection is meant to be unstable and fragile. Framing control
introduces lot of state in the yet another transport layer. In fact we
have the SIP protocol to refer. As we consider TCP only, the job can
be done by just two headers: Call-ID and CSeq, or squeeze them into
just one header: Transaction-ID. That's much more simple, and much
cleaner.

Speaking about the server notification, I think long-pulling is a very
good way to go. As we are defining new protocol here, we can make the
long-pull operation really very long, maybe as long as the connection
stays alive. If we do that, there is no performance difference between
long-pull and direct push. But for the compatibility sake, long-pull
is far more superior.

For every framing mechanism, the chunked transfer encoding is the
nightmare. We can leverage the 1xx provisional response to eliminate
it, and still retain the streaming ability. This way, network
intermediate will function more effectively.

We don't need another layer of "TCP", do we?

Regards,
Figroc

On Mon, Jul 30, 2012 at 4:14 AM, Henrik Frystyk Nielsen
<[email-masked]> wrote:
> prototyping and testing.
> through:
> benefits in different areas that the discussion can choose from.
>
>
>
> It takes a “clean slate” approach to improving HTTP.
>
>
>
> To compare the performance of SPDY with HTTP/1.1 we have run tests comparing
> download times of several public web sites using a controlled tested study.
> The test uses publically available software run with mostly default
> configurations while applying all the currently available optimizations to
> HTTP/1.1. You can find a preliminary report on the test results here:
> http://research.microsoft.com/apps/pubs/?id=170059. The results mirror other
> data (http://www.guypo.com/technical/not-as-spdy-as-you-thought) that
> indicate mixed results with SPDY performance.
>
>
>
> Our results indicate almost equal performance between SPDY and HTTP/1.1 when
> one applies all the known optimizations to HTTP/1.1. SPDY’s performance
> improvements are not consistent and significant. We will continue our
> testing, and we welcome others to publish their results so that HTTP/2.0 can
> choose the best changes and deliver the best possible performance and
> scalability improvements compared to HTTP/1.1.
>
>
>
>
>
> consensus that any one of the proposals is the clear choice or even starting
> point for HTTP/2.0 (based on our reading the Expressions of Interest and
> discussions on this mailing list. A good example of this is the vigorous
> discussion around mandating TLS encryption
> We think a good approach for HTTP/2.0 is to take the best solution for each
> of these areas from each of the proposals.  This approach helps us focus the
> discussion for each area of the protocol. Of course, this approach would
> still allow the standard to benefit from the extensive knowledge gained from
> implementing existing proposals.
>
>
>
> We believe that the group can converge on consensus in the following areas,
> based on our reading of the Expressions of Interest, by starting from the
> different proposals.
>
>
>
> appears to be forming within the Working Group.
>
>
>
> 1. Compression
>
>
>
> with it. The HTTP/2.0 discussion has been rich but with little consensus.
>
> promise, other studies show this compression to be prohibitively onerous for
> intermediary devices. More information here would help us make sure we’re
> making the Web faster and better.
>
>
>
> Also, an entire segment of implementers are not interested in compression as
> defined in SPDY.  That’s a challenge because the latest strawman for the
> working group charter
> Content Delivery Networks).”
>
>
>
> that there is rough consensus around the SPDY framing for multiplexing.
>
>
>
> captures the current consensus.
>
>
>
> preliminary note from the chair
> provides options to turn TLS on (or not).
>
>
>
> 4. Negotiation
>
>
>
> (The SPDY proposal does not specify a negotiation method. Current prototype
> implementations use the TLS-NPN
> (http://tools.ietf.org/html/draft-agl-tls-nextprotoneg) extension.  While
> have expressed non-support for this method as well.)
>
>
>
> starting point because they are the only ones that have any language in this
> respect.
>
>
>
> There are tradeoffs between a server push model and a client pull model. The
> main question is how to improve performance while respecting bandwidth and
> client caches.
>
>
>
> Server Push has not had the same level of implementation and experimentation
> as the other features in SPDY. More information here would help us make sure
> we’re making the Web faster and better.
>
>
>
> There has only been limited discussion in the HTTPbis working group on flow
> control. Flow Control offers a lot of opportunity make the Web faster as
> well as to break it; for example, implementations need to figure out how to
> optimize for opposing goals (like throughput and responsiveness) at the same
> time.
>
>
>
> with many settings are that are not well-defined. The Speed+Mobilty proposal
> has a simplified flow control model based on certain assumptions. More
> experimentation and information here would help us make sure we’re making
> the Web faster and better.
>
>
>
> as suggested in the introduction of the Speed+Mobility proposal.
>
>
>
>
>
> --- Moving forward ---
>
>
>
> We’re excited for the Web to get faster, more stable, and more capable, and
> HTTP/2.0 is an important part of that.
>
>
>
> HTTP Speed+Mobility, and Network-Friendly HTTP Upgrade proposals is the best
> approach to make that happen.
>
>
>
> Based on the discussions on the HTTPbis mailing list, we’ve suggested which
> proposals make the most sense to start from for each of the areas that
> HTTP/2.0 is addressing. Each of these areas needs more prototyping and

---

From: Henrik Frystyk Nielsen <[email-masked]>
Date: Mon, 30 Jul 2012 06:32:07 +0000

Martin,

What scenarios do you have in mind where pipelining does not work? Obviously it only works for GET so for web sites using lots of non-GET requests it won't work but GET is the predominant method on most sites.

We took some pretty normal sites so I do think they represent reasonable data. We'd be happy to expand the sites but the whole point was to show differences in relatively common sites. It is correct that we didn't use cookies. Cookie and UA-header sizes can indeed vary a lot but that is a much more isolated problem to solve that can be done in any number of ways.

As for video (assuming you use TCP) the bottleneck is likely going to be set by TCP throughput. The overhead of HTTP headers will be negligible due to the size of the payload. We'd be happy to do some tests but all things being equal I can say based on experience that adding a credit-based session layer such as SPDY will not perform as well as running straight over TCP for large payloads. The reason is that the credit scheme by its very purpose acts as a throttle so that one session doesn't take the entire bandwidth.

As for minification, the intention is not to do those unbeknownst to the website designer -- it is something that is an integral part of the design of a web site so there is no reason why this isn't a valid optimization.

Thanks,

Henrik

-----Original Message-----
From: Martin Nilsson [mailto:[email-masked]] 
Sent: Sunday, July 29, 2012 3:17 PM
To: [email-masked]
Subject: Re: HTTP 2.0 and a Faster, more Mobile-friendly web

/Martin Nilsson

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 30 Jul 2012 07:02:45 +0000

>2. Multiplexing
>
>All three proposals define similar multiplexing models. We haven't had

... or that more important topics have stolen the time.

(Absense of evidence is not evidence of absence.)

>unless the working group supports a "TLS is mandatory" position.

It is not clear to me exactly what these major implementers mean when
they say "TLS is mandatory"

Do they mean "TLS MUST be supported" or "TLS MUST be used" ?

It is also not clear to me which property of TLS they are after, is
it identification, authentication, secrecy or privacy they want ?

I also wonder about their motives:  Do they really insist on TLS
in order to shut out intermediaries ?

>and client caches.

I would Server Push for HTTP/2.1, but keep an eye out for what it will
need while designing HTTP/2.0.

> control. 

One very important aspect of flow control is DoS mitigation, and we have
barely mentioned that until now.

It would be a great step forward in DoS resistance if HTTP/2.0 put
a very tight limit on the initial "resource window" the client can
use, in order to give the server maximum control over allocation
of resources, from the very begining.

If the limit was set around "max 1 connection, max 6 pipelines requests
no request bodies larger than 4k" until the server says otherwise,
DoS attacks against HTTP/2.0 will be easier to cope with.

I too agree that all three proposals bring good things to the table,
but I think we would be better off starting from scratch, cherry picking
from the three suggestions as we go along, rather than try to evict
text already added to one of the proposals.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: William Chan (陈智昌) <[email-masked]>
Date: Mon, 30 Jul 2012 02:25:26 -0700

Thanks for sending this out Henrik! I appreciate the work you guys have
done to gather data here. There's a lot to say about this (and it looks
like Roberto already commented a lot on the test results themselves), and I
ought to sleep now, but I wanted to point out some initial things first.

On Sun, Jul 29, 2012 at 1:14 PM, Henrik Frystyk Nielsen <
[email-masked]> wrote:

> results mirror other
Is the PDF broken? I don't see a link to the dummy webpage in the report
PDF.

> indicate mixed results with SPDY performance.****
>
> **
>
I find it very curious you mention these two results in conjunction when
discussing SPDY performance. The former relies heavily on pipelining,
which, if you believe Guypo's test results (which I more or less do,
although I don't necessarily agree with the title for his spdy post), show
that pipelining has mixed results. Please refer to
http://www.guypo.com/technical/http-pipelining-not-so-fast-nor-slow/.
Anyhow, I don't feel like we need to belabor pipelining too much, since it
seems like everyone's in support of multiplexing anyway, which I think is
great.

> performance and scalability improvements compared to HTTP/1.1.
>
I'm trying to understand the comments here. Can you clarify some
ambiguities?
- When you say SPDY's performance improvements are not consistent and
significant, are you saying compared to the HTTP+pipelining+minify results
or just compared to naive HTTP? I find the fact that Microsoft's synthetic
test results help validate what we've already known, that reducing
roundtrips is important. Lots of the pipelining and minify techniques are
designed to help reduce roundtrips. SPDY does this too via features like
multiplexing.
- Are you asserting that, because there are ways to workaround HTTP level
bottlenecks via application level optimizations, we shouldn't address HTTP
issues? While I do wish that more websites applied web performance
techniques and leveraged tools like mod_pagespeed, I think it's a sad
reality that most websites do not. It'd be great if we could resolve many
HTTP level performance bottlenecks so that all users of HTTP/2.0 get those
gains, not just the ones who are willing to put in extra time to obtain
them.

> from the different proposals.****
Great!

> The main question is how to improve performance while respecting bandwidth
I want to re-emphasize that server push, as defined by SPDY, provides for
better inlining. Even Microsoft's tests results seem to leverage inlining
to reduce roundtrips. People complain about server push for not respecting
bandwidth and client caches, but current inlining techniques already do
that.

> help us make sure we’re making the Web faster and better.
>
Totally agreed, and we're working on this. I hope in the future we will be
able to contribute much more data here.

> Senior Program Manager, Microsoft Open Technologies, Inc.****
Btw, since the subject of this email talks about a mobile friendly web, can
you speak to what features in HTTP/2.0 will help make the web more
mobile-friendly? I've got my own ideas, relating to SPDY, but I'm curious
what Microsoft thinks here, especially since HTTP S+M references mobile in
its name.

---

From: patrick mcmanus <[email-masked]>
Date: Mon, 30 Jul 2012 08:09:57 -0700

> Do they mean "TLS MUST be supported" or "TLS MUST be used" ?

I mean that HTTP/2 must be secure against (at least) passive 
eavesdropping attacks at all times. TLS is the bird-in-hand for that 
right now, but it does not exclude other solutions. Other properties of 
TLS are desirable too, but they don't necessarily rise to the level of 
mandatory to implement for me.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 30 Jul 2012 15:21:27 +0000

In message <[email-masked]>, patrick mcmanus writes:

>eavesdropping attacks at all times. 

Pardon me for being a bit blunt:   You and what army ?

I can understand if you insist that _your_ website can always
be protected if you desire that.

But I utterly fail to see what mandate you have to insist that some
random 3rd party must protect their website at all times, countrary
to their own wishes, and in particular when they may be legally
prevented from doing so by applicable regulations.

---

From: patrick mcmanus <[email-masked]>
Date: Mon, 30 Jul 2012 08:37:05 -0700

> What scenarios do you have in mind where pipelining does not work? Obviously it only works for GET so for web sites using lots of non-GET requests it won't work but GET is the predominant method on…

I'm a believer that we can get more benefit from pipelines than we 
currently do, and I remain committed to seeing that happen on today's 
http/1 web. But the challenges go beyond non-get and mangling 
intermediaries(*).. so in a pro-pipeline, but eyes-wide-open spirit:

1] you still need parallelism because of priority head of line blocking 
issues. But now some of the parallel flows are filling the pipe and 
buffers very well, and its hard to manage the prioritization 
effectively. Using a priority mux instead of pipeline+parallelism makes 
that work a bit better (buffering in the network remains a problem - 
though with fewer connections they should at least not be filled as deeply)

2] even on get - blocking responses are common and they screw up the 
pipeline. database operations, web service callouts, etc all can create 
variability in response times, and that creates time-to-utilization gaps

3] http/1 cancellation semantics don't exist other than closing the 
connection, and pipelines make this worse. This comes up all the time in 
the form of links clicked on partially rendered pages. (think "next" on 
a webmail inbox). To process the click you need to decide between 
waiting for the old one to drain, adding more to its existing pipeline, 
or making a new connection.. this problem always existed without 
pipelines but the amount of data in the queue has now gone up.

[*] in my experience the most significant intermediary challenges these 
days come from AV and firewall software.. actual network devices and 
servers fair ok with a couple notable exceptions. I don't know that this 
makes it any easier :)

pipelines are useful things - but I think we can do better than just 
working around their warts.

> We took some pretty normal sites so I do think they represent reasonable data. We'd be happy to expand the sites but the whole point was to show differences in relatively common sites. It is correc…

cookies and pipeline performance are pretty tightly tied. Pipeline depth 
is driven by client cwnd and the difference in pipeline depth on a cold 
TCP session goes from 10 to 2 as cookie sizes go from 0 to 2KB. An 
HTTP/2 approach that removes header redundancy (whether that be by some 
manual table approach like network-friendly, or a automagic gzip 
approach like spdy) is critical to addressing the fundamental round trip 
problem.

cheers,
-Patrick

---

From: patrick mcmanus <[email-masked]>
Date: Mon, 30 Jul 2012 08:43:50 -0700

On 7/30/2012 8:21 AM, Poul-Henning Kamp wrote:
> In message<[email-masked]>, patrick mcmanus writes:
>

please don't be so difficult. You asked what I (and others) meant by 
what you called mandatory TLS and I provided clarification for my pov.

The merits have been discussed here before (and will be again) but I 
hoped my clarification, that the notion-of-mandatory-tls isn't quite 
accurate, was helpful.

---

From: Yoav Nir <[email-masked]>
Date: Mon, 30 Jul 2012 18:46:32 +0300

> Pardon me for being a bit blunt:   You and what army ?

Him and all the proponents of SPDY (I'm not one of them)

> be protected if you desire that.

If not all HTTP has TLS, then an active attacker can lead viewers to a lookalike website, so _his_ website (or rather, his users) is not protected against active attackers.

> prevented from doing so by applicable regulations.

+1

Additionally, TLS requires the client to check revocation of the server certificate. Some browsers don't, but that's besides the point. Checking revocation involves fetching either a CRL or an OCSP response, and they are typically fetched over HTTP. If HTTP has to have TLS we have a bootstrap problem, unless checking revocation is relegated back down to HTTP/1.0.

If we're only interested in preventing passive attacks, then we don't really need to check revocation, but then what of websites that do care about active attacks?  They do need to fetch revocation data, so are you going to have them fetch it over insecure TLS?

---

From: patrick mcmanus <[email-masked]>
Date: Mon, 30 Jul 2012 08:52:51 -0700

> Additionally, TLS requires the client to check revocation of the server certificate. Some browsers don't, but that's besides the point. Checking revocation involves fetching either a CRL or an OCSP…
that's not a roadblock.. we can address this largely via ocsp stapling.. 
also ocsp with a ca can be done over tls without cert verification 
because the ocsp response is signed separately.

---

From: "Martin Nilsson" <[email-masked]>
Date: Mon, 30 Jul 2012 18:36:19 +0200

Henrik,

Current HTTP pipelining works fine when it works, but there are a lot of  
servers that doesn't support it. As soon as you start having more than one  
outstanding request on a connection you'll run into issues ranging from  
ignoring subsequent requests (forcing a reissue of the queued requests),  
to closing the connection as soon as a second request is received  
(truncating the first response), to scrambling the response (i.e. multiple  
responses written to the same socket at the same time). Due to the real  
world complexities and the amount of heuristics needed, browser support  
has been low.

All three HTTP/2 proposals address this by explicitly mandating  
pipelining, and improves it with out-of-order response to address stalling  
of a specific resource. Compared with HTTP/1.1 it would likely make little  
difference, as today resources with similar response times are grouped on  
different hosts.

The biggest increase in header size is, to the best of my knowledge,  
ironically on the mobile side, where the use of UA-prof-diff-headers have  
resulted in several kilobytes of data extra per request on some devices.  
This is again addressed in all the proposals, deflate for SPDY and a  
bucket based delta compression for Network-Friendly. We use a straight  
delta compression (only send headers that changed from previous request)  
in all of Operas own HTTP-replacements. Comparing against HTTP/1.1 should  
give a good improvement in radio time allocation in a low bandwidth, high  
latency environment.

Video, specifically rate limited video, adds the requirement of  
multiplexing the data streams, as an HTTP/1.1 pipeline would stall as soon  
as the video starts being transferred. All three proposals should give  
much better user experience than HTTP/1.1 when a page consists of both  
very large and very small resources (e.g. large background and small  
inline images). In terms of raw numbers, multiplexing (insignificantly)  
increases the amount of data.

As for minification, it only adds confusion. You can take a site, minify  
the resources and pack it all together into a 7zip package, and no one  
will be surprised that downloading it is faster than loading the  
unprocessed site. But it doesn't tell you anything about the protocols  
used to perform the downloads.

/Martin Nilsson

On Mon, 30 Jul 2012 08:32:07 +0200, Henrik Frystyk Nielsen  
<[email-masked]> wrote:

> Martin,
>
> What scenarios do you have in mind where pipelining does not work?  
> Obviously it only works for GET so for web sites using lots of non-GET  
> requests it won't work but GET is the predominant method on most sites.
>
> We took some pretty normal sites so I do think they represent reasonable  
> data. We'd be happy to expand the sites but the whole point was to show  
> differences in relatively common sites. It is correct that we didn't use  
> cookies. Cookie and UA-header sizes can indeed vary a lot but that is a  
> much more isolated problem to solve that can be done in any number of  
> ways.
>
> As for video (assuming you use TCP) the bottleneck is likely going to be  
> set by TCP throughput. The overhead of HTTP headers will be negligible  
> due to the size of the payload. We'd be happy to do some tests but all  
> things being equal I can say based on experience that adding a  
> credit-based session layer such as SPDY will not perform as well as  
> running straight over TCP for large payloads. The reason is that the  
> credit scheme by its very purpose acts as a throttle so that one session  
> doesn't take the entire bandwidth.
>
> As for minification, the intention is not to do those unbeknownst to the  
> website designer -- it is something that is an integral part of the  
> design of a web site so there is no reason why this isn't a valid  
> optimization.
>
> Thanks,
>
> Henrik
>

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 30 Jul 2012 16:41:33 +0000

In message <op.wh9pqtcwiw9drz@uranium>, "Martin Nilsson" writes:

>responses written to the same socket at the same time). 

A surprising number of intermediaries expect HTTP/1.1 request to start
at a packet boundary...

---

From: "Martin Nilsson" <[email-masked]>
Date: Mon, 30 Jul 2012 19:14:19 +0200

> mandatory to implement for me.

I think such requirement will immediately prompt development of  
HTTP/2-light without TLS for use in intra-data-center communications (and  
China and India).

/Martin Nilsson

---

From: Jitu Padhye <[email-masked]>
Date: Mon, 30 Jul 2012 17:36:06 +0000

Ø  Was the client/server speaking SPDY/2 or SPDY/3?
      Spdy/2

Ø  A percentage of the time, Chrome will not use SPDY (it decides this at startup). Did you verify that it was always speaking SPDY? (I'd guess yes!)

Yes.

Ø  Was the server using the CWND set sent in the SETTINGS frame, and was it sending it to the client?

I did not check this - I will go back and look.

Ø  What was the server's initcwnd?

Whatever the default is. I think it was 2 or 4. I will take a look.

Ø  Was slow-start-after-idle turned off on the kernel?

No. Again, we only used default settings.

Ø  How long were the various connections in slow start, and did they ever exit slow-start?

I will have to take a look.

Ø  Was the SSL cert chain the same size and using the same algorithms?

Yes.

Ø  Was the dummynet (or equivalent) configured to have appropriate buffer size (else it does packet drops silently)?

Yes, and I know this, because I had initially screwed this up :)

Ø  Were these all cold-connection, cold-cache first-page load timings? The use of keep-alive in figure 3 would seem to indicate otherwise, but I'm not sure which are which...

All cold cache. Keep alive is for only fetches while loading the first page.

Ø  Did you put Chromium into 'replay' mode (where date and random always return in the same manner, regardless of actual date)?

No, but I agree that this setting would have made more sense.

Ø  Any particular reason that you turned off entity-body compression in Apache? Were the resources saved verbatim from WinHTTrack (in which case it'd make sense)?

Not quite sure what you are asking here ... can you please clarify?

Ø  Was the server emulating the original server's think times? This can have a very, very large impact on pipelined HTTP performance.
      Only "index.html" was loaded for each site. I should clarify that in text. So think times are not an issue.

Ø  We've found that the various automatic page-is-done signals are often less useful than something which measures the above-the-fold page load time. This generally involves cameras pointed at screens and a human looking at the video to determine at which frame the page is useful, and designating that frame as 'done-for-above-the-fold'

I agree that age load time, as we measured it, is not a foolproof metric. The problem is that all other alternatives are too cumbersome to measure (and worse) replicate correctly.

Ø  I'd love to see if we could get a common suite of test data over which we could all run experiments/benchmarks. Is anyone else interested in doing so?

This is definitely interesting. Something like RFC5033?

From: Roberto Peon [mailto:[email-masked]]
Sent: Sunday, July 29, 2012 3:16 PM
To: Henrik Frystyk Nielsen
Cc: [email-masked]; Gabriel Montenegro; Rob Trace; Adalberto Foresti (MS OPEN TECH)
Subject: Re: HTTP 2.0 and a Faster, more Mobile-friendly web

Was the client/server speaking SPDY/2 or SPDY/3?
A percentage of the time, Chrome will not use SPDY (it decides this at startup). Did you verify that it was always speaking SPDY? (I'd guess yes!)
Was the server using the CWND set sent in the SETTINGS frame, and was it sending it to the client?
What was the server's initcwnd?
Was slow-start-after-idle turned off on the kernel?
How long were the various connections in slow start, and did they ever exit slow-start?
Was the SSL cert chain the same size and using the same algorithms?
Was the dummynet (or equivalent) configured to have appropriate buffer size (else it does packet drops silently)?
Were these all cold-connection, cold-cache first-page load timings? The use of keep-alive in figure 3 would seem to indicate otherwise, but I'm not sure which are which...
Did you put Chromium into 'replay' mode (where date and random always return in the same manner, regardless of actual date)?
Any particular reason that you turned off entity-body compression in Apache? Were the resources saved verbatim from WinHTTrack (in which case it'd make sense)?
Was the server emulating the original server's think times? This can have a very, very large impact on pipelined HTTP performance.

We've found that the various automatic page-is-done signals are often less useful than something which measures the above-the-fold page load time. This generally involves cameras pointed at screens and a human looking at the video to determine at which frame the page is useful, and designating that frame as 'done-for-above-the-fold'

I'd love to see if we could get a common suite of test data over which we could all run experiments/benchmarks.
Is anyone else interested in doing so?

-=R
On Sun, Jul 29, 2012 at 1:14 PM, Henrik Frystyk Nielsen <[email-masked]<mailto:[email-masked]>> wrote:

Dear All,

--- Taking the Best from Each ---

Area              | Opinion that

                  | seems to prevail

2. Multiplexing   | SPDY

                  |   Speed+Mobility

      Server Push |

6. Flow Control   | SPDY

1. Compression

2. Multiplexing

4. Negotiation

6. Flow Control

7. WebSockets

--- Moving forward ---

Sincerely,

Henrik Frystyk Nielsen

Gabriel Montenegro

Rob Trace

Adalberto Foresti

---

From: Jonathan Ballard <[email-masked]>
Date: Mon, 30 Jul 2012 13:22:02 -0700

Where that is all cold there is the medium expiration in the mentioned
common suite: chrome://chrome/settings/content/expiration/medium?true.

SPDY+TLS offers no immediate exception except recompile resources to local
storage, and all of that is said off-topic at Vancouver on or about USB.

On Monday, July 30, 2012, Jitu Padhye wrote:

>  **Ø  **Was the client/server speaking SPDY/2 or SPDY/3?****
>
>       Spdy/2****
>
> ** **
>
> **Ø  **A percentage of the time, Chrome will not use SPDY (it decides
> this at startup). Did you verify that it was always speaking SPDY? (I'd
> guess yes!)****
>
> Yes. ****
>
> ** **
>
> **Ø  **Was the server using the CWND set sent in the SETTINGS frame, and
> was it sending it to the client?****
>
> I did not check this – I will go back and look. ** **
>
> ** **
>
> **Ø  **What was the server's initcwnd?****
>
> Whatever the default is. I think it was 2 or 4. I will take a look. ****
>
> ** **
>
> **Ø  **Was slow-start-after-idle turned off on the kernel?****
>
> No. Again, we only used default settings. ****
>
> ** **
>
> **Ø  **How long were the various connections in slow start, and did they
> ever exit slow-start?****
>
> I will have to take a look. ****
>
> ** **
>
> **Ø  **Was the SSL cert chain the same size and using the same algorithms?
> ****
>
> Yes. ****
>
> ** **
>
> **Ø  **Was the dummynet (or equivalent) configured to have appropriate
> buffer size (else it does packet drops silently)?****
>
> Yes, and I know this, because I had initially screwed this up J****
>
> ** **
>
> **Ø  **Were these all cold-connection, cold-cache first-page load
> timings? The use of keep-alive in figure 3 would seem to indicate
> otherwise, but I'm not sure which are which...****
>
> All cold cache. Keep alive is for only fetches while loading the first
> page. ****
>
> ** **
>
> **Ø  **Did you put Chromium into 'replay' mode (where date and random
> always return in the same manner, regardless of actual date)?****
>
> No, but I agree that this setting would have made more sense. ****
>
> ** **
>
> **Ø  **Any particular reason that you turned off entity-body compression
> in Apache? Were the resources saved verbatim from WinHTTrack (in which case
> it'd make sense)?****
>
> Not quite sure what you are asking here … can you please clarify?****
>
> ** **
>
> **Ø  **Was the server emulating the original server's think times? This
> can have a very, very large impact on pipelined HTTP performance. ****
>
>       Only “index.html” was loaded for each site. I should clarify that in
> text. So think times are not an issue. ****
>
> ** **
>
> **Ø  **We've found that the various automatic page-is-done signals are
> often less useful than something which measures the above-the-fold page
> load time. This generally involves cameras pointed at screens and a human
> looking at the video to determine at which frame the page is useful, and
> designating that frame as 'done-for-above-the-fold'****
>
> I agree that age load time, as we measured it, is not a foolproof metric.
> The problem is that all other alternatives are too cumbersome to measure
> (and worse) replicate correctly. ****
>
> ** **
>
> **Ø  **I'd love to see if we could get a common suite of test data over
> which we could all run experiments/benchmarks. Is anyone else interested in
> doing so?****
>
> This is definitely interesting. Something like RFC5033?****
>
> ** **
>
> ** **
>
> *From:* Roberto Peon [mailto:[email-masked] <javascript:_e({}, 'cvml',
> [email-masked]');>]
> *Sent:* Sunday, July 29, 2012 3:16 PM
> *To:* Henrik Frystyk Nielsen
> *Cc:* [email-masked] <javascript:_e({}, 'cvml',
> [email-masked]');>; Gabriel Montenegro; Rob Trace; Adalberto
> Foresti (MS OPEN TECH)
> *Subject:* Re: HTTP 2.0 and a Faster, more Mobile-friendly web****
>
> ** **
>
> ** **
>
> Interesting data! Questions about test methodology:****
>
> ** **
>
> Was the client/server speaking SPDY/2 or SPDY/3?****
>
> sending it to the client?****
>
> What was the server's initcwnd?****
>
> Was slow-start-after-idle turned off on the kernel?****
>
> How long were the various connections in slow start, and did they ever
> exit slow-start?****
>
> Was the SSL cert chain the same size and using the same algorithms?****
>
> Was the dummynet (or equivalent) configured to have appropriate buffer
> size (else it does packet drops silently)?****
>
> Were these all cold-connection, cold-cache first-page load timings? The
> use of keep-alive in figure 3 would seem to indicate otherwise, but I'm not
> sure which are which...****
>
> return in the same manner, regardless of actual date)?****
>
> it'd make sense)?****
> Was the server emulating the original server's think times? This can have
> a very, very large impact on pipelined HTTP performance. ****
>
> ** **
>
> that frame as 'done-for-above-the-fold'****
>
> ** **
>
> could all run experiments/benchmarks.****
>
> Is anyone else interested in doing so?****
>
> ** **
>
> -=R****
>
> [email-masked]> wrote:****
>
> Dear All,****
>
>  ****
>
> optimizations to HTTP/1.1. You can find a prelim
>

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Mon, 30 Jul 2012 22:31:00 +0000

------ Original Message ------
From: "patrick mcmanus" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sent: 31/07/2012 3:09:57 a.m.
Subject: Re: HTTP 2.0 and a Faster, more Mobile-friendly web
>right now, but it does not exclude other solutions. Other properties 
>of TLS are desirable too, but they don't necessarily rise to the level 
>of mandatory to implement for me. 
>

by "must be secure", I take it you mean "TLS must be used".  Correct?

Otherwise you'd have said "must be securable" or similar?

Adrien

---

From: Henrik Frystyk Nielsen <[email-masked]>
Date: Tue, 31 Jul 2012 03:49:39 +0000

Sorry for the delay -- been traveling from Denmark to Seattle and will go to Vancouver tomorrow morning -- hope to see many of you there!

To be clear, I am not claiming that HTTP pipelining is the end-all-be-all -- it clearly has limitations -- but it does allow us to compare on a more apples-to-apples basis what actually improves performance, what doesn't, and what is in the noise. I think that is necessary to know in order to be able to evaluate whether we are headed in the right direction. I hope we can build on the data presented here and get a better understanding of where we are.

I hope we get some time to discuss some of the issues such as cancellation etc. -- lots of interesting (and tricky) issues!

Henrik

-----Original Message-----
From: patrick mcmanus [mailto:[email-masked]] 
Sent: Monday, July 30, 2012 08:37
To: [email-masked]
Subject: Re: HTTP 2.0 and a Faster, more Mobile-friendly web

> What scenarios do you have in mind where pipelining does not work? Obviously it only works for GET so for web sites using lots of non-GET requests it won't work but GET is the predominant method on…

I'm a believer that we can get more benefit from pipelines than we currently do, and I remain committed to seeing that happen on today's
http/1 web. But the challenges go beyond non-get and mangling intermediaries(*).. so in a pro-pipeline, but eyes-wide-open spirit:

1] you still need parallelism because of priority head of line blocking issues. But now some of the parallel flows are filling the pipe and buffers very well, and its hard to manage the prioritization effectively. Using a priority mux instead of pipeline+parallelism makes that work a bit better (buffering in the network remains a problem - though with fewer connections they should at least not be filled as deeply)

2] even on get - blocking responses are common and they screw up the pipeline. database operations, web service callouts, etc all can create variability in response times, and that creates time-to-utilization gaps

3] http/1 cancellation semantics don't exist other than closing the connection, and pipelines make this worse. This comes up all the time in the form of links clicked on partially rendered pages. (think "next" on a webmail inbox). To process the click you need to decide between waiting for the old one to drain, adding more to its existing pipeline, or making a new connection.. this problem always existed without pipelines but the amount of data in the queue has now gone up.

[*] in my experience the most significant intermediary challenges these days come from AV and firewall software.. actual network devices and servers fair ok with a couple notable exceptions. I don't know that this makes it any easier :)

pipelines are useful things - but I think we can do better than just working around their warts.

> We took some pretty normal sites so I do think they represent reasonable data. We'd be happy to expand the sites but the whole point was to show differences in relatively common sites. It is correc…

cookies and pipeline performance are pretty tightly tied. Pipeline depth is driven by client cwnd and the difference in pipeline depth on a cold TCP session goes from 10 to 2 as cookie sizes go from 0 to 2KB. An
HTTP/2 approach that removes header redundancy (whether that be by some manual table approach like network-friendly, or a automagic gzip approach like spdy) is critical to addressing the fundamental round trip problem.

cheers,
-Patrick

---

From: Mike Belshe <[email-masked]>
Date: Mon, 30 Jul 2012 21:34:13 -0700

Henrik and team -

Awesome paper!

One thing I notice is that your test pages did not use many resources.  (3
for the dummy web page and 11 for the cnn page).  As you know (
http://httparchive.org/trends.php) pages today tend to have ~80 resources
per page.  Its the large-resource pages where HTTP slows down, as it can
only fetch 6 at a time (2 if you stick to the spec).   It would be
interesting to load up a more modern page, like a facebook.com wall page,
with dozens of images and resources to really exercise the multiplexing.

The biggest conclusion I read from this is that SSL is hard to make fast
(and also very true!) :-)  I'm glad you benchmarked with Chromium because
it uses SSL FalseStart to reduce a round trip.  I'm curious if you did
anything for certificate validation in this test?  It looks like you're
generally seeing a 2RTT overhead of SSL in your tests, which is indicative
of the SSL cold-start case.  Can you confirm you wiped the session-ID cache
between runs?  It's also indicative of not really exercising the
multiplexing, where the SSL overhead gets mitigated as HTTP slows down with
6-at-a-time round trips.

Mike

---

From: Roberto Peon <[email-masked]>
Date: Tue, 31 Jul 2012 00:15:00 -0700

Thanks for answering by questions!

> Whatever the default is. I think it was 2 or 4. I will take a look.

A default of 10 is suggested. (This is the case for newer linux kernels by
default).

> Yes, and I know this, because I had initially screwed this up J
>

Yea, it is easy to mess this one up-- good catch!

> page.
It would be instructive to look at more than just the first page load. I
suspect most site designers and sites intend to have the user continue
interacting with the site for more than just the one page view :)

This is the curse of the benchmark-- someone always demanding more. In this
case, I do think it is important, though, since the pattern is further site
involvement much of the time.

> Not quite sure what you are asking here … can you please clarify?
>

I think this question may not make sense if resources other than index.html
were not served from the server as I think you're saying elsewhere?
What I mean was: If the entity-body for the site was originally compressed,
are you sending it compressed in your tests, or would it end up
uncompressed?

> text. So think times are not an issue.
>

The resources referenced from index.html are, or are not part of the test?
I'd guess that they are, and I've confused you somehow.

> This is definitely interesting. Something like RFC5033?

I'm not finding what I'm suggesting in that RFC, but it is possible it is
there :)

All I mean is that we grab a number of pages that represent important
use-cases, then, as necessary, clean up the resources so that there are no
legal issues w.r.t. distribution, and then have this used as a basis for
benchmarking and testing of corner-cases.
-=R

> **
>
> ** **
>
> ** **
>
> *From:* Roberto Peon [mailto:[email-masked]]
> *Cc:* [email-masked]; Gabriel Montenegro; Rob Trace; Adalberto
> Foresti (MS OPEN TECH)
>

---

From: Jitu Padhye <[email-masked]>
Date: Tue, 31 Jul 2012 17:54:57 +0000

> It would be instructive to look at more than just the first page load. I suspect most site designers and sites intend to have the user continue interacting with the site for more than just the one page view :) This is the curse of the benchmark-- someone always demanding more. In this case, I do think it is important, though, since the pattern is further site involvement much of the time.

Yes, I agree. This will be more useful.  I will add it for things to check in the next set. 
 
> What I mean was: If the entity-body for the site was originally compressed, are you sending it compressed in your tests, or would it end up uncompressed?
 
Ah. I see. I did not check what the original compression status was.

> The resources referenced from index.html are, or are not part of the test? I'd guess that they are, and I've confused you somehow. 
 
They are .... (e.g. images, javascripts, css files etc. )
 
> I'm not finding what I'm suggesting in that RFC, but it is possible it is there :)

I only meant it as an attempt to do something similar ... I worked on initial version of that along time ago (a common suite of tests/scenarios to test proposed congestion control algorithms), so I thought of it by analogy. 

 

 
-=R
On Sun, Jul 29, 2012 at 1:14 PM, Henrik Frystyk Nielsen <[email-masked]> wrote:
Dear All,

 

--- A Genuinely Faster Web ---
 
The SPDY proposal has been great for raising awareness of Web performance. It takes a “clean slate” approach to improving HTTP.

Our results indicate almost equal performance between SPDY and HTTP/1.1 when one applies all the known optimizations to HTTP/1.1. SPDY’s performance improvements are not consistent and significant. We will continue our testing, and we welcome others to publish their results so that HTTP/2.0 can choose the best changes and deliver the best possible performance and scalability improvements compared to HTTP/1.1.

--- Taking the Best from Each ---
 
Speed is one of several areas of improvement. Currently, there’s no clear consensus that any one of the proposals is the clear choice or even starting point for HTTP/2.0 (based on our reading the Expressions of Interest and discussions on this mailing list. A good example of this is the vigorous discussion around mandating TLS encryption (http://tools.ietf.org/html/rfc5246) for HTTP/2.0. 

 

1. Compression
 
Compression is simple to conceptualize and implement, and it is important. Proxies and other boxes in the middle on today’s Web often face problems with it. The HTTP/2.0 discussion has been rich but with little consensus.
Though some studies suggest that SPDY’s header compression approach shows promise, other studies show this compression to be prohibitively onerous for intermediary devices. More information here would help us make sure we’re making the Web faster and better. 
 
Also, an entire segment of implementers are not interested in compression as defined in SPDY.  That’s a challenge because the latest strawman for the working group charter (http://lists.w3.org/Archives/Public/ietf-http-wg/2012JulSep/0784.html) states that the “resulting specification(s) are expected to be meet these goals for common existing deployments of HTTP; in particular, … intermediation (by proxies, corporate firewalls, ‘reverse’ proxies and Content Delivery Networks).” 

2. Multiplexing
 
All three proposals define similar multiplexing models. We haven’t had substantial discussion on the differences. This lack of discussion suggests that there is rough consensus around the SPDY framing for multiplexing. 

 
There is definitely no consensus to mandate TLS for all Web communication, but some major implementers have stated they will not to adopt HTTP/2.0 unless the working group supports a “TLS is mandatory” position. A very preliminary note from the chair (http://lists.w3.org/Archives/Public/ietf-http-wg/2012JulSep/0601.html) states that there is a lack of consensus for mandating TLS. 

4. Negotiation

 

 
Server Push has not had the same level of implementation and experimentation as the other features in SPDY. More information here would help us make sure we’re making the Web faster and better. 

6. Flow Control

The current version of the SPDY proposal specifies a flow control message with many settings are that are not well-defined. The Speed+Mobilty proposal has a simplified flow control model based on certain assumptions. More experimentation and information here would help us make sure we’re making the Web faster and better. 

7. WebSockets

--- Moving forward ---
 
We’re excited for the Web to get faster, more stable, and more capable, and HTTP/2.0 is an important part of that. 

Based on the discussions on the HTTPbis mailing list, we’ve suggested which proposals make the most sense to start from for each of the areas that HTTP/2.0 is addressing. Each of these areas needs more prototyping and experimentation and data. We’re looking forward to the discussion this week.
 
Sincerely,
 
Henrik Frystyk Nielsen
 
Gabriel Montenegro
 
Rob Trace
 
Adalberto Foresti

---

From: Jitu Padhye <[email-masked]>
Date: Tue, 31 Jul 2012 17:56:53 +0000

Mike,

Yes, we do need to explore a larger variety of web pages, including web pages with more resources.

I did not explicitly wipe session state cache between runs. Is there a Chrome switch to do this easily?

Jitu

From: Mike Belshe [mailto:[email-masked]]
Sent: Monday, July 30, 2012 9:34 PM
To: Henrik Frystyk Nielsen
Cc: [email-masked]; Gabriel Montenegro; Rob Trace; Adalberto Foresti (MS OPEN TECH)
Subject: Re: HTTP 2.0 and a Faster, more Mobile-friendly web

Henrik and team -

Awesome paper!

One thing I notice is that your test pages did not use many resources.  (3 for the dummy web page and 11 for the cnn page).  As you know (http://httparchive.org/trends.php) pages today tend to have ~80 resources per page.  Its the large-resource pages where HTTP slows down, as it can only fetch 6 at a time (2 if you stick to the spec).   It would be interesting to load up a more modern page, like a facebook.com<http://facebook.com> wall page, with dozens of images and resources to really exercise the multiplexing.

The biggest conclusion I read from this is that SSL is hard to make fast (and also very true!) :-)  I'm glad you benchmarked with Chromium because it uses SSL FalseStart to reduce a round trip.  I'm curious if you did anything for certificate validation in this test?  It looks like you're generally seeing a 2RTT overhead of SSL in your tests, which is indicative of the SSL cold-start case.  Can you confirm you wiped the session-ID cache between runs?  It's also indicative of not really exercising the multiplexing, where the SSL overhead gets mitigated as HTTP slows down with 6-at-a-time round trips.

Mike

Dear All,

--- Taking the Best from Each ---

Area              | Opinion that

                  | seems to prevail

2. Multiplexing   | SPDY

                  |   Speed+Mobility

      Server Push |

6. Flow Control   | SPDY

1. Compression

2. Multiplexing

4. Negotiation

6. Flow Control

7. WebSockets

--- Moving forward ---

Sincerely,

Henrik Frystyk Nielsen

Gabriel Montenegro

Rob Trace

Adalberto Foresti

---

From: James M Snell <[email-masked]>
Date: Tue, 31 Jul 2012 12:59:52 -0700

Excellent to see this kind of analysis being performed. A few comments
inline..

[snip]

> mixed results with SPDY performance.
>
One of the frustrating issues for me, so far, is that nearly all of the
discussion around HTTP/2.0 to date has centered around the requirements for
browser clients; given the ever growing use HTTP as an enabler for
integration APIs and the ever increasing number of non-browser HTTP client
applications, it would be very interesting to formally compare and analyze
the performance of SPDY vs. current HTTP/1.1 for *API* clients and servers.
In my informal testing, I have seen very little difference, if any, between
using SPDY* and a client using persistent HTTP/1.1 connections with
pipelined API requests.
  * with the current http mapping defined in the spdy spec

With a few tweaks here and there to the SPDY+HTTP mapping (e.g. the binary
header encoding I had kicked around previously) I was able to squeeze a bit
more out the SPDY option but the results were certainly not conclusive.

This, of course, was not a formal study and my methodology was far from
rigorous. A more thorough analysis would be required.

- James

---

From: Jitu Padhye <[email-masked]>
Date: Tue, 31 Jul 2012 20:45:07 +0000

We have done a bit of work on looking at how SPDY works for apps (mostly simple trace analysis), but we don’t have anything significant to report yet. But this is indeed something that needs looking into.

From: James M Snell [mailto:[email-masked]]
Sent: Tuesday, July 31, 2012 1:00 PM
To: Henrik Frystyk Nielsen
Cc: [email-masked]; Gabriel Montenegro; Rob Trace; Adalberto Foresti (MS OPEN TECH)
Subject: Re: HTTP 2.0 and a Faster, more Mobile-friendly web

Excellent to see this kind of analysis being performed. A few comments inline..
On Sun, Jul 29, 2012 at 1:14 PM, Henrik Frystyk Nielsen <[email-masked]<mailto:[email-masked]>> wrote:
[snip]

One of the frustrating issues for me, so far, is that nearly all of the discussion around HTTP/2.0 to date has centered around the requirements for browser clients; given the ever growing use HTTP as an enabler for integration APIs and the ever increasing number of non-browser HTTP client applications, it would be very interesting to formally compare and analyze the performance of SPDY vs. current HTTP/1.1 for *API* clients and servers. In my informal testing, I have seen very little difference, if any, between using SPDY* and a client using persistent HTTP/1.1 connections with pipelined API requests.
  * with the current http mapping defined in the spdy spec

With a few tweaks here and there to the SPDY+HTTP mapping (e.g. the binary header encoding I had kicked around previously) I was able to squeeze a bit more out the SPDY option but the results were certainly not conclusive.

This, of course, was not a formal study and my methodology was far from rigorous. A more thorough analysis would be required.

- James

---

From: Benjamin Carlyle <[email-masked]>
Date: Wed, 1 Aug 2012 11:42:38 +1000

On Aug 1, 2012 6:47 AM, "Jitu Padhye" <[email-masked]> wrote:
> We have done a bit of work on looking at how SPDY works for apps (mostly
simple trace analysis), but we don’t have anything significant to report
yet. But this is indeed something that needs looking into.

The use case where SPDY should offer an advantage is when many requests are
in transit concurrently and there is significant variability in response
times.
Responsiveness of both http and SPDY will depend on the protocol stack's
ability to keep the TCP buffer full of messages, and the real test cases
might be between proxies at the edges of data centres or on other high
volume links.

I have had success with pipelined http as part of SCADA user interfaces. In
this use case we want to bring up a display containing ten thousand
individual data points that each have individual urls. With ordinary
pipelined http/1.1 and a good http implementation I can complete the
initial GET requests plus establish subscriptions and perform all rendering
within a couple of seconds on commodity hardware. In this case response
times are very consistent so server push aside SPDY should not offer
significant raw performance improvement. For workloads with variable
response times I use draft- mogul- http- ooo- 00, which works very well
when supported consistently across the architecture.

I wonder a little as to whether browser protocol stacks will be
appropriately tuned for high message loads to provide good baseline http
figures. A fairer comparison might be obtained by essentially dumping bytes
across the network and comparing on that basis. Alternatively SPDY should
be compared not to http but instead to existing enterprise message queuing
protocols.

In fact when it comes down to it if we're defining a new http primarily for
the purposes of improving performance then existing http isn't really a
good baseline. Instead we should probably be looking at the best performing
available protocols and comparing against those as baselines. It's
potentially very easy to be faster. The question is have we achieved "fast"
such that it would be pointless for anyone to build and use custom
protocols in place of http for transporting http message content except in
extreme cases? I think enterprise experience as to what constitutes "fast"
or "fast enough" and what constitutes an ordinary workload should be
considered.

Benjamin