Subject: Performance implications of Bundling and Minification on HTTP/1.1

From: Henrik Frystyk Nielsen <[email-masked]>
Date: Fri, 22 Jun 2012 17:18:52 +0000

We just published a blog [1] analyzing the performance implications of content optimizations such as bundling and minification on the performance of web pages. The data shows that by applying bundling and minification along with compression and pipelining it is possible to get significant gains in the time it takes to get the content necessary to render a page as well as the overall time it takes to download the data.

Not only does optimizing the content save bytes but it also has savings in the number of requests and responses that need to be processed as well as faster render times due to being able to retrieve the HTML, CSS, and JS up front. In the test evaluated, the speedup was from 638 ms (uncompressed, unbundled, unminified, and not pipelined) down to 146 ms for the equivalent compressed, bundled, minified, and pipelined content. However, by just looking at the data necessary to lay out the page (HTML, CSS, and JS but not images), the time went from 631 ms to 126 ms with the images being finalized within the remaining timespan from 126 to 146 ms.

It is the hope that this data can contribute to providing a baseline for evaluating HTTP/2.0 proposals compared to how an efficient HTTP/1.x implementation can perform while leverage optimizations throughout the stack to provide better user experience.

Comments welcome!

Thanks,

Henrik

[1] http://blogs.msdn.com/b/henrikn/archive/2012/06/17/performance-implications-of-bundling-and-minification-on-http.aspx

---

From: William Chan (陈智昌) <[email-masked]>
Date: Fri, 22 Jun 2012 11:30:14 -0700

Thanks for posting data here! Very much appreciated. I'm curious if you
have any normative claims to make about how this should impact HTTP/2.0
proposals. I can see arguments for how some of these techniques rightly
belong in the application layer, whereas some are working around issues in
HTTP, and we may want to address in HTTP/2.0. Oh, and I'm also curious
about your thoughts with regard to pipelining, since you brought it up in
this post and have noted that it has practical deployment issues.

On Fri, Jun 22, 2012 at 10:18 AM, Henrik Frystyk Nielsen <
[email-masked]> wrote:

>  We just published a blog [1] analyzing the performance implications of
> content optimizations such as bundling and minification on the performance
> of web pages. The data shows that by applying bundling and minification
> along with compression and pipelining it is possible to get significant
> gains in the time it takes to get the content necessary to render a page as
> well as the overall time it takes to download the data.****
>
> ** **
>
> Not only does optimizing the content save bytes but it also has savings in
> the number of requests and responses that need to be processed as well as
> faster render times due to being able to retrieve the HTML, CSS, and JS up
> front. In the test evaluated, the speedup was from 638 ms (uncompressed,
> unbundled, unminified, and not pipelined) down to 146 ms for the equivalent
> compressed, bundled, minified, and pipelined content. However, by just
> looking at the data necessary to lay out the page (HTML, CSS, and JS but
> not images), the time went from 631 ms to 126 ms with the images being
> finalized within the remaining timespan from 126 to 146 ms.****
>
> ** **
>
> It is the hope that this data can contribute to providing a baseline for
> evaluating HTTP/2.0 proposals compared to how an efficient HTTP/1.x
> implementation can perform while leverage optimizations throughout the
> stack to provide better user experience.****
>
> ** **
>
> Comments welcome!****
>
> ** **
>
> Thanks,****
>
> ** **
>
> Henrik****
>
> ** **
>
> [1]
> http://blogs.msdn.com/b/henrikn/archive/2012/06/17/performance-implications-of-bundling-and-minification-on-http.aspx
> ****
>
> ** **
>

---

From: Henrik Frystyk Nielsen <[email-masked]>
Date: Fri, 22 Jun 2012 19:05:06 +0000

The main point is to establish a baseline for how well an optimized HTTP/1.1 implementation with optimized content can perform. Given such a baseline it is much easier to compare HTTP/2.0 ideas and evaluate the positives and/or negatives of alternative solutions. As such I am not making any normative claims per se but there indeed are two points that I think are important to observe:

1)      It seems safe to assert that whatever we come up with in HTTP/2.0 should be significantly faster than what can be achieved with an optimized HTTP/1.1 implementation. I don’t think speed is the only reason for considering HTTP/2.0 but it is an important one and so we have to be able to compare numbers.

2)      Without taking a broader view of performance that includes the content as an integral part you simply cannot meaningfully expect to get a fast system. In other words, there is no way that an application protocol can compensate for badly composed content. For example, if you put your links to CSS and JS deep down in your HTML or rely on 100 requests to be complete in order to render a page then no amount of protocol optimizations can effectively help you. Mechanisms such as bundling, minification, and compression can play a significant role here.

As to pipelining, the reality is that being able to send requests and responses without delay is a must in any environment with noticeable RTTs. Whether it happens using HTTP/1.1 pipelining or via some other mechanism can be discussed but there is no way that we can get better performance without it. I do understand that there are limitations in how well it is deployed but I am dubious that just deploying something different inherently will solve that unless we know the root cause.

Thanks,

Henrik

From: [email-masked] [mailto:[email-masked]] On Behalf Of William Chan (???)
Sent: Friday, June 22, 2012 11:30 AM
To: Henrik Frystyk Nielsen
Cc: HTTP Working Group; Howard Dierking
Subject: Re: Performance implications of Bundling and Minification on HTTP/1.1

Thanks for posting data here! Very much appreciated. I'm curious if you have any normative claims to make about how this should impact HTTP/2.0 proposals. I can see arguments for how some of these techniques rightly belong in the application layer, whereas some are working around issues in HTTP, and we may want to address in HTTP/2.0. Oh, and I'm also curious about your thoughts with regard to pipelining, since you brought it up in this post and have noted that it has practical deployment issues.

On Fri, Jun 22, 2012 at 10:18 AM, Henrik Frystyk Nielsen <[email-masked]<mailto:[email-masked]>> wrote:
We just published a blog [1] analyzing the performance implications of content optimizations such as bundling and minification on the performance of web pages. The data shows that by applying bundling and minification along with compression and pipelining it is possible to get significant gains in the time it takes to get the content necessary to render a page as well as the overall time it takes to download the data.

Comments welcome!

Thanks,

Henrik

---

From: Roberto Peon <[email-masked]>
Date: Fri, 22 Jun 2012 13:08:45 -0700

Nice data!

On Fri, Jun 22, 2012 at 12:05 PM, Henrik Frystyk Nielsen <
[email-masked]> wrote:

>  The main point is to establish a baseline for how well an optimized
> HTTP/1.1 implementation with optimized content can perform. Given such a
> baseline it is much easier to compare HTTP/2.0 ideas and evaluate the
> positives and/or negatives of alternative solutions. As such I am not
> making any normative claims per se but there indeed are two points that I
> think are important to observe:****
>
> ** **
>
> **1)      **It seems safe to assert that whatever we come up with in
> HTTP/2.0 should be significantly faster than what can be achieved with an
> optimized HTTP/1.1 implementation. I don’t think speed is the only reason
> for considering HTTP/2.0 but it is an important one and so we have to be
> able to compare numbers.****
>
> **
>
I'd argue another point.
The amount of work necessary to optimize site performance for HTTP/1.1
today is large. The amount of knowledge necessary to do it properly is also
large.
This is not the way it should be!

The protocol should make it easier to do things right, and it should help
in the (extremely frequent and likely) case that the site designer gets it
wrong in little ways.

> **
>
> **2)      **Without taking a broader view of performance that includes
> the content as an integral part you simply cannot meaningfully expect to
> get a fast system. In other words, there is no way that an application
> protocol can compensate for badly composed content. For example, if you put
> your links to CSS and JS deep down in your HTML or rely on 100 requests to
> be complete in order to render a page then no amount of protocol
> optimizations can effectively help you. Mechanisms such as bundling,
> minification, and compression can play a significant role here.
>
"...no amount of protocol optimizations can effectively help you."
Note that this isn't strictly true :)
Feeding back into the statement I made above, as an example: response
reordering/interleaving, etc. can go pretty far in mitigating the damage
done by suboptimal link ordering. It can't help in some pathological cases,
true, but I'd guess that most sites don't present the pathological worst
case...
A protocol which allows for bundling without requiring the site developer
to deal with it increases the chance that a site's latency approaches
closer to optimal.

> ****
>
> ** **
>
> As to pipelining, the reality is that being able to send requests and
> responses without delay is a must in any environment with noticeable RTTs.
> Whether it happens using HTTP/1.1 pipelining or via some other mechanism
> can be discussed but there is no way that we can get better performance
> without it. I do understand that there are limitations in how well it is
> deployed but I am dubious that just deploying something different
> inherently will solve that unless we know the root cause.
>

But don't we already understand the root cause? Isn't it intercepting
proxies?
Basically, that a number of proxies considered parts of the spec optional,
and took shortcuts, and are deployed in such a fashion as to intercept and
disrupt communications between parties that would otherwise be able to
communicate without problem?

Deploying over a channel that does not have these intercepting proxies
solves this problem. We know that we don't have this problem to any degree
that matters for SPDY, for instance.
Is it worth generating a separate table for doing the various optimizations
over HTTPS?
-=R

> ****
>
> ** **
>
> Thanks,****
>
> ** **
>
> Henrik****
>
> ** **
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of *William
> Chan (???)
> *Sent:* Friday, June 22, 2012 11:30 AM
> *To:* Henrik Frystyk Nielsen
> *Cc:* HTTP Working Group; Howard Dierking
> *Subject:* Re: Performance implications of Bundling and Minification on
> HTTP/1.1****
>
> ** **
>
> this post and have noted that it has practical deployment issues.****
>
> ** **
>
> [email-masked]> wrote:****
>

---

From: Karl Dubost <[email-masked]>
Date: Fri, 22 Jun 2012 16:54:50 -0400

Henrik,

Le 22 juin 2012 à 13:18, Henrik Frystyk Nielsen a écrit :
> We just published a blog [1] analyzing the performance implications of content optimizations such as bundling and minification on the performance of web pages.

Very nice !

> leverage optimizations throughout the stack to provide better user experience.

All the tests and benchmarks we see (even if made on real Web sites) are what I call "cold start testing". Basically every is set to 0, we download everything and look at the protocol design impacts on performances. 

But when we talk about better *user* experience, we might want to take into account the impact on performances when the user is doing multiple actions on the same set of resources. HTTP caching enters into play.

Is the performance gain that big? Are there things which can be done in the protocol to take into account longterm usage scenario?

-- 
Karl Dubost - http://dev.opera.com/
Developer Relations, Opera Software

---

From: Henrik Frystyk Nielsen <[email-masked]>
Date: Fri, 22 Jun 2012 21:04:14 +0000

Karl,

Indeed, the data here is from cold-start.

It would be nice to check various forms of caching as well but up front I would observe that bundling and minification again would likely help you as you only have to manage 2 cached resources and not N resources that may all have to get validated individually. Bundled/Minified content are just bigger documents.

Also, having the links way at the top of the HTML will also help as you would get to the links faster and as a result be able to start validating them with the cache faster.

Henrik

-----Original Message-----
From: Karl Dubost [mailto:[email-masked]] 
Sent: Friday, June 22, 2012 1:55 PM
To: Henrik Frystyk Nielsen
Cc: HTTP Working Group; Howard Dierking
Subject: Re: Performance implications of Bundling and Minification on HTTP/1.1

Henrik,

Very nice !

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 22 Jun 2012 21:45:23 +0000

In message <[email-masked]
rod.outlook.com>, Henrik Frystyk Nielsen writes:

Henrik, thanks for some very interesting data.

> 1)      It seems safe to assert that whatever we come up with in
> HTTP/2.0 should be significantly faster than what can be achieved
> with an optimized HTTP/1.1

I expect we can all agree on this, the fight will break out at 9pm,
when we try to define the meaning of "faster", "significantly" etc :-)

"faster" is not as unambiguous as most people think.

We have all seen the CNN exponential traffic plots from that day[1]
and if anything we must expect that such "event driven peaks" will
be even more summitous in the future.

In the generalized "#neilwebfail" scenario, "faster" for the
individual client means "when does something appear on my screen?"
whereas "faster" for the server means "how soon can I get something
to appear on many/most screens".

Addressing this in HTTP/2.0 with "Tough luck, buy more bandwidth &
servers", is not good enough.

Content-level design has a lot to do with this, as CNN showed with
their static HTML.

If I understand your data right, what you have done is server-neutral,
but HTTP/2.0 ideas like "default pipeline/parallism windows of 6"
will not be server neutral, and are almost guaranteed to make any
load problem at least six times worse for servers.

So, yes, I agree: HTTP/2.0 must be faster than HTTP1.1, but we need
to talk about "faster for who ?" and "faster how ?"

Poul-Henning

[1] Deliberate "avoid using T-word" obfuscation.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Henrik Frystyk Nielsen <[email-masked]>
Date: Fri, 22 Jun 2012 21:55:25 +0000

These are all great questions. This is exactly why we tried to get some numbers so that we can at least have something to agree/disagree over so that we can start to formulate goals for what faster means. It is also clear that performance gain for one individual cannot come at the expense of performance of other individuals -- we all have to play nice with a shared resource. One of the points were indeed that you *can* get higher performance with HTTP/1.1 while using significantly less resources in terms of bytes and connections needed to render a page.

I am not sure what you are referring to with "default pipeline/parallism windows of 6" -- I used two persistent TCP connections which I think is on the low end of what most browsers do today. 

Henrik

-----Original Message-----
From: Poul-Henning Kamp [mailto:[email-masked]] 
Sent: Friday, June 22, 2012 2:45 PM
To: Henrik Frystyk Nielsen
Cc: William Chan (陈智昌); HTTP Working Group; Howard Dierking
Subject: Re: Performance implications of Bundling and Minification on HTTP/1.1

> HTTP/2.0 should be significantly faster than what can be achieved with 
> an optimized HTTP/1.1

I expect we can all agree on this, the fight will break out at 9pm, when we try to define the meaning of "faster", "significantly" etc :-)

We have all seen the CNN exponential traffic plots from that day[1] and if anything we must expect that such "event driven peaks" will be even more summitous in the future.

In the generalized "#neilwebfail" scenario, "faster" for the individual client means "when does something appear on my screen?"
whereas "faster" for the server means "how soon can I get something to appear on many/most screens".

Addressing this in HTTP/2.0 with "Tough luck, buy more bandwidth & servers", is not good enough.

Content-level design has a lot to do with this, as CNN showed with their static HTML.

If I understand your data right, what you have done is server-neutral, but HTTP/2.0 ideas like "default pipeline/parallism windows of 6"
will not be server neutral, and are almost guaranteed to make any load problem at least six times worse for servers.

So, yes, I agree: HTTP/2.0 must be faster than HTTP1.1, but we need to talk about "faster for who ?" and "faster how ?"

Poul-Henning

---

From: Henrik Frystyk Nielsen <[email-masked]>
Date: Fri, 22 Jun 2012 22:01:31 +0000

Thanks! Actually, bundling, minification (and moving those links to the top of the HTML) together with compression are fairly straight forward mechanisms with relatively big bang for the buck. I don’t claim that they solve all problems – they clearly don’t – but for the simple case of downloading content to render a web page they can have a big impact.

Saying that you can magically fix performance in the underlying protocol without taking into account the content I think is a hard argument to make. For example, if the links to CSS or JS is several kBytes down in the HTML and is required to render the page then it simply takes time to get to that data. It would be interesting to see exactly that the impact would be using interleaving or multiplexing so that we can compare data.

SPDY’s trick to get around intercepting proxies is to use SSL but that can be done with HTTP just as well. The real question to ask is why these intercepting proxies are there in the first place. My sense (although I don’t claim to know all the nuances) is that when you are deploying in a public site you simply can’t get set up unless you use HTTP and it can’t be SSL. Public internet providers really want you to sign up and click some buttons and the only way to do that today is to sniff the traffic. That is a real problem that I think needs solving but it has little to do with the application protocol.

I don’t have data for SSL but that is something we could do.

Henrik

From: Roberto Peon [mailto:[email-masked]]
Sent: Friday, June 22, 2012 1:09 PM
To: Henrik Frystyk Nielsen
Cc: William Chan (陈智昌); HTTP Working Group; Howard Dierking
Subject: Re: Performance implications of Bundling and Minification on HTTP/1.1

Nice data!
On Fri, Jun 22, 2012 at 12:05 PM, Henrik Frystyk Nielsen <[email-masked]<mailto:[email-masked]>> wrote:
The main point is to establish a baseline for how well an optimized HTTP/1.1 implementation with optimized content can perform. Given such a baseline it is much easier to compare HTTP/2.0 ideas and evaluate the positives and/or negatives of alternative solutions. As such I am not making any normative claims per se but there indeed are two points that I think are important to observe:

I'd argue another point.
The amount of work necessary to optimize site performance for HTTP/1.1 today is large. The amount of knowledge necessary to do it properly is also large.
This is not the way it should be!

The protocol should make it easier to do things right, and it should help in the (extremely frequent and likely) case that the site designer gets it wrong in little ways.

2)      Without taking a broader view of performance that includes the content as an integral part you simply cannot meaningfully expect to get a fast system. In other words, there is no way that an application protocol can compensate for badly composed content. For example, if you put your links to CSS and JS deep down in your HTML or rely on 100 requests to be complete in order to render a page then no amount of protocol optimizations can effectively help you. Mechanisms such as bundling, minification, and compression can play a significant role here.
"...no amount of protocol optimizations can effectively help you."
Note that this isn't strictly true :)
Feeding back into the statement I made above, as an example: response reordering/interleaving, etc. can go pretty far in mitigating the damage done by suboptimal link ordering. It can't help in some pathological cases, true, but I'd guess that most sites don't present the pathological worst case...
A protocol which allows for bundling without requiring the site developer to deal with it increases the chance that a site's latency approaches closer to optimal.

But don't we already understand the root cause? Isn't it intercepting proxies?
Basically, that a number of proxies considered parts of the spec optional, and took shortcuts, and are deployed in such a fashion as to intercept and disrupt communications between parties that would otherwise be able to communicate without problem?

Deploying over a channel that does not have these intercepting proxies solves this problem. We know that we don't have this problem to any degree that matters for SPDY, for instance.
Is it worth generating a separate table for doing the various optimizations over HTTPS?
-=R

Thanks,

Henrik

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of William Chan (???)

Comments welcome!

Thanks,

Henrik

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 22 Jun 2012 22:05:51 +0000

I was referring to earlier discussions where an initial window of
six requests per TCP connection was proposed, based on the current
browsers "6 TCP connections" default param.

It is much easier and painless for a server to ignore SYN packets
than to tell every client "Yeah, I know the RFC says you can send
me 6 requests in parallel but..." for every SYN accepted.

---

From: Mike Belshe <[email-masked]>
Date: Fri, 22 Jun 2012 15:55:11 -0700

On Fri, Jun 22, 2012 at 10:18 AM, Henrik Frystyk Nielsen <
[email-masked]> wrote:

> stack to provide better user experience.
>

Thanks, Henrik.

One point this article proves to me is just why protocol features must be
mandatory.  I hope HTTP/2.0 learns from this data.

To illustrate - look at how important gzip was in the performance measured
here.  If we consider the maximum benefit in all of these test cases
(reduced PLT from 638 to 146ms) - 93% of that benefit is accomplished with
gzip alone (no pipelines, no bundles, no minification, no spdy, nothing!).

This is why so many protocol writers are adamantly against optional
features.   If gzip were mandatory in HTTP/1.1, 93% of the benefit of this
test would be achieved for everyone today.  Instead, enabled by optional
compression, we know from other research** that ~1/3 of all compressable
resources lack compression, and over SSL about half of compressable
resources lack compression.  We'd be able to stop testing the uncompressed
case for sure!

An interesting test would be to measure if pipelining, bundling, and
minification without gzip can outperform gzip alone.  (spoiler alert -
nope! :-)

Mike

** https://developers.google.com/speed/articles/web-metrics

---

From: "Roy T. Fielding" <[email-masked]>
Date: Fri, 22 Jun 2012 17:10:43 -0700

HTTP predates gzip and cookies. The notion that such a choice had anything to do with optional features is just silly. It is optional because it barely even worked in practice until 1999 and certainly wasn't feasible as a default until a few years after that as CPU capability increased.

....Roy

On Jun 22, 2012, at 3:55 PM, Mike Belshe <[email-masked]> wrote:

> 
> 
> On Fri, Jun 22, 2012 at 10:18 AM, Henrik Frystyk Nielsen <[email-masked]> wrote:
> One point this article proves to me is just why protocol features must be mandatory.  I hope HTTP/2.0 learns from this data.
> 
> To illustrate - look at how important gzip was in the performance measured here.  If we consider the maximum benefit in all of these test cases (reduced PLT from 638 to 146ms) - 93% of that benefit is accomplished with gzip alone (no pipelines, no bundles, no minification, no spdy, nothing!).
> 
> This is why so many protocol writers are adamantly against optional features.   If gzip were mandatory in HTTP/1.1, 93% of the benefit of this test would be achieved for everyone today.  Instead, enabled by optional compression, we know from other research** that ~1/3 of all compressable resources lack compression, and over SSL about half of compressable resources lack compression.  We'd be able to stop testing the uncompressed case for sure!
> 
> An interesting test would be to measure if pipelining, bundling, and minification without gzip can outperform gzip alone.  (spoiler alert - nope! :-)
> 
> Mike
>

---

From: "Martin Nilsson" <[email-masked]>
Date: Sat, 23 Jun 2012 02:13:51 +0200

> resources lack compression.  We'd be able to stop testing the  

Also, some HTTP requests are rewritten by proxies and anti-virus  
applications to disable compression, so compression will be used even less.

> minification without gzip can outperform gzip alone.  (spoiler alert -

A set of tests with different permutations of these techniques would  
indeed be interesting to see.

Minification is something that can only be done by the content provider,  
in the general case, as it requires knowledge about future use of a  
resource. For that reason I don't think you can use minification when  
creating a HTTP/1.1 baseline to compare HTTP/2.0 against.

Bundling is problematic in two ways. If you dynamically bundle the  
resources needed, you will end up with non-reusable resources, which works  
great in cold testing as Karl points out, but not over a longer session.  
If you statically bundle the resources, you benefit in a longer session,  
but cold testing is going to perform worse than without bundling. A push  
mechanism would solve this in the protocol layer, but has its own set of  
issues to solve.

Reference reordering is, like minification, destructive to the content and  
can not be done safely in the protocol layer. Both push and hinting solves  
this (again, with its issues to solve).

/Martin Nilsson

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

---

From: Mike Belshe <[email-masked]>
Date: Fri, 22 Jun 2012 17:42:35 -0700

On Fri, Jun 22, 2012 at 5:10 PM, Roy T. Fielding <[email-masked]> wrote:

> HTTP predates gzip and cookies. The notion that such a choice had anything
> to do with optional features is just silly. It is optional because it
> barely even worked in practice until 1999 and certainly wasn't feasible as
> a default until a few years after that as CPU capability increased.
>

I'm sorry if this offends - it wasn't intended as a dig against HTTP.

The point is that the #1 performance impact in this test was compression,
which optional in HTTP, and that optional features are not used as widely
as they could be.

Mike

>> Not only does optimizing the content save bytes but it also has savings
>> in the number of requests and responses that need to be processed as well
>> as faster render times due to being able to retrieve the HTML, CSS, and JS
>> up front. In the test evaluated, the speedup was from 638 ms
>> (uncompressed, unbundled, unminified, and not pipelined) down to 146 ms for
>> the equivalent compressed, bundled, minified, and pipelined content.
>> However, by just looking at the data necessary to lay out the page (HTML,
>> CSS, and JS but not images), the time went from 631 ms to 126 ms with the
>> images being finalized within the remaining timespan from 126 to 146 ms.*
>> ***
>>
>> ** **
>>

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 23 Jun 2012 11:21:03 +1000

Hi Henrik,

Thanks very much for doing this. 

I'd like to see much more data like this; without it, it's too easy to make decisions based upon popularity and momentum rather than reasoned goals, especially when engineers clamour for Shiny New Things. Mike and the Google guys have done some fantastic work in this area, but we need multiple sources and repeatable tests.

It'd be nice to see this in a more permanent form; e.g., as an I-D, or copied to the WG wiki. Blog entries have a habit of disappearing. (Mike et al, if you'd like to copy your data over as well, please get in touch; we could start a bit of a repository there).

Also, I'd *really* like to see more data about higher-latency connections. Personally, the majority of Web sites I use on a daily basis have a ping time in excess of 200ms, and that's going to be the experience of many across the globe (not just those in .au and .nz).

Cheers,

--
Mark Nottingham   http://www.mnot.net/

---

From: Henrik Frystyk Nielsen <[email-masked]>
Date: Sat, 23 Jun 2012 01:29:27 +0000

That sounds fantastic!  I'd be happy to put it into ID form or the blog  -- might have to redo the graphics ;)

As for more data in high-latency environments -- I completely agree -- it's on the list and I hope to provide some data next week. I would also like to see data through proxies.

Btw, I hope to make the data/source available soon so that people can try it out as well.

Henrik

-----Original Message-----
From: Mark Nottingham [mailto:[email-masked]] 
Sent: Friday, June 22, 2012 18:21
To: Henrik Frystyk Nielsen
Cc: HTTP Working Group; Howard Dierking
Subject: Re: Performance implications of Bundling and Minification on HTTP/1.1

Hi Henrik,

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 23 Jun 2012 11:35:09 +1000

On 23/06/2012, at 10:42 AM, Mike Belshe wrote:

> The point is that the #1 performance impact in this test was compression, which optional in HTTP, and that optional features are not used as widely as they could be.

It's actually very widely deployed among browsers.

The issue isn't that it's optional, the issue is that because it's negotiated, some intermediaries disable the negotiation to make application of policy easier. 

We can have a discussion about whether or not we want to prioritise performance over these use cases, but let's not kid ourselves that it's as simple as "optional = bad." Sometimes there are good reasons to make something optional.

Cheers,

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 23 Jun 2012 10:01:47 +0200

> We can have a discussion about whether or not we want to prioritise performance over these use cases, but let's not kid ourselves that it's as simple as "optional = bad." Sometimes there are good r…

Indeed! I'd prefer having my traffic pass through a non-compressing proxy
than through a proxy which has trouble compressing chunked-encoded data.
And I'm still regularly observing some of them lying around from time to
time. It's not just a matter of applying a policy easier, it's also a
matter of getting the code do the right thing and reliably.

The most features we'll make mandatory, the less reliable components we'll
get.

Also, while I'm not fundamentally against having servers compress data by
default, I'm against having them do it *systematically* and against using
gzip *only*. Gzip is now old and new algos are much more powerful at the
expense of CPU power on the compressing size. But caching compressed
objects sometimes allows a server to prepare contents that are delivered
much faster to capable consumers. That's why large files are commonly
delivered in tar.xz archives nowadays instead of tar.gz.

I think that the main reason for servers not always compressing contents
right now is that the underlying HTTP/1 protocol doesn't make this easy.
In my opinion, we should design HTTP/2 so that it doesn't get tricky
anymore to send compressed contents, and that way we'll encourage servers
to deliver compressed contents (whether they do it on the fly or select a
pre-compressed alternate file).

Regards,
Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 23 Jun 2012 08:29:47 +0000

In message <[email-masked]>, "Martin Nilsson" 

>applications to disable compression, so compression will be used even less.

... and they have a good reason to disable gzip:  These devices sit at the
"choke-points" in the network and see very high if not the highest
HTTP-traffic densities of any devices in the HTTP domain.

There are two subcases, and they are quite different:

"Incoming"
----------

Typically a load-balancer which needs only to inspect the "Host:"
header and/or the URI in the request and the status code of the
reponse.

These are the devices I call "HTTP routers", and they are where
all the traffic bottlenecks when the entire world tries to find
out what happened in Dallas.

HTTP/2.0 should serialize (at least) these crucial fields without
gzip and preferably in a way that makes it very easy and cheap to
find them.

"Outgoing"
----------

Almost always content-scanning, and since there are legitimate
use cases (Prison inmates for instance) we have to accept this
role as legitimate[1].

A legitimate argument exists, that censors should pay the cost
of censorship.  If we accept that, these boxes should not be
able to force clients/servers to forego compression.

---

From: Roberto Peon <[email-masked]>
Date: Sat, 23 Jun 2012 15:08:45 -0700

Negotiated options can work IFF the options are always available and work
reliably.

Proxies as have been deployed for http have made many things that otherwise
should theoretically work not reliably work. An example is gzip
compression. Another example was the amazing amount of problems when
attempting to use a different compression algorithm (even when both client
and server spoke it).

Proxy writers can (and often do) have different motivations from the client
and content-provider.
When this happens, optional features/negotiated features are harmful to the
clients and content providers since clients and content-providers are
effectively held hostage by the proxy provider who has all the power.

I am unabashedly against hostage taking. :)
-=R
On Jun 22, 2012 6:37 PM, "Mark Nottingham" <[email-masked]> wrote:

> > HTTP predates gzip and cookies. The notion that such a choice had
> anything to do with optional features is just silly. It is optional because
> it barely even worked in practice until 1999 and certainly wasn't feasible
> as a default until a few years after that as CPU capability increased.
> >
> > The point is that the #1 performance impact in this test was
> compression, which optional in HTTP, and that optional features are not
> used as widely as they could be.
>
> The issue isn't that it's optional, the issue is that because it's
> negotiated, some intermediaries disable the negotiation to make application
> of policy easier.
>
> We can have a discussion about whether or not we want to prioritise
> performance over these use cases, but let's not kid ourselves that it's as
> simple as "optional = bad." Sometimes there are good reasons to make
> something optional.
>
> Cheers,
>
> --

---

From: Roberto Peon <[email-masked]>
Date: Sat, 23 Jun 2012 15:15:25 -0700

There are certainly valid reasons to interpose mitm. I've submitted an I-D
for explicit proxies. I'd be happy if we had this conversation in that
context.

Here, however we are talking about what exists today. I'm perfectly happy
to require the prisions to do zip and so marginally increase their costs if
it would increase the likelihood that many more users are ensured they can
benefit from it.

The sum total benefit is hopefully clearly higher when 1% of a population
does not affect the remaining 99%'s ability to use a feature.

-=R

---

From: "Martin Nilsson" <[email-masked]>
Date: Sun, 24 Jun 2012 00:27:47 +0200

> There are two subcases, and they are quite different:

This very quickly moves into the area of policy and not technology, but  
those decisions must be made as well. These are the real world cases I can  
think of right now

- Reading request headers (loadbalancers, logging, url filtering)
- Reading the request payload (legal intercept)
- Modifying request headers (header enrichment)
- Replacing the request (access point sign up, interstitial)
- Reading the response (content based filtering, legal intercept)
- Modifying the response (injected content)
- Replacing the response (block page)

Many of them could perhaps better be solved in a different place than HTTP  
(like access point sign up through DHCP negotiated URL, MSISDN header  
enrichment through IP options etc). For the remaining ones, is the answer  
that they are supported by HTTP/2.0, or is "use HTTP/1.1 instead" an  
acceptable fallback?

/Martin Nilsson

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sun, 24 Jun 2012 06:44:59 +0000

In message <[email-masked]>, "Martin Nilsson" 
writes:

> or is "use HTTP/1.1 instead" an  acceptable fallback?

I think it would be an acceptable fallback, but I don't think it
would be a wise fallback.

First of all, it's not like we can prevent people from doing those
actions on HTTP/2.0 anyway, so whatever our feelings might be, people
will do what they have/want to do anyway.

Second, it would prevent us from ever getting rid of HTTP/1.x

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Mon, 25 Jun 2012 00:37:27 +0200

* Karl Dubost wrote:
>All the tests and benchmarks we see (even if made on real Web sites) are
>what I call "cold start testing". Basically every is set to 0, we
>download everything and look at the protocol design impacts on
>performances. 
>
>But when we talk about better *user* experience, we might want to take
>into account the impact on performances when the user is doing multiple
>actions on the same set of resources. HTTP caching enters into play.

Well, if the site does not load quickly on first visit, you might not
develop a user base that cares, while, when you are an established ser-
vice, users notice delays less due to familiarity, so these are really
two aspects of user experience, and one would look at the "cold start"
independently of the experience for staying visitors if new users are
a concern.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Patrick McManus <[email-masked]>
Date: Mon, 25 Jun 2012 13:48:16 -0400

Henirk, 

Thanks for the blog post! I've got a couple questions that came to
mind.. I'll just put the gzip one in this post.

" our data shows that compression works better on a bundled document due
to the larger size and hence the opportunity for better compression
ratio. 

Compressing 7 JS individually 161178 bytes
Compressing the minified JS bundle 137817 bytes

[.. redacted more or less the same for css ..]
"

what about compressing the concat'd 7 js files and therefore extending
the compression context across resource boundaries? Also, what was
window size?

SPDY actually dropped a data-compression option in the transport
(instead it uses resource level compression and makes support of that
mandatory), but if there is reason to suggest significantly better rates
doing it that way rather than per-resource then that's probably an
interesting property to move along as we discuss http/2 (independent of
whether it is implicit in a bundle or layered on as part of the
transport). You could even imagine a small array of content-specific
contexts (one for headers, one for js, one for html, whatever)..

---

From: Henrik Frystyk Nielsen <[email-masked]>
Date: Mon, 25 Jun 2012 20:12:22 +0000

Patrick,

We used the default static file compression in IIS -- I don't know off the top of my head what the parameter settings are but I can find out. Are you thinking of externalizing the compression dictionary and reuse it across resources? When I looked at compression many years ago we did look at external dictionaries but if I remember correctly it is only identified by a 32 bit CRC so it is not very unique. The challenge here is that this effectively makes compression a hop-by-hop mechanism which can have a much higher cost if you have intermediaries in the chain.

Thanks,

Henrik

[1] http://www.iis.net/ConfigReference/system.webServer/httpCompression

-----Original Message-----
From: Patrick McManus [mailto:[email-masked]] 
Sent: Monday, June 25, 2012 10:48 AM
To: [email-masked]
Subject: Re: Performance implications of Bundling and Minification on HTTP/1.1

Henirk, 

Thanks for the blog post! I've got a couple questions that came to mind.. I'll just put the gzip one in this post.

" our data shows that compression works better on a bundled document due to the larger size and hence the opportunity for better compression ratio. 

Compressing 7 JS individually 161178 bytes Compressing the minified JS bundle 137817 bytes

[.. redacted more or less the same for css ..] "

what about compressing the concat'd 7 js files and therefore extending the compression context across resource boundaries? Also, what was window size?

SPDY actually dropped a data-compression option in the transport (instead it uses resource level compression and makes support of that mandatory), but if there is reason to suggest significantly better rates doing it that way rather than per-resource then that's probably an interesting property to move along as we discuss http/2 (independent of whether it is implicit in a bundle or layered on as part of the transport). You could even imagine a small array of content-specific contexts (one for headers, one for js, one for html, whatever)..

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 26 Jun 2012 00:00:18 +0200

>[1] http://blogs.msdn.com/b/henrikn/archive/2012/06/17/performance-implications-of-bundling-and-minification-on-http.aspx

"We only looked at CSS and JS coming from the same DNS domain (i.e. for
digg.com we looked at anything under *.digg.com." As far as I can tell,
http://digg.com/ does not load any scripts from digg.com, most digg-
specific scripts seem to come from *.diggstatic.com instead. The same
goes for many of the other examples, http://www.bbc.co.uk/ loads from
*.bbcimg.co.uk and *.static.bbci.co.uk, http://www.huffingtonpost.com/
loads from *.huffpost.com (and a single one from *.huffingtonpost.com),
and so on. At least for the BBC I am quite sure this has not changed in
recent weeks, so it seems a different measure was used than the above.

I am not sure why the bundles are several kilobytes bigger than the sum
of the individual sizes, it would seem to take about one byte to conca-
tenate them (e.g., bloomberg.com has "js size (kb)" 408 and "js bundle
size (kb)" 410, which would make for a kilobyte of difference even if
there are rounding problems). If headers were counted, the size should
go down, as the bundle comes with less header overhead.

"In addition to removing white space, minification typically shortens
variable names and other identifiers and removes pieces that are not
used." I do not think changing variable names is "typical" for minifi-
cation. I am not sure what to look at due to the first issue above, but
the minification savings seem unreasonable; as far as comment removal
and white space normalization goes, the sites seem to mostly use mini-
fied scripts already, but they might not try to shorten variable names,
though the referenced blog posting explaining minification also doesn't
really mention that as an optimization. For CSS the results are closer,
but for, say, http://www.bbc.co.uk/ I get more like 13% minification
savings than 26% as in the blog posting (results vary with browsers and
other things, there are, for instance, "conditional comments" that hide
some references to external content from some browsers).

So overall the trend seems right, but this isn't very reproducible. It's
also somewhat disappointing to see that, for instance, digg.com loads an
old version of jQuery, and various jQuery extensions, "jScrollPane",
"AJAX Upload", "In-Field Label", and so on with full comments and unmi-
nified when loading the front page. Last year I wrote a tool that strips
several kilobytes http://bjoern.hoehrmann.de/pngwolf/ off google.com but
I guess collectively we are not quite there where we'd care about that.

(My general impression is that these kinds of optimizations are underde-
veloped because they are quite hard to automate, with HTTP compression
early on there had been many bugs in browsers and intermediaries and
caches in both, there were concerns about drive space costs and CPU con-
sumption, nowadays, to pick the example of PNG optimization, there is a
lack of "cooperation", tools that outperform my `pngwolf` tend to be
scripts that use multiple optimization tools, `pngwolf` selects the best
scanline filters, kzip/pngout has the best Deflate implementation as far
as size is concerned, various tools compete in the area of selecting be-
tween palette images and RGB/A images for losless recompression, there
are tools to optimize Huffman tables that work very good but happen to
be unfree and closed source so I can't link them into `pngwolf` and so
on. And, more importantly, the tools go largely unused. Google's mod_-
pagespeed for instance includes OptiPNG, but runs it at settings that
favour compression throughput over compression ratio, mainly, I suspect,
because amortizing the cost in form of some cache is hard to implement,
with proper cache invalidation and whatever else might be needed. In my
case of `pngwolf`, I was surprised even how trivial it was to come up
with a scanline filter selection that outperforms the only known one,
the one offered in the specification, and how even expensive commercial
tools fail to use even trivial heuristics. Overall, there is a lack of
tools to tell when your site is much slower/bigger/... than necessary,
a problem that starts with HTML and CSS being hard to parse properly.)
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Henrik Frystyk Nielsen <[email-masked]>
Date: Mon, 25 Jun 2012 23:03:57 +0000

Bjoern,

You are correct regarding the wording of where the scripts were from. What I should have said is that we looked at content that was from the same "site". This was to clarify that we were not trying to bundle scripts from many different sites as this likely would not be practical. The goal was to use a version of bundling and minification that could be applied immediately.

The compressed size only includes the content size, not any HTTP overhead. The way we got the data was serving the files through IIS with Fiddler as the client sending two requests: one without accept-encoding and one with accept-encoding. We then got the size as the content-lengths of the responses. That way, we were guaranteed to use the same compression algorithm in all cases.

As for comparing the minification vs. just bundling (concatenation), here is the data if we just concatenate the CSS content from huffingtonpost.com and then compressed them as one document. The size difference is still noticeable (75230 - 69803 = 5427 bytes):

Compressing 6 CSS individually:               77495 bytes
Compressing the minified CSS bundle:    69803 bytes
Compression the 6 concatenated files:  75230 bytes

We'll be giving out the raw data in a day or two so that you can have a look for yourself.

As to your general point, these are relatively high-traffic public sites that clearly have gone through some level of optimization yet still leaves a lot of room open for more to be done. While I do agree that tooling support is evolving I don't think it takes away from the larger point that we have to take both protocol and content into consideration when considering performance and that there are obvious things that can help a great deal simply using existing infrastructure.

Henrik

-----Original Message-----
From: Bjoern Hoehrmann [mailto:[email-masked]] 
Sent: Monday, June 25, 2012 3:00 PM
To: Henrik Frystyk Nielsen
Cc: HTTP Working Group; Howard Dierking
Subject: Re: Performance implications of Bundling and Minification on HTTP/1.1

* Henrik Frystyk Nielsen wrote:
>[1] 
>http://blogs.msdn.com/b/henrikn/archive/2012/06/17/performance-implicat
>ions-of-bundling-and-minification-on-http.aspx

"We only looked at CSS and JS coming from the same DNS domain (i.e. for digg.com we looked at anything under *.digg.com." As far as I can tell, http://digg.com/ does not load any scripts from digg.com, most digg- specific scripts seem to come from *.diggstatic.com instead. The same goes for many of the other examples, http://www.bbc.co.uk/ loads from *.bbcimg.co.uk and *.static.bbci.co.uk, http://www.huffingtonpost.com/ loads from *.huffpost.com (and a single one from *.huffingtonpost.com), and so on. At least for the BBC I am quite sure this has not changed in recent weeks, so it seems a different measure was used than the above.

I am not sure why the bundles are several kilobytes bigger than the sum of the individual sizes, it would seem to take about one byte to conca- tenate them (e.g., bloomberg.com has "js size (kb)" 408 and "js bundle size (kb)" 410, which would make for a kilobyte of difference even if there are rounding problems). If headers were counted, the size should go down, as the bundle comes with less header overhead.

"In addition to removing white space, minification typically shortens variable names and other identifiers and removes pieces that are not used." I do not think changing variable names is "typical" for minifi- cation. I am not sure what to look at due to the first issue above, but the minification savings seem unreasonable; as far as comment removal and white space normalization goes, the sites seem to mostly use mini- fied scripts already, but they might not try to shorten variable names, though the referenced blog posting explaining minification also doesn't really mention that as an optimization. For CSS the results are closer, but for, say, http://www.bbc.co.uk/ I get more like 13% minification savings than 26% as in the blog posting (results vary with browsers and other things, there are, for instance, "conditional comments" that hide some references to external content from some browsers).

So overall the trend seems right, but this isn't very reproducible. It's also somewhat disappointing to see that, for instance, digg.com loads an old version of jQuery, and various jQuery extensions, "jScrollPane", "AJAX Upload", "In-Field Label", and so on with full comments and unmi- nified when loading the front page. Last year I wrote a tool that strips several kilobytes http://bjoern.hoehrmann.de/pngwolf/ off google.com but I guess collectively we are not quite there where we'd care about that.

(My general impression is that these kinds of optimizations are underde- veloped because they are quite hard to automate, with HTTP compression early on there had been many bugs in browsers and intermediaries and caches in both, there were concerns about drive space costs and CPU con- sumption, nowadays, to pick the example of PNG optimization, there is a lack of "cooperation", tools that outperform my `pngwolf` tend to be scripts that use multiple optimization tools, `pngwolf` selects the best scanline filters, kzip/pngout has the best Deflate implementation as far as size is concerned, various tools compete in the area of selecting be- tween palette images and RGB/A images for losless recompression, there are tools to optimize Huffman tables that work very good but happen to be unfree and closed source so I can't link them into `pngwolf` and so on. And, more importantly, the tools go largely unused. Google's mod_- pagespeed for instance includes OptiPNG, but runs it at settings that favour compression throughput over compression ratio, mainly, I suspect, because amortizing the cost in form of some cache is hard to implement, with proper cache invalidation and whatever else might be needed. In my case of `pngwolf`, I was surprised even how trivial it was to come up with a scanline filter selection that outperforms the only known one, the one offered in the specification, and how even expensive commercial tools fail to use even trivial heuristics. Overall, there is a lack of tools to tell when your site is much slower/bigger/... than necessary, a problem that starts with HTML and CSS being hard to parse properly.)
--
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 26 Jun 2012 13:33:37 +1000

> The protocol should make it easier to do things right, and it should help in the (extremely frequent and likely) case that the site designer gets it wrong in little ways.

This is definitely an area that should be discussed. I've heard a few people express skepticism about multiplexing overall, because it requires the server to prioritise what's in the pipe, which in turn requires greater knowledge (and probably a bucketload of heuristics).

Right now those heuristics are applied to how browsers make requests, but at least the data is applied in the same place it's most usefully sourced, and of course there are fewer browser implementations than there are server deployments (which is potentially the level that this kind of tuning would need to take place for multiplexing).

Discuss :)

---

From: William Chan (陈智昌) <[email-masked]>
Date: Mon, 25 Jun 2012 21:20:06 -0700

> > This is not the way it should be!

+1

> >
> > The protocol should make it easier to do things right, and it should
> help in the (extremely frequent and likely) case that the site designer
> gets it wrong in little ways.
>
> This is definitely an area that should be discussed. I've heard a few
> people express skepticism about multiplexing overall, because it requires
> the server to prioritise what's in the pipe, which in turn requires greater
> knowledge (and probably a bucketload of heuristics).

> Right now those heuristics are applied to how browsers make requests, but
> at least the data is applied in the same place it's most usefully sourced,
> and of course there are fewer browser implementations than there are server
> deployments (which is potentially the level that this kind of tuning would
> need to take place for multiplexing).
>
> Discuss :)
>

I'm a bit surprised people express much skepticism. Basic heuristics can
get you a long way. For example, simply prioritizing html/js/css over other
resources like images does a lot. Remember that the majority of
implementations, including Chromium's and Google's, are still in the early
stages of optimization.

---

From: Henrik Frystyk Nielsen <[email-masked]>
Date: Tue, 26 Jun 2012 05:02:17 +0000

I do agree that there are serious questions as to what exactly multiplexing can and cannot solve. The purpose of multiplexing multiple sub-streams over a single reliable stream is to get a higher degree of responsiveness of each of the individual sub-streams. That is, the premise is that by interleaving the sub-streams, it is possible to make progress on each of the streams individually. However, this necessarily requires that each sub-stream gets a relatively small window in which to transmit data. If this window gets too large then only the active sub-stream will make progress and the other sub-streams will get blocked. Getting the window size and hence the degree of responsiveness right without penalizing throughput slowing down all the sub-streams requires a fair amount of information about network conditions, the relative importance of the sub-streams, and what gives the user the best experience for the given data. Not to mention that this has to happen between two arbitrary implementations.

The argument that the protocol magically can solve this problem for any content I simply don't think holds true so the question then becomes whether a more complex protocol inherently will make it easier to get the behavior right for any particular content. I am not saying that this is an impossible task but I think it is fair to say that this remains to be seen; and also whether it is to a degree that truly makes it worthwhile.

At the same time I think it is reasonable to point out that the use of optimizations such as bundling, minification, and compression are evolving and they can have as big if not bigger impact on the user experience than anything we can do at the protocol level. If there are things we can do to help these optimizations work better in practice then that would be great.

Roberto mentions that there are lots of challenges doing this today -- could we get these on the table and quantify them?

Henrik

-----Original Message-----
From: Mark Nottingham [mailto:[email-masked]] 
Sent: Monday, June 25, 2012 20:34
To: Roberto Peon
Cc: Henrik Frystyk Nielsen; William Chan (陈智昌); HTTP Working Group; Howard Dierking
Subject: Re: Performance implications of Bundling and Minification on HTTP/1.1

Discuss :)

---

From: Robert Mattson <[email-masked]>
Date: Tue, 26 Jun 2012 15:29:57 +1000 (EST)

Hi Henrik,

I wanted to mention that I've done some simulation with approximately 200
web pages over symmetric and asymmetric networks using response
multiplexing with server side and client side order prioritisation.

This work may be relevant in the deliberations mentioned below.

The experiments and results are available in my academic thesis and begin
on page 168 of
http://www.mattson.com.au/robert/fulltext/MattsonPhdThesis.pdf

or

http://www.mattson.com.au/robert/fulltext/MattsonPhdThesis/

Sincerely,
Rob Mattson, MACS (CT), MIEEE

On Tue, 26 Jun 2012, Henrik Frystyk Nielsen wrote:

> Cc: Henrik Frystyk Nielsen; William Chan (???); HTTP Working Group; Howard Dierking

---

From: Henrik Frystyk Nielsen <[email-masked]>
Date: Tue, 26 Jun 2012 05:36:31 +0000

Rob,

This looks like great stuff -- looking forward to study it closer.

Henrik

-----Original Message-----
From: Robert Mattson [mailto:[email-masked]] 
Sent: Monday, June 25, 2012 22:30
To: Henrik Frystyk Nielsen
Cc: Mark Nottingham; William Chan (陈智昌); [email-masked]
Subject: RE: Performance implications of Bundling and Minification on HTTP/1.1

Hi Henrik,

I wanted to mention that I've done some simulation with approximately 200 web pages over symmetric and asymmetric networks using response multiplexing with server side and client side order prioritisation.

The experiments and results are available in my academic thesis and begin on page 168 of http://www.mattson.com.au/robert/fulltext/MattsonPhdThesis.pdf

or

Sincerely,

On Tue, 26 Jun 2012, Henrik Frystyk Nielsen wrote:

> I do agree that there are serious questions as to what exactly 
> multiplexing can and cannot solve. The purpose of multiplexing 
> multiple sub-streams over a single reliable stream is to get a higher 
> degree of responsiveness of each of the individual sub-streams. That 
> is, the premise is that by interleaving the sub-streams, it is 
> possible to make progress on each of the streams individually. 
> However, this necessarily requires that each sub-stream gets a 
> relatively small window in which to transmit data. If this window gets 
> too large then only the active sub-stream will make progress and the 
> other sub-streams will get blocked. Getting the window size and hence 
> the degree of responsiveness right without penalizing throughput 
> slowing down all the sub-streams requires a fair amount of information 
> about network conditions, the relative importance of the sub-streams, 
> and what gives the user the best experience for the given data. Not to 
> mention that this has to happen between two arb
 itrary implementations.

---

From: William Chan (陈智昌) <[email-masked]>
Date: Mon, 25 Jun 2012 22:49:19 -0700

On Mon, Jun 25, 2012 at 10:02 PM, Henrik Frystyk Nielsen <
[email-masked]> wrote:

> multiplexing can and cannot solve. The purpose of multiplexing multiple
> sub-streams over a single reliable stream is to get a higher degree of
> responsiveness of each of the individual sub-streams. That is, the premise
> is that by interleaving the sub-streams, it is possible to make progress on
> each of the streams individually. However, this necessarily requires that
> each sub-stream gets a relatively small window in which to transmit data.
> If this window gets too large then only the active sub-stream will make
> progress and the other sub-streams will get blocked. Getting the window
> size and hence the degree of responsiveness right without penalizing
> throughput slowing down all the sub-streams requires a fair amount of
> information about network conditions, the relative importance of the
> sub-streams, and what gives the user the best experience for the given
> data. Not to mention that this has to happen between two arbitrary
> implementations.
>

Just to doublecheck, are you asserting that you need a complicated frame
sizing implementation in order to get good performance out of multiplexing?
In Chromium, we don't do much complicated for SPDY frame sizes. We clamp to
2*MSS. Please refer to:
http://code.google.com/searchframe#OAMlx_jo-ck/src/net/spdy/spdy_session.h&l=41
http://code.google.com/searchframe#OAMlx_jo-ck/src/net/spdy/spdy_session.cc&exact_package=chromium&l=634

I am assuming when you say window you mean frame, and you're not actually
trying to use flow control windows to achieve responsiveness. Pardon me if
I am misinterpreting your terminology. If you actually are talking about
flow control windows, then we ought to have a separate discussion thread,
as that's a whole other can of worms you're opening up :)

> The argument that the protocol magically can solve this problem for any
> content I simply don't think holds true so the question then becomes
> whether a more complex protocol inherently will make it easier to get the
> behavior right for any particular content. I am not saying that this is an
> impossible task but I think it is fair to say that this remains to be seen;
> and also whether it is to a degree that truly makes it worthwhile.
>

Can you clarify which problem you're referring to? Do you mean frame
sizing? If so, I think I've shown the simplicity of Chromium's SPDY
algorithm above. If you meant prioritization of substreams, here's our
algorithm:
http://code.google.com/searchframe#OAMlx_jo-ck/src/content/browser/renderer_host/resource_dispatcher_host_impl.cc&exact_package=chromium&type=cs&l=206.
As you can see, it's a simple switch statement. Can we do better? Very
possibly!

>
> At the same time I think it is reasonable to point out that the use of
> optimizations such as bundling, minification, and compression are evolving
> and they can have as big if not bigger impact on the user experience than
> anything we can do at the protocol level. If there are things we can do to
> help these optimizations work better in practice then that would be great.
>

I agree that application level improvements have a bigger impact than
protocol level improvements. But I think that it'd be great for the
protocol to eliminate deficiencies for which application developers have to
work around (e.g. domain sharding). There are so many web performance
guidelines that developers need to keep in mind...the less that they have
to do to work around protocol deficiencies, the better.

> Roberto mentions that there are lots of challenges doing this today --
> could we get these on the table and quantify them?
>
> Henrik
>

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 26 Jun 2012 16:13:39 +1000

On 26/06/2012, at 3:49 PM, William Chan (陈智昌) wrote:

> I agree that application level improvements have a bigger impact than protocol level improvements. But I think that it'd be great for the protocol to eliminate deficiencies for which application developers have to work around (e.g. domain sharding). There are so many web performance guidelines that developers need to keep in mind...the less that they have to do to work around protocol deficiencies, the better.

Agreed on all points. The situation that we should avoid, I think, are protocol-level gains that *require* lots of tweaking / expertise on the server side. It's fine to accommodate them, of course, as long as they don't hurt the common case.

Regards,

---

From: Roberto Peon <[email-masked]>
Date: Mon, 25 Jun 2012 23:28:57 -0700

I hope we're all in agreement on that :)

-=R
On Jun 25, 2012 11:13 PM, "Mark Nottingham" <[email-masked]> wrote:

> Agreed on all points. The situation that we should avoid, I think, are
> protocol-level gains that *require* lots of tweaking / expertise on the
> server side. It's fine to accommodate them, of course, as long as they
> don't hurt the common case.
>
> Regards,
>
> --

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 26 Jun 2012 06:41:14 +0000

In message <[email-masked]>, Mark Nottingham wri
tes:

>protocol to eliminate deficiencies for which application developers have 
>to work around (e.g. domain sharding). 

One of the biggest deficiencies in HTTP/1.x is the lack of a "session"
concept, which lead to the Cookie-Hack with all its pessimizing and
privacy-violating issues.

Even if used right (one cryptosigned session-identifier, all actual
data stored serverside) it has the sideeffect of blanket disabling
caching.

HTTP/2.0 should do better.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 26 Jun 2012 17:29:55 +1000

> HTTP/2.0 should do better.

I have a hard time seeing how that's in the current scope of discussion.

Part of the reason that this sort of thing is out of scope is that it's a big rat hole, and the more we take on redesigning, the less likely we'll be able to ship something and have it deployed.

New features can and should be layered onto HTTP independently of the version identifier, which is for identifying the wire protocol, not fine-grained feature negotiation. 

See <http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.1>: 

> The protocol versioning policy is intended to allow the sender to indicate the format of a message and its capacity for understanding further HTTP communication, rather than the features obtained via that communication. No change is made to the version number for the addition of message components which do not affect communication behavior or which only add to extensible field values. The <minor> number is incremented when the changes made to the protocol add features which do not change the general message parsing algorithm, but which may add to the message semantics and imply additional capabilities of the sender.

... and RFC2145.

Cheers,

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 26 Jun 2012 07:36:56 +0000

In message <[email-masked]>, Mark Nottingham wri
tes:

>Part of the reason that this sort of thing is out of scope is that it's
>a big rat hole, and the more we take on redesigning, the less likely
>we'll be able to ship something and have it deployed.

Rule #1 in design: Always make it easier for people to do the right thing.

HTTP/2.0 should provide a session-concept which makes it easier to
do the right thing.

If we do that right, cookies will die in their own time.

---

From: Patrick McManus <[email-masked]>
Date: Tue, 26 Jun 2012 08:29:06 -0400

> Patrick,

Thanks Henrik,

I was just trying to tease out the byte savings of minification, vs
bundling, vs compression in all of their combinations which the blog
post didn't get into.. 

> Are you thinking of externalizing the compression dictionary and reuse it across resources? 

It's a possibility to consider - that's all. SPDY already does it with
headers. I could imagine having a small set of dictionaries for
different classes of content (headers, markup, etc..).. I'm not
advocating that here, just trying to scrounge info from your data
points :)

---

From: Henrik Frystyk Nielsen <[email-masked]>
Date: Tue, 26 Jun 2012 14:49:35 +0000

Patrick,

Bjoern asked a similar question and so I computed the "just concatenation plus compression" numbers listed in [1] but included here for convenience:

The size difference is still noticeable (75230 - 69803 = 5427 bytes):

Henrik

[1] http://lists.w3.org/Archives/Public/ietf-http-wg/2012AprJun/0785.html

-----Original Message-----
From: Patrick McManus [mailto:[email-masked]] 
Sent: Tuesday, June 26, 2012 05:29
To: Henrik Frystyk Nielsen
Cc: [email-masked]
Subject: RE: Performance implications of Bundling and Minification on HTTP/1.1

Thanks Henrik,

I was just trying to tease out the byte savings of minification, vs bundling, vs compression in all of their combinations which the blog post didn't get into.. 

> Are you thinking of externalizing the compression dictionary and reuse it across resources? 

It's a possibility to consider - that's all. SPDY already does it with headers. I could imagine having a small set of dictionaries for different classes of content (headers, markup, etc..).. I'm not advocating that here, just trying to scrounge info from your data points :)