Subject: expected substantial and measurable improvements in WG charter

From: Larry Masinter <[email-masked]>
Date: Wed, 11 Mar 2015 05:17:43 +0000

The working group seems to have come to the rough consensus that “HTTP/2 is good enough to ship”. And that’s fine.

That’s not the same as consensus as to whether HTTP/2 meets the “It is expected that” list in the charter. In particular, the charter expects HTTP/2 to substantially and measurably improve end-user perceived latency in most cases.  But it seems there mostly agreement (not consensus to the contrary) that to substantially improve end-user perceived latency in “most cases”, you not only need HTTP/2 but also a good deal of mainly undisclosed magic. And that quite a few sites will see worse performance if they merely replace HTTP/1.1 with HTTP/2 (with the necessary shift to TLS).

 Perhaps the tight group developing  HTTP/2 represent interests with knowledge of how to rearrange their servers to actually see substantial and measurable performance improvements, but the means are not clear, and might even be proprietary. It is  a disservice to  ship HTTP/2 without clearly documenting what you have to do to actually get the substantial improvements expected.

Now, you might argue that no work meets all of the expectations placed on it, and perhaps HTTP/2 isn’t quite as good as expected, but that’s how it always works. But the words in the charter need to have some purpose beyond a wish list:   Meet these expectations or explain when and why they cannot be met.

The working group should focus on the task of documenting clearly the deployment steps needed to get the expected benefit. Without doing so, HTTP/2 isn’t complete.

And the working group should do this before taking on other topics proposed in the draft agenda — all of which seem to be out of scope for the current charter.

I see no point in slowing the publication  of HTTP/2 as Proposed Standard, I’m just calling for full disclosure.

Larry
—
http://larry.masinter.net

---

From: Yoav Nir <[email-masked]>
Date: Wed, 11 Mar 2015 16:10:45 +0200

Hi, Larry.

> On Mar 11, 2015, at 7:17 AM, Larry Masinter <[email-masked]> wrote:
> 

Do we have this information to disclose?  As you say, it’s likely that the tight group knows how to tweak Google, Twitter and Facebook servers to improve end-user experience with HTTP/2.  I don’t know if what they know applies to banking websites, news sites, forums, and storefronts. It may or it may not. They’re not talking about any “undisclosed magic”, and we can’t make them.

Give it a year, and we’ll read about people outside the small group attempting to deploy HTTP/2. Maybe they’ll love the improved user experience. Maybe they’ll measure it and conclude that it didn’t help and didn’t hurt. Maybe they’ll find that it makes things worse and revert. Then we might be able to make a better when and why document.

Yoav

---

From: Matthew Kerwin <[email-masked]>
Date: Thu, 12 Mar 2015 11:30:23 +1000

On 12 March 2015 at 00:10, Yoav Nir <[email-masked]> wrote:

>
> Give it a year, and we’ll read about people outside the small group
> attempting to deploy HTTP/2. Maybe they’ll love the improved user
> experience. Maybe they’ll measure it and conclude that it didn’t help and
> didn’t hurt. Maybe they’ll find that it makes things worse and revert. Then
> we might be able to make a better when and why document.
>
>
Is that an argument for making HTTP/2 an experimental standard? (I'm not
whipping a dead horse, but genuinely questioning the meaning of
"experimental" vs "proposed" standards.)

Is it well and truly time for us, as well, to start working on the
apocryphal "how to use HTTP/2" document that's been mentioned from time to
time over the years? I'd start, but I don't know where to begin (since I'm
partly the target audience, wanting to know why I need to use TLS to work
with Chrome and Firefox, etc.).

-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: Roberto Peon <[email-masked]>
Date: Wed, 11 Mar 2015 18:50:29 -0700

Larry--
HTTP/2 substantially and measurably improves end-user perceived latency.
It doesn't in all cases, however, when the SHOULD about using one
connection is followed, as it should be (not following this will eventually
make things worse).

The most important of these is when the transport makes having 6
connections outperform one link with the same peers.
Fixing this (the transport) is *the* thing to do.
For congestion algorithms like cubic you need to have appropriate initcwnd
settings (between 10 and 32), and you need to react appropriately to loss
(e.g. as having K (or 6) connections would).
If you're using different congestion avoidance algorithms, the tweaks to
make the transport behave sanely are likely to be different in the details,
and the same in principle.

We've said this a number of times, so I don't know why there is any
impression that there is something proprietary here.

-=R

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 12 Mar 2015 14:04:54 +1100

Hi Larry,

> On 11 Mar 2015, at 4:17 pm, Larry Masinter <[email-masked]> wrote:
> 

"Seems to" is an understatement -- it's done and dusted, and the time to dispute that has truly past. Not saying that you're doing so, but it does appear that some folks might not understand the process.

> That’s not the same as consensus as to whether HTTP/2 meets the “It is expected that” list in the charter. In particular, the charter expects HTTP/2 to substantially and measurably improve end-user…

That's not the agreement that I see at all. Most people with operational experience of the protocol have said that one can expect a 5-15% end-user perceived performance benefit "out of the box" with a reasonable implementation, and substantially more with some tweaking (e.g., removing spriting/inlining/sharding/concatenation, adjusting prioritisation algorithms and thinking about server push). Those numbers don't hold for every site on every network, but that's the nature of the Internet.

>  Perhaps the tight group developing  HTTP/2 represent interests with knowledge of how to rearrange their servers to actually see substantial and measurable performance improvements, but the means are not clear, and might even be proprietary.

That sounds a lot like FUD, Larry.

> It is  a disservice to ship HTTP/2 without clearly documenting what you have to do to actually get the substantial improvements expected.
> 

No one has made that argument. 

As you and I have discussed before, there's a substantial community around Web Performance, and I fully expect it to step up and develop best practices, tooling, etc. around H2 in time. The Velocity conferences over the next few years should be interesting.

h2 is only recently available in commonly available browsers, and server-side support is still in progress for the platforms that most people use (if folks haven't seen it yet, you may be interested in <https://icing.github.io/mod_h2/>). Since most ops and perf people are by nature hands-on, most have only recently had a chance to get familiar with the practicalities of the protocol.

WRT Google and others who have existing deployment experience with SPDY and early H2 - in the discussions I've seen, they've been pretty open about the "tricks" they use to get decent perf out of the protocol; much of it has to do with TCP and TLS tuning (see <http://istlsfastyet.com>). We could perhaps do more to collect h2-specific information together in one place -- but I'm not sure that the right place to put it is in an RFC. 

> And the working group should do this before taking on other topics proposed in the draft agenda — all of which seem to be out of scope for the current charter.

Our charter explicitly allows us to "define additional extensions to HTTP." I don't see how you get to "out of scope" from here.

> I see no point in slowing the publication  of HTTP/2 as Proposed Standard, I’m just calling for full disclosure.

That's an interesting phrase to use.

Regards,

--
Mark Nottingham   https://www.mnot.net/

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 12 Mar 2015 07:34:12 +0100

Hi Mark, Larry,

On Thu, Mar 12, 2015 at 02:04:54PM +1100, Mark Nottingham wrote:
> > That???s not the same as consensus as to whether HTTP/2 meets the ???It is
> > expected that??? list in the charter. In particular, the charter expects
> > HTTP/2 to substantially and measurably improve end-user perceived latency
> > in most cases.  But it seems there mostly agreement (not consensus to the
> > contrary) that to substantially improve end-user perceived latency in
> > ???most cases???, you not only need HTTP/2 but also a good deal of mainly
> > undisclosed magic. And that quite a few sites will see worse performance if
> > they merely replace HTTP/1.1 with HTTP/2 (with the necessary shift to TLS).
> 
> That's not the agreement that I see at all. Most people with operational
> experience of the protocol have said that one can expect a 5-15% end-user
> perceived performance benefit "out of the box" with a reasonable
> implementation, and substantially more with some tweaking (e.g., removing
> spriting/inlining/sharding/concatenation, adjusting prioritisation algorithms
> and thinking about server push). Those numbers don't hold for every site on
> every network, but that's the nature of the Internet.

I think there is no reason to worry much on the server side. In practice,
it will be for every web site just like when admins try to enable HTTPS :
some will not observe any cost change because the application's resources
usage is so high that TLS is very cheap, some will observe a significant
degradation (resource usage or extra requests due to proxies not caching
contents anymore) and will have to decide whether it's for them or not
(just like when switching to HTTPS), some will see a significant
improvement and will want to evangelize the new protocol all around them.

Given what we've seen with sites switching to HTTPS, I think that most
deployments will experience a small difference in either direction and
will consider that it's worth leaving it enabled at least to embrace
upcoming technology improvements in various products (eg: server push).

So I think we should wait and observe, even before trying to fix anything.
We all know there's a lot of room for improvement in this protocol and
possibly we've all missed much more important parts that non-wg users
will loudly complain about, so let's calmly wait for this feedback before
declaring any difficulty, failure or whatever.

Willy