Subject: Experiences with HTTP/2 server push

From: Tom Bergan <[email-masked]>
Date: Thu, 04 Aug 2016 00:21:00 +0000

Hi all,

Our team has been experimenting with H2 server push at Google for a few months. We found that it takes a surprising amount of careful reasoning to understand why your web page is or isn't seeing better performance with H2 push. We also encountered a lack of good documentation: How should one go about using H2 push? What are the best practices? We tried to distill our experiences into five "rules of thumb" that are described in this doc:
https://docs.google.com/document/d/1K0NykTXBbbbTlv60t5MyJvXjqKGsCVNYHyLEXIxYMv0 <https://docs.google.com/document/d/1K0NykTXBbbbTlv60t5MyJvXjqKGsCVNYHyLEXIxYMv0>

The doc is a little long, but the first two pages give a decent tl;dr. I suspect the ideas and conclusions will be "obvious" to many people on this mailing list, at least in hindsight. Hopefully other folks interested in H2 server push will find this useful. Let us know if you have any comments.

-Tom, Simon, and Michael

---

From: Martin Thomson <[email-masked]>
Date: Fri, 5 Aug 2016 22:28:24 +1000

I didn't read through this in detail, but there is a fairly big error
here when it comes to the description of using priorities.  You say:

> One way to implement this is for the server to update its HTTP/2 priority tree, **then send PRIORITY frames to the client that make A the exclusive parent of C** and C the exclusive parent of D. This is an attractive implementation because the server can continue using the HTTP/2 priority tree to order requests C, D, and B.

The server can't send PRIORITY frames in this way.  Or at least, that
won't have the effect you think it does.

The server can (and should) just send as it sees fit, using as input
its own knowledge and the priority that the client has provided.  If
the server sends PRIORITY, that is to affect client processing (hint:
that's not going to happen here). Given that the space that you are
examining is a problem for server-to-client transmission only, the
server expressing priority is pointless.

On 4 August 2016 at 10:21, Tom Bergan <[email-masked]> wrote:
> Hi all,
>
> Our team has been experimenting with H2 server push at Google for a few
> months. We found that it takes a surprising amount of careful reasoning to
> understand why your web page is or isn't seeing better performance with H2
> push. We also encountered a lack of good documentation: How should one go
> about using H2 push? What are the best practices? We tried to distill our
> experiences into five "rules of thumb" that are described in this doc:
> https://docs.google.com/document/d/1K0NykTXBbbbTlv60t5MyJvXjqKGsCVNYHyLEXIxYMv0
>
> The doc is a little long, but the first two pages give a decent tl;dr. I
> suspect the ideas and conclusions will be "obvious" to many people on this
> mailing list, at least in hindsight. Hopefully other folks interested in H2
> server push will find this useful. Let us know if you have any comments.
>
> -Tom, Simon, and Michael

---

From: Tom Bergan <[email-masked]>
Date: Fri, 5 Aug 2016 08:20:35 -0700

On Fri, Aug 5, 2016 at 5:28 AM, Martin Thomson <[email-masked]>
wrote:

> > One way to implement this is for the server to update its HTTP/2
> priority tree, **then send PRIORITY frames to the client that make A the
> exclusive parent of C** and C the exclusive parent of D. This is an
> attractive implementation because the server can continue using the HTTP/2
> priority tree to order requests C, D, and B.
>

Yes, we agree with you. The point of that section is that the server should
*not* send PRIORITY frames like that. Here's the next paragraph:

> However, this is fundamentally racey: if both ends (client and server)
update the priority tree concurrently, it can easily become out-of-sync.
For this reason, we advocate not mutating the priority tree on the server.

Let's say the server wants to prioritize a subset of streams differently
than the priorities specified by the client, or differently from the
default priorities. How should it actually implement this? The simplest
implementation is to mutate the H2 priority tree directly. This makes the
H2 priority tree the single prioritization data structure in the server.
It's also attractive because H2 priorities can be communicated to lower
layers like QUIC
<https://tools.ietf.org/html/draft-hamilton-early-deployment-quic-00#section-9>.
We are aware of a few servers that update the priority tree like this,
e.g., see Apache's h2_session_set_prio
<https://github.com/icing/mod_h2/blob/master/mod_http2/h2_session.c#L1245>.

However, if the server does this, it has a problem: the H2 priority tree is
a shared data structure. If it makes a change, its local copy of the data
structure can be out-of-sync relative to the client's copy. A future
PRIORITY frame from the client may have a different meaning than intended
if the server has already changed its tree locally. The sentence you quoted
describes the reactions of a naive server to this problem: Maybe I can keep
the client's tree in sync by sending a PRIORITY frame? (Sorry for not
making this more clear.) Of course, this doesn't actually solve the
problem, since the server's PRIORITY frames could race with the client's.
(Note that we're not aware of any servers that actually do this; we were
just hoping to prevent any from trying.)

RFC 7540 talks about another kind of race: removing closed streams from the
tree. The solution proposed by the RFC is to keep closed streams in the
tree as long as possible. The RFC does not discuss this other kind of race
-- reprioritizing streams on the server -- and this seems like something
servers are very interested in doing. AFAIK, no one has really studied the
impacts of this race nor provided guidance about how to avoid it. We don't
have any great solutions, either, we just wanted to raise the problem to be
sure that server implementors are aware of it.

---

From: Alcides Viamontes E <[email-masked]>
Date: Fri, 5 Aug 2016 21:07:56 +0200

> However, if the server does this, it has a problem: the H2 priority tree
> is a shared data structure. If it makes a change, its local copy of the
> data structure can be out-of-sync relative to the client's copy. A future

Hi. Great work over there.

If browser and server are a bit far apart re-prioritization may arrive a
bit too late to the server to be effective. Our solution for both  the race
conditions and the RTT problem  is having our server learn the priorities
and dependencies, build a delivery plan once and use it many times. In that
sense, priorities and dependencies on the HTTP/2 spec as it is today is
good enough for us. And the implementation complexity is about the same as
implementing on-the-flight re-prioritization.

>
> RFC 7540 talks about another kind of race: removing closed streams from
> the tree. The solution proposed by the RFC is to keep closed streams in the

This thing with closed streams takes a bit of getting used to. We have to
be very careful to discard as much information as possible about closed
streams as early as possible, but still keep some around for a little while
to know which stream references from the browser are valid. Since we are
not handling priority frames online, the amount of information we have to
save is relatively small ("was this stream ever used?"), but if we were
following the letter of the spec this would be a very worrying issue.

>> > push.
Oh, but it is a lot of fun :-)

In our experience as well the biggest performance killer of HTTP/2 Push is
TCP slow start and the fact that push promises are bulky. Send many of them
and an ACK round-trip will be needed.

However, HTTP/2 Push *is* useful at other times as well. For example, if
the server is using cache digests via cookies and it positively knows that
the browser doesn't have a script referenced at the bottom of the page,
like this:

       ...something
      <script src="/jquery.js?vh=3fhhwq"></script>,

it can pause the HTML stream a little bit before "<script src", send a push
promise for "/jquery.js?vh=3fhhwq", and resume sending the HTML document.
Chances are that the TCP window is bigger by then.

Also notice a related scenario, which is a counter-pattern from the times
of HTTP/1.1: instead of making a big HTML file including all  parts of a
page, use (the closest thing to) HTML imports. If elements of a page that
seldom change like navigation bar and visual footer are made imports, then
they can be cached and traffic to the server can be reduced. Nobody does
that because of latency. Using  HTTP/2 Push in the way described before, it
becomes possible at no performance cost.

Looked that way, HTTP/2 Push is a big deal for web components. And it is
not far fetched, we are planning to release these features for ShimmerCat
1.7. The only thing we require from browser  is that they check if there is
a push promise for a resource strictly -- but as late as possible -- before
starting a fetch.

The same can be done with hierarchies of scripts, although we will have to
wait a bit for people to stop making big .js blobs....

-- 
Alcides Viamontes E.
Chief Executive Officer, Zunzun AB
(+46) 722294542
(www.shimmercat.com is a property of Zunzun AB)

---

From: Amos Jeffries <[email-masked]>
Date: Sat, 6 Aug 2016 09:04:50 +1200

> wait a bit for people to stop making big .js blobs....

Any particular reason to wait?

Implementing it early allows demonstration that its faster not to use
blobs. Without that visible reason to change its unlikely that blobs
will disappear.

Amos

---

From: Tom Bergan <[email-masked]>
Date: Mon, 8 Aug 2016 11:40:10 -0700

Thanks all for the feedback.

> implementing on-the-flight re-prioritization.

If you can come up with a complete delivery plan on the server, that is
definitely ideal. There is some academic work that tries to do this (see
Klotski: https://www.usenix.org/node/189011). I also liked your
"Coordinated Image Loading" article as an example of why server-defined
priorities can be a good idea. I am curious though -- how do you deal with
cases where the page is heavily dynamic and some of the important content
may be fetched by XHRs? Do you expect that the full dependency information
is available a priori, before the main response is served to the client?

> Oh, but it is a lot of fun :-)

It is for us too :-), but I imagine many devs would find it frustrating.
Hence our attempt to try to distill our experiences into a "best practices"
doc.

> and an ACK round-trip will be needed.

Interesting point about needing ACK round-trips just to send the push
promises. We hadn't run across that problem specifically. Is this because
you're sending many push promises? Is there some reason why hpack cannot
compress a sequence of push promises?

If you've released any data about HTTP/2 Push performance in ShimmerCat,
I'd be interested to read it. I did notice one article on your site,
although that only dealt with one page with a fixed network connection:
https://www.shimmercat.com/en/info/articles/performance/

On Sun, Aug 7, 2016 at 7:02 PM, Martin Thomson <[email-masked]>
wrote:
>
> I think that you still misunderstand.  The priority tree is the
> client's data.  The server cannot mutate it because it doesn't own it.
> As you say, if the server makes changes, the state is ruined.

I'm not sure what I misunderstand? It sounds like we're saying the same
thing: the server should not mutate the priority tree if it wants to
continue using priority information from the client. (The counter-point is
Alcides' example, where the server uses its own delivery plan and ignores
the client's PRIORITY frames entirely.)

Note that no client I'm aware of will do anything with the server's
> PRIORITY frames, let alone adjust its own priority tree.
>

To be clear:

* As I mentioned above, I'm not aware of any server that tries to send
PRIORITY frames to the client. That discussion was intended to dissuade a
naive server from attempting to do so. AFAIK, servers that do mutate the
priority tree just mutate the server's in-memory representation of the
tree, e.g., I believe this is what Apache does with h2_session_set_prio,
which is applied to pushed streams (more info here
<https://httpd.apache.org/docs/trunk/mod/mod_http2.html#h2pushpriority>).

* Apache is technically not doing anything illegal. Servers are free to do
anything they want with priority information. (From Section 5.3: "priority
for a stream is input to a prioritization process. It does not guarantee
any particular processing or transmission order for the stream relative to
any other stream ... priority is therefore only a suggestion".) The problem
comes when servers want to mix their own priorities (e.g., for pushes) with
the client's priorities. It is tempting to do this by mutating a single
data structure. RFC 7540 doesn't discuss this temptation at all, which is
why we discussed it in our document.

---

From: Alcides Viamontes E <[email-masked]>
Date: Mon, 8 Aug 2016 22:04:35 +0200

> is available a priori, before the main response is served to the client?

Thanks for the link! I will take a closer look.

About dynamic content, so far we have focused on xhr content which is
fetched on initial page load. From version 1.0 up to 1.4 we have included a
learning mode that introduces small, artificial random pauses between DATA
frames of an HTTP/2 stream. We also make the  frames purposely small and
variable, say arond 512 bytes in length. Then we record the time when each
frame is delivered (as well as our server can know from the blanket of
abstractions that the OS provides). During initial page load multiple
streams are being downloaded in parallel, and the pauses help to improve
(up to a point, before the law of large numbers kicks in) overall
randomness in the process.  Therefore, on repeated loads of the same page,
frames from different streams which are almost always delivered very close
in time point out to some correlation between their streams. We sort the
correlations on the order of possible causality, and we postulate that
there is a dependency between the two streams. This works well even for xhr
content, where the browser doesn't provide explicit dependency information.

In simpler words, we use statistics to try to infer dependencies from the
browser's behavior. The only downside of our technique is that it uses too
much computing power to be able to run under 10 seconds on a developer's
laptop, because each page of the site which is significantly different to
the others needs to be fetched several times by a typical user agent, e.g.,
a browser. It would work great as a cloud service though. And it is also
good enough to run as part of CI workflows.

>> In our experience as well the biggest performance killer of HTTP/2 Push
>> is TCP slow start and the fact that push promises are bulky. Send many of
>> them and an ACK round-trip will be needed.
>>
>
Yes, in our early prototypes we just wanted to see how far the technique
would take us, so we grabbed a bloated HTML template and tried to push all
of it in the order our algorithms were spitting out ;-) .

> https://www.shimmercat.com/en/info/articles/performance/
What would you consider an interesting, standard measure for this case? We
are mainly interested on reducing the impact of latency on page load time.
So we tend to measure how the page load time decreases  for a client  with
latency around 120 ms. That's easy to standardize, but the other variable
is the website itself and that one is harder to standardize. In our
consultancy projects we tend to set a 30 to 50% improvement in load time
and time-to-start-reading over baseline as an achievable target goal, but
each project is quite unique :-( .

---

From: Martin Thomson <[email-masked]>
Date: Mon, 8 Aug 2016 12:02:14 +1000

On 6 August 2016 at 01:20, Tom Bergan <[email-masked]> wrote:
>> update the priority tree concurrently, it can easily become out-of-sync. For
>> this reason, we advocate not mutating the priority tree on the server.

I think that you still misunderstand.  The priority tree is the
client's data.  The server cannot mutate it because it doesn't own it.
As you say, if the server makes changes, the state is ruined.  Note
that no client I'm aware of will do anything with the server's
PRIORITY frames, let alone adjust its own priority tree.

> We are aware of a few servers that update the priority tree like
> this, e.g., see Apache's h2_session_set_prio.

Stefan, is this right?  See above.

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 15 Aug 2016 06:15:20 +0900

> As you say, if the server makes changes, the state is ruined.

I agree with Martin that a server should never send a PRIORITY frame.

OTOH let me note that a server can also send priority information as
part of a PUSH_PROMISE frame. This way, the priority tree does not get
ruined.

Ideally, I think clients should send PRIORITY frames when it finds out
how the content of a pushed stream is used, so that a server (that
consider clients to have better understanding of how the resources
should be prioritized) can respect the updated tree to prioritize the
pushed streams.

> Stefan, is this right?  See above.

In case of H2O, we prioritize pushes of certain media types, but that
is done out of the HTTP/2 prioritization tree. I think that is the way
to go.

-- 
Kazuho Oku

---

From: Martin Thomson <[email-masked]>
Date: Mon, 15 Aug 2016 11:14:30 +1000

> part of a PUSH_PROMISE frame. This way, the priority tree does not get

I don't think that makes sense.  If the client processes the
PUSH_PROMISE and immediately reprioritizes the push, then the PRIORITY
frame that appears afterwards will be exactly as meaningless or
destructive as anything else.

> should be prioritized) can respect the updated tree to prioritize the

This is good advice.

> is done out of the HTTP/2 prioritization tree. I think that is the way

I couldn't parse this.  Do you mean that you ignore the client's
express priorities, or work within the client's priorities?

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 15 Aug 2016 10:58:30 +0900

> destructive as anything else.

Sorry, I agree with you on this point. My misunderstanding was that it
was possible to embed priority information within PUSH_PROMISE frame
like the way that can be done with a HEADER frame.

> express priorities, or work within the client's priorities?

Generally speaking, a server should respect the priority tree
constructed by the client if it considers the client's knowledge of
how the responses should be ordered superior to the knowledge of the
server. OTOH, if a server considers itself to have better knowledge on
how to prioritize the streams, then it should ignore client-driven
prioritization.

Since the client cannot accurately prioritize a pushed stream until
client observes how the resource is used, server-driven prioritization
of pushed streams is necessary until client updates the priority of a
pushed stream.

So our approach is to create a separate tiny pool of streams _above_
the HTTP/2 prioritization tree, and initially register the pushed
streams with certain media types (e.g. CSS, JavaScript) to the tiny
pool. The server does not send PRIORITY frames for the pushed streams.

This means that until the client sends a PRIORITY frame to update the
precedence of a pushed stream, pushed CSS and JS will be sent before
any other responses. Once the server receives a PRIORITY frame, then
the pushed streams will be prioritized as specified by the client.

-- 
Kazuho Oku

---

From: Matthew Kerwin <[email-masked]>
Date: Mon, 15 Aug 2016 12:07:34 +1000

> destructive as anything else.
​I think the key here is "part of"... unlike HEADERS, PUSH_PROMISE doesn't
have priority fields. If it did, this would be true. I'm still not sure
what it would mean, though.

Cheers
-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: Tom Bergan <[email-masked]>
Date: Wed, 24 Aug 2016 10:20:03 -0700

> the pushed streams will be prioritized as specified by the client.

Belated response: I think this is the right approach and I think Apache
should switch to an implementation more like this.

---

From: Kazuho Oku <[email-masked]>
Date: Thu, 1 Sep 2016 13:57:15 +0900

Hi,

> https://docs.google.com/document/d/1K0NykTXBbbbTlv60t5MyJvXjqKGsCVNYHyLEXIxYMv0

Sorry for the late response. I have recently had a chance to discuss
the details of the document.

It is a great summary of what the issues related to push are. On the
other hand, it makes me sad that the fifth rule states "Consider
Preload Instead of Push".

While I understand that not using push could be a short term answer on
some deployments, I think we need to discuss how we should fix the
issues (detailed as first four rules) to make the Web faster using
push in the long term.

And my take against the issues raised in the document are as follows:

"Rule 1: Push Enough to Fill Idle Network Time, and No More":

Technically, HoB caused by a low-priority stream is an issue irrelevant to push.

Consider the case where a large HTML that loads a CSS is sent over the
wire. In a typical implementation, the server will pass a block of
HTML much larger than INITCWND to the TCP stack before recognizing the
request for CSS. So the client would need to wait for multiple RTTs
before starting to receive the CSS.

It is only the case that such inversion of priority (a
lower-prioritized response blocking a higher one) might happen more
frequently when push is used.

That said, as discussed at the workshop, it is possible to implement a
HTTP/2 server that does not get affected by HoB between the different
streams (see https://github.com/HTTPWorkshop/workshop2016/blob/master/talks/tcpprog.pdf).

I would suggest that regardless of whether or not push is used, server
implementors should consider adopting such approach to minimize the
impact of HoB.

It should also be noted that with QUIC such HoB would not be an issue
since there would no longer be a send buffer within the kernel.

"Rule 2: Push Resources in the Right Order"

My take is that the issue can / should be solved by clients sending
PRIORITY frames for pushed resources when they observe how the
resources are used, and that until then servers should schedule the
pushed streams separately from the client-driven prioritization tree
(built by using the PRIORITY frames).

Please refer to the discussion in the other thread for details:
https://lists.w3.org/Archives/Public/ietf-http-wg/2016JulSep/0453.html

"Rule 3: Don’t Push Resources the Client has Already Cached"

As the document states, the short-time solution would be for a server
to track what it has pushed to the client, and ultimately (and
hopefully) the standardization of cache-digests would solve the issue.

"Rule 4: Use the Right Cookies"

I think this is a very interesting point.

As a server implementor, I have always dreamt of cancelling a push
after sending a PUSH_PROMISE. In case a resource we want to push
exists on a dedicate cache that cannot be reached synchronously from
the HTTP/2 server, the server needs to send PUSH_PROMISE without the
guarantee that it would be able to push a valid response.

It would be great if we could have an error code that can be sent
using RST_STREAM to notify the client that it should discard the
PUSH_PROMISE being sent, and issue a request by itself.

With such an error code, it would be possible for a server to
speculatively push a resource by fetching it on behalf of the client
(without cookies etc.) and if it fails, then let the client fetch the
resource (with all the appropriate credentials / conditions).

-- 
Kazuho Oku