Subject: Implementation Notes on Server Push

From: Patrick McManus <[email-masked]>
Date: Tue, 14 May 2013 15:14:59 -0400

I'm wrapping up a Firefox implementation of spdy/3 server push and wanted
to provide some ietf level feedback because that mechanism is largely
intact in the current http2 work in progress draft.

   - 304's seem to be a common cause of wasted pushed streams. I would see
   servers respond with a 304 for index.html and still push 200 responses for
   a.css and b.js when in a non-push world that would have been either 1 or 3
   304's. Maybe we should have a rule that you can only push to an
   assoc-stream of 2xx ?
   - There is language in the spec that if the client resets a stream that
   it implicitly resets any associated streams too. That was complicated to
   implement and pretty much broke my stream state model internally - while
   researching it it appears that mod_spdy and chrome don't implement it at
   all. The spec has an editorial note about removing that feature and I favor
   that removal.
   - I found flow control for pushed streams immensely helpful. It lets the
   client bound how much data can be pushed before there is a local GET
   matched up with that. Relatedly, I changed my default SETTINGS window size
   from a ~infinite value to be a smaller push-apropos value and then
   pipelined a window update with each odd SYN_STREAM to make pulled streams
   ~infinite again while preserving the smaller limit for push and this worked
   fine with all existing servers to my pleasant surprise. That seems to mean
   at least the spdy/3 windowing mechanism is simple enough for people to get
   right.
   - As with other parts of the spec, I was faced in some implementations
   with some very large data frames (90+KB) from servers when testing. Such
   frames are impervious to CANCEL or any mechanism we might use to change
   priorities driven by the client :(..  HTTP2 is doing the right thing by
   creating a smallish max frame size.

Obviously this is implementation experience with testing and isn't real
operational experience yet.. that will come.

-P

---

From: William Chan (陈智昌) <[email-masked]>
Date: Tue, 14 May 2013 16:34:01 -0300

Thanks for the feedback, this is great to hear.

On Tue, May 14, 2013 at 4:14 PM, Patrick McManus <[email-masked]>wrote:

>    - 304's seem to be a common cause of wasted pushed streams. I would
>    see servers respond with a 304 for index.html and still push 200 responses
>    for a.css and b.js when in a non-push world that would have been either 1
>    or 3 304's. Maybe we should have a rule that you can only push to an
>    assoc-stream of 2xx ?
>
>
I'm hesitant to mandate this behavior. I think it's best to say that
servers should figure this out. Server push is an advanced feature and I
think it'll take us a little while to develop best practices around its
use. This sounds great for a best practice recommendation.

>
>    - There is language in the spec that if the client resets a stream
>    that it implicitly resets any associated streams too. That was complicated
>    to implement and pretty much broke my stream state model internally - while
> The interesting thing to note is our two major server implementations of
push (mod_spdy & Jetty) both did not implement the implicit reset. But
which side bears the responsibility for explicitly resetting the stream?
Smart servers will reset the push streams still in order to save sending
unnecessary data, which is kinda what the implicit reset would suggest
they'd do, except without explicitly sending the RST_STREAM. None of our
existing implementations were that smart though (I doubt any initial server
push implementation would be very well tuned, so I'm not dinging the
implementers. I mean, our Chromium implementation admittedly sucks and we
find rough edges all the time). If the server doesn't explicitly do a
RST_STREAM, then I think smart clients will probably send RST_STREAMs when
the parent stream is canceled.

>
>    - I found flow control for pushed streams immensely helpful. It lets
>    the client bound how much data can be pushed before there is a local GET
> I assume you're testing against mod_spdy or Jetty. If you see this, we
should fix those implementations. I think the developers will be very
welcoming of feedback.

---

From: Roberto Peon <[email-masked]>
Date: Tue, 14 May 2013 12:25:56 -0700

I don't favor any MUST in the spec about requiring 304s in the first
scenario, but informing people that this is silly would be good :)
Stupid servers will be suboptimal, yes, but if a server has a strong reason
to believe that the next thing the client will be requesting is something
which will be stale, pushing the resource is completely reasonable.
-=R

---

From: Martin Thomson <[email-masked]>
Date: Tue, 14 May 2013 16:57:32 -0700

Great feedback.

On 14 May 2013 12:14, Patrick McManus <[email-masked]> wrote:
> 304's seem to be a common cause of wasted pushed streams. I would see
> 304's. Maybe we should have a rule that you can only push to an assoc-stream
> of 2xx ?

I don't know if outright prohibition is necessary.  It might be that
a.css has changed, but index.html hasn't.  But I consider that
unlikely in the general case.  A word of caution seems appropriate.

> There is language in the spec that if the client resets a stream that it
> implicitly resets any associated streams too. That was complicated to

I added that text, so I tend to agree. :)

> I found flow control for pushed streams immensely helpful. It lets the
> client bound how much data can be pushed before there is a local GET matched
> up with that. Relatedly, I changed my default SETTINGS window size from a
> ~infinite value to be a smaller push-apropos value and then pipelined a
> window update with each odd SYN_STREAM to make pulled streams ~infinite
> again while preserving the smaller limit for push and this worked fine with
> all existing servers to my pleasant surprise. That seems to mean at least
> the spdy/3 windowing mechanism is simple enough for people to get right.

That's interesting.  Do you believe that it would be worthwhile
splitting the initial window size setting in two?  One for streams I
initiate, one for streams you initiate?  That would save you a window
update on every new stream.

> As with other parts of the spec, I was faced in some implementations with
> some very large data frames (90+KB) from servers when testing. Such frames
> are impervious to CANCEL or any mechanism we might use to change priorities
> driven by the client :(..  HTTP2 is doing the right thing by creating a
> smallish max frame size.

Small is good, but how small is small enough?

---

From: Roberto Peon <[email-masked]>
Date: Tue, 14 May 2013 17:08:10 -0700

inline

On Tue, May 14, 2013 at 4:57 PM, Martin Thomson <[email-masked]>wrote:

> assoc-stream

I agree it should be removed. If we need a mechanism to close many streams,
we can add a new opcode type instead of some implicit requirement.

> matched
As usual: It depends (TM), but so long as the max frame size, etc. is small
enough that the mechanisms to deal with it exist, changing it becomes as
painless as changing a constant or config param.

-=R

---

From: James M Snell <[email-masked]>
Date: Tue, 14 May 2013 18:24:29 -0700

On Tue, May 14, 2013 at 5:08 PM, Roberto Peon <[email-masked]> wrote:
>[snip]

If I recall correctly, everyone seemed to be nodding in agreement on this one.

>> update on every new stream.

"Splitting the initial window" seems wrong to me. Of course, I don't
have a better suggestion at the moment, but if I did I'm not certain
it would be this ;-)

[snip]

- James

---

From: Martin Thomson <[email-masked]>
Date: Wed, 15 May 2013 13:36:05 -0700

> have a better suggestion at the moment, but if I did I'm not certain

Sorry, I wasn't clear enough.  That would be crazy-talk.  I meant to
split the initial window size configuration items.  Rather than "my
receive window starts at X", it will be "my receive window for
odd-numbered streams is X but for even-numbered stream it is Y."

---

From: Eliot Lear <[email-masked]>
Date: Wed, 15 May 2013 22:44:16 +0200

> scenario, but informing people that this is silly would be good :)

Typically you only use a MUST on interoperability or security, and not
for performance (unless that turns somehow into a security issue, which
I don't see here).

Eliot

---

From: William Chan (陈智昌) <[email-masked]>
Date: Wed, 15 May 2013 17:48:04 -0300

>    at least the spdy/3 windowing mechanism is simple enough for people to get
Given the amount of discussion here, I figured I should chime in on this
point and ask a question. Are you doing this for memory bounding purposes?
Or is this for prioritization purposes?

As I've said before, I think we should not attempt to leverage flow control
mechanisms for prioritization. The client should advise the server as best
as possible the appropriate prioritization, and then trust the server to
serve responses (including pushed streams) in the best ordering.

---

From: Patrick McManus <[email-masked]>
Date: Wed, 15 May 2013 17:18:41 -0400

I think this makes sense. Alternatively we could just add it as 4 bytes to
the (bi-directional) SYN_STREAM.

Even at its current 16 byte overhead using window_update its not a big deal
afaict, though separate settings for things that are logically quite
separate is more efficient and a good idea. I mostly mentioned it because I
was impressed that I didn't find a server unhappy with using a
window_update to grow the window to a value larger than default established
by SETTINGS.

Try doing the rough equivalent to TCP with setsockopt and SO_RECVBUF and
you will get decidedly inconsistent results.

---

From: Patrick McManus <[email-masked]>
Date: Wed, 15 May 2013 17:21:40 -0400

Hi Will,

On Wed, May 15, 2013 at 4:48 PM, William Chan (陈智昌)
<[email-masked]>wrote:

> As I've said before, I think we should not attempt to leverage flow
> control mechanisms for prioritization. The client should advise the server
> as best as possible the appropriate prioritization, and then trust the
> server to serve responses (including pushed streams) in the best ordering.
>
>

I'm doing it for resource bounding only because I anticipate we will add a
mechanism for the client to advise the server of the priority of a pushed
stream.

Its pretty obvious that there are going to be a fair number of pushed
streams that never get used.  I think there are lots of scenarios where
that's true - but we can start with things like AdBlockPlus and
GreaseMonkey scripts where the view the client has of the dom is pretty
different than the server's view. And the savings of push is really just on
the order of 1 RTT for the whole page load. So client implementations need
to draw a balance between the costs of buffering and transferring that data
and the RTT savings.

Its a particular quandary on mobile. The RTT is certainly costlier.. but
storage is costlier too and more importantly to me the data transfer can be
exceptionally expensive.

I've got mobile carriers right now (mostly emergent markets) who are asking
for ways to disable existing HTTP speculative operations (e.g. DNS or
TCP/SSL handshakes) on Firefox OS phones because their top priority is
reducing data charges. (That confuses me a bit too - I thought they were in
the business of selling data, but anyhow.)

Flow controlling push provides a nice way to participate in the push
without an open ended commitment to it... and in the end I want it on the
client for the same reason server operators want it on the other end - I
can't accept an arbitrary amount of data without an established sink to
send it to.

---

From: William Chan (陈智昌) <[email-masked]>
Date: Wed, 15 May 2013 18:41:38 -0300

Terse because I'm running off to catch a bus to Mendoza.

On Wed, May 15, 2013 at 6:21 PM, Patrick McManus <[email-masked]>wrote:

> Hi Will,
>
> On Wed, May 15, 2013 at 4:48 PM, William Chan (陈智昌) <[email-masked]
> > wrote:
>
>>
> I'm doing it for resource bounding only because I anticipate we will add
> a  mechanism for the client to advise the server of the priority of a
> pushed stream.
>

Resource bounding SGTM.

> to draw a balance between the costs of buffering and transferring that data

SETTINGS_MAX_CONCURRENT_STREAMS is another option here.

> storage is costlier too and more importantly to me the data transfer can be

When you say storage, do you mean memory or disk?

>
> I've got mobile carriers right now (mostly emergent markets) who are
> asking for ways to disable existing HTTP speculative operations (e.g. DNS
> or TCP/SSL handshakes) on Firefox OS phones because their top priority is

Yep, that's fair.

> can't accept an arbitrary amount of data without an established sink to
What do you think about SETTINGS_MAX_CONCURRENT_STREAMS vs flow control
here? Also, when you say an established sink, how about the browser cache?

---

From: James M Snell <[email-masked]>
Date: Wed, 15 May 2013 14:50:24 -0700

On Wed, May 15, 2013 at 2:41 PM, William Chan (陈智昌)
<[email-masked]> wrote:
[snip]
>> that's true - but we can start with things like AdBlockPlus and GreaseMonkey
>> scripts where the view the client has of the dom is pretty different than
>> the server's view. And the savings of push is really just on the order of 1
>> RTT for the whole page load. So client implementations need to draw a
>> balance between the costs of buffering and transferring that data and the
>> RTT savings.
>
>

Bounding the max concurrent open streams is not quite enough here...
for one, if we limit the number of concurrent streams without limiting
the amount of data pushed within those streams, developers could
simply end up resorting to inlining again... one stream for all their
js files, one stream for all their css, etc.

I, for one, am planning an experiment that uses a combination of DATA
frames and a modified version of a HEADERS frame as a way of replacing
the use of multipart mime. Using that approach, it would be rather
simple for me to work around the max concurrent streams limit by
simply sequencing out multiple resources in a single stream.

- James

---

From: Patrick McManus <[email-masked]>
Date: Wed, 15 May 2013 17:59:56 -0400

> When you say storage, do you mean memory or disk?

could be either, or both. depending on deployment.

> here?

deserves experimentation.. though flow control seems more fine grained to
me.

> Also, when you say an established sink, how about the browser cache?
>

For a pretty significant set of deployments I would rather wait 1 rtt than
deal with storage i/o :)

---

From: William Chan (陈智昌) <[email-masked]>
Date: Thu, 16 May 2013 11:11:40 -0300

On Wed, May 15, 2013 at 6:59 PM, Patrick McManus <[email-masked]>wrote:

>
> On Wed, May 15, 2013 at 5:41 PM, William Chan (陈智昌) <[email-masked]
> > wrote:
>
>>

It depends. Both are just knobs with partial control. Stream flow control
needs to leave the window large enough to save RTTs, but can put a cap on
how much any single response can be. But it doesn't really help if you just
open a bunch of streams. Connection flow control is too coarse grained
though, since it's shared by pull streams. Martin's proposal is pretty
simple, yet useful. I like it. It's a specific instance of stream group
flow control, much like connection flow control is. But it seems to serve a
good purpose and doesn't require a group definition mechanism, since
connection and directionality of streams are good definitions for
establishing the groupings.

That said, I'd like to hold back from adding any mechanisms unless we have
implementers that will commit to experimenting with it.

I think a lot of this also goes back to whether or not we want to define
strict mechanisms in the protocol for controlling this, or we want to
advertise hints in the semantic layer (HTTP client hints or whatever).

Yeah, we need to experiment. I think your current flow control usage is
interesting and probably a good idea. You guys are way ahead of us on
tuning push :) We're still in the ironing out bug phase.

> For a pretty significant set of deployments I would rather wait 1 rtt than

Hah, fair enough :)