Subject: Re: HTTP/3 Prioritization Proposal

From: Ian Swett <[email-masked]>
Date: Fri, 3 May 2019 16:26:07 -0400

On Tue, Jan 29, 2019 at 3:12 PM Patrick McManus <[email-masked]>
wrote:

>
> On Tue, Jan 29, 2019 at 2:56 PM Patrick Meenan <[email-masked]>
> wrote:
>
>> As far as I can tell, the placeholder streams serve to handle the Firefox
>> use case of using idle streams for groupings,
>>
>
>  yes.. and you can probably solve for that in a simpler way by having an
> explicit set of groups with simple ways to share between them.
>
> But what I think you really need to do with your proposal is address what
> you're giving up by removing the tree structure because it was an explicit
> choice to include it.
>
> That structure exists because Google convinced the WG that it was
> important to be able to combine an arbitrarily large number of sets of
> streams together fairly. (and the solution allowed generalized sharing, not
> just fairness).
>

This thread got far ahead of me, but I wanted to ask for more motivation
behind the tree structure(links welcome).  'Google' may have argued for it,
but that doesn't mean it was ever used as envisioned.  Is anyone else
taking advantage of it?

> In short, if you've got a set of streams from tabs A, B, and C you cannot
> really expect them to be coordinated in an absolute priority sense - but if
> they were all rooted at the same level in a tree they could share fairly
> and then the streams within the tab could locally coordinate their priority.
>

For a given application(browser, app, etc), I'd expect absolute priority to
be a fairly good indicator across connections, because that's easy and the
alternatives are harder.

>
> This is a much more important property in an aggregator like a CDN who
> might be bringing different front end connections into a single backend
> connection.. the priority expressed by the client should exist in some ways
> e2e (css before imgs!), but in other ways hop to hop (you don't want every
> css to stall every browser's images).. the tree allows that.
>

This statement concerns me for a few reasons.  One is I doubt any CDNs can
pull this off at scale, so I don't think it's practical.  Someone should
correct me if I'm wrong.  Another is that to pull this off, you'd need
reliable ways to know that a single user was the owner of two different
connections, which seems potentially concerning from a privacy
perspective?  Lastly, I don't think it would result in optimal loading.  If
one could do this, strict numerical priorities would likely work better,
because they'd preserve most of the clients original intent instead of
equally sharing bandwidth between blocking resources(ie: HTML, CSS) and
non-blocking ones(ie: images).

---

From: Andy Green <[email-masked]>
Date: Sat, 4 May 2019 06:23:01 +0100

On 03/05/2019 21:26, Ian Swett wrote:
> 
> On Tue, Jan 29, 2019 at 3:12 PM Patrick McManus <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
> 
>     On Tue, Jan 29, 2019 at 2:56 PM Patrick Meenan <[email-masked]
>     <mailto:[email-masked]>> wrote:
> 
>         As far as I can tell, the placeholder streams serve to handle
>         the Firefox use case of using idle streams for groupings,
> 
> 
>       yes.. and you can probably solve for that in a simpler way by
>     having an explicit set of groups with simple ways to share between them.
> 
>     But what I think you really need to do with your proposal is address
>     what you're giving up by removing the tree structure because it was
>     an explicit choice to include it.
> 
>     important to be able to combine an arbitrarily large number of sets
>     of streams together fairly. (and the solution allowed generalized
>     sharing, not just fairness).
> 
> 
> behind the tree structure(links welcome).  'Google' may have argued for 
> it, but that doesn't mean it was ever used as envisioned.  Is anyone 
> else taking advantage of it?

libwebsockets supports h2 server, h2 client with stream bundling, and 
ws-over-h2 server... it ignores PRIORITY and I've only ever been asked 
about it one time.  It just uses a round-robin scheduler between streams 
that have tx credit + more to send to allow them to write frames on the 
network connection.

>     In short, if you've got a set of streams from tabs A, B, and C you
>     cannot really expect them to be coordinated in an absolute priority
>     sense - but if they were all rooted at the same level in a tree they
>     could share fairly and then the streams within the tab could locally
>     coordinate their priority.
> 
> 
> For a given application(browser, app, etc), I'd expect absolute priority 
> to be a fairly good indicator across connections, because that's easy 
> and the alternatives are harder.

PRIORITY and the stream tx credit scheme have an almost complete 
overlap.  If the default stream tx credit is small, or it's updated to 
walk the credit back after opening the stream, the client can use 
modulation of that per-stream to enforce the detailed priority it wants 
without PRIORITY or trees of PRIORITY or whatever being an explicit 
thing on the wire told to the server at all.

And since properly managing tx credit is a bug-magnet, from that 
perspective it would've been better to exercise and re-use that instead 
of PRIORITY.

So IMHO, the whole of PRIORITY is a white elephant.

>     This is a much more important property in an aggregator like a CDN
>     who might be bringing different front end connections into a single
>     backend connection.. the priority expressed by the client should
>     exist in some ways e2e (css before imgs!), but in other ways hop to
>     hop (you don't want every css to stall every browser's images).. the
>     tree allows that.
> 
> 
> This statement concerns me for a few reasons.  One is I doubt any CDNs 
> can pull this off at scale, so I don't think it's practical.  Someone 
> should correct me if I'm wrong.  Another is that to pull this off, you'd 
> need reliable ways to know that a single user was the owner of two 
> different connections, which seems potentially concerning from a privacy 
> perspective?  Lastly, I don't think it would result in optimal loading.  
> If one could do this, strict numerical priorities would likely work 
> better, because they'd preserve most of the clients original intent 
> instead of equally sharing bandwidth between blocking resources(ie: 
> HTML, CSS) and non-blocking ones(ie: images).

If the CDN had the information to do that well, it can also express it 
by stream tx credit modulation.

-Andy

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 4 May 2019 16:32:19 +1000

> On 4 May 2019, at 6:26 am, Ian Swett <[email-masked]> wrote:
> 
>> This is a much more important property in an aggregator like a CDN who might be bringing different front end connections into a single backend connection.. the priority expressed by the client should exist in some ways e2e (css before imgs!), but in other ways hop to hop (you don't want every css to stall every browser's images).. the tree allows that.
> 
> This statement concerns me for a few reasons.  One is I doubt any CDNs can pull this off at scale, so I don't think it's practical.  

Any particular reason why? Most CDNs have a hierarchy of some kind, to protect the origin from having to serve too many CDN nodes directly. 

This came up at the latest HTTP Workshop; no one with an intermediary was using priorities in this way -- but that's because most intermediaries aren't using H2 on the back-end connection. Once they do, the next hurdle they'll hit is that H2's use of TCP means that using a single connection to origin is unattractive, so they'll use a number in parallel. In theory, H3 should address that; in practice, we'll see.

Regardless, it's not unreasonable to think that an intermediary will multiplex more than one client's traffic onto a connection (either to a parent node, or to the origin). 

> Someone should correct me if I'm wrong.  Another is that to pull this off, you'd need reliable ways to know that a single user was the owner of two different connections, which seems potentially concerning from a privacy perspective?

Why would that be? 

>  Lastly, I don't think it would result in optimal loading.  If one could do this, strict numerical priorities would likely work better, because they'd preserve most of the clients original intent instead of equally sharing bandwidth between blocking resources(ie: HTML, CSS) and non-blocking ones(ie: images).

I'm not sure what you mean here. With strict numerical priorities, an intermediary would either have to pass through the priorities unchanged (in which case a misbehaving client could claim everything is e.g., priority 255 and hog resources), or it'd have to normalise the priorities in some fashion, which practically (if not completely) precludes preserving the relative priority of a given client's requests.

Cheers,

--
Mark Nottingham   https://www.mnot.net/

---

From: Patrick Meenan <[email-masked]>
Date: Sat, 4 May 2019 09:26:46 -0400

tx credit round robin is basically the default HTTP/2 prioritization of
even weighting across streams, isn't it? If so and the content being served
is web pages to a browser it can be as much as an order of magnitude slower
than if the priorities were honored (Chrome mitigates this a bit currently
by holding back requests because...wait for it....the current state of
priority support is pretty bad).

If you have a typical page with a few blocking scripts and stylesheets in
the head, a bunch of images (some visible, some not) and more scripts at
the end of the document, delivering them in priority order lets the browser
start rendering the content by just loading the scripts/stylesheets in the
head (and then prioritizing the images).  If you round-robin all of the
streams the page will be blank until everything finishes downloading and
then it will finally display. Not only is that an order of magnitude slower
compared with HTTP/2 servers that do support priorities, it's also MUCH
slower than HTTP/1.1.

I'm sure there are cases where it doesn't matter (and maybe those are the
cases where libwebsockets is used) but it is absolutely critical for
browser <-> server connections. There's also a good chance that your users
don't realize it isn't supported or don't realize it isn't working (which
is entirely believable given the current situation
<https://github.com/andydavies/http2-prioritization-issues#cdns--cloud-hosting-services>
).

---

From: Patrick Meenan <[email-masked]>
Date: Sat, 4 May 2019 09:30:08 -0400

I'm still wondering if maybe we can find a way to handle the multiplexing
case without necessarily needing full tree knowledge/management on both
sides.  It does mean making things more complicated than a simple
"priority" but maybe there is a middle-ground that can work.  Something
like having groups that streams can belong to and the prioritization
applies within a group but then groups get weighted against each other.  It
feels like a bit of a slippery slope though that as we try to solve each
corner case we end up with something that looks a lot like the current
HTTP/2 scheme.

On Sat, May 4, 2019 at 2:32 AM Mark Nottingham <[email-masked]> wrote:

> can pull this off at scale, so I don't think it's practical.
>
> Any particular reason why? Most CDNs have a hierarchy of some kind, to
> protect the origin from having to serve too many CDN nodes directly.
>
> This came up at the latest HTTP Workshop; no one with an intermediary was
> using priorities in this way -- but that's because most intermediaries
> aren't using H2 on the back-end connection. Once they do, the next hurdle
> they'll hit is that H2's use of TCP means that using a single connection to
> origin is unattractive, so they'll use a number in parallel. In theory, H3
> should address that; in practice, we'll see.
>
> Regardless, it's not unreasonable to think that an intermediary will
> multiplex more than one client's traffic onto a connection (either to a
> parent node, or to the origin).
>
>
> > Someone should correct me if I'm wrong.  Another is that to pull this
> off, you'd need reliable ways to know that a single user was the owner of
> two different connections, which seems potentially concerning from a
> privacy perspective?
>
> Why would that be?
>
>
> >  Lastly, I don't think it would result in optimal loading.  If one could
> do this, strict numerical priorities would likely work better, because
> they'd preserve most of the clients original intent instead of equally
> sharing bandwidth between blocking resources(ie: HTML, CSS) and
> I'm not sure what you mean here. With strict numerical priorities, an
> intermediary would either have to pass through the priorities unchanged (in
> which case a misbehaving client could claim everything is e.g., priority
> 255 and hog resources), or it'd have to normalise the priorities in some
> fashion, which practically (if not completely) precludes preserving the
> relative priority of a given client's requests.
>
> Cheers,
>
> --

---

From: Lucas Pardue <[email-masked]>
Date: Sat, 4 May 2019 15:24:25 +0100

> but that doesn't mean it was ever used as envisioned.  Is anyone else

For convenience, here is a potted history I recently put together, it
doesn't answer all questions:

* HTTP/2 up to draft 10 had a scheme similar to SPDY/3.1
* Proposal for stream dependencies in SPDY/4 - Oct 2012 [1]
* "http/2 prioritization/fairness bug with proxies" [2] - mailig list
thread Feb 2013
* "Restarting the discussion on HTTP/2 stream priorities" [3] - mailing
list thread Oct 2013 alluding to Seattle interim discussion.
* HTTP/2 Stream Dependencies I-D - Jan 2014 [4]
* Mailing list thread for comments on the I-D [5]
* Introduction of dependencies to HTTP/2 draft 11 diff, Feb 2014 [6]
* Diff of changes to PRIORITY frame between draft 11 and RFC 7540 [7]

[1]
https://docs.google.com/document/d/1pNj2op5Y4r1AdnsG8bapS79b11iWDCStjCNHo3AWD0g/edit#heading=h.fu9pi3zfp7k5
[2] https://lists.w3.org/Archives/Public/ietf-http-wg/2013JanMar/0554.html
[3] https://lists.w3.org/Archives/Public/ietf-http-wg/2013OctDec/0366.html
[4] https://tools.ietf.org/html/draft-chan-http2-stream-dependencies-00
[5] https://lists.w3.org/Archives/Public/ietf-http-wg/2014JanMar/0090.html
[6] https://tools.ietf.org/rfcdiff?url2=draft-ietf-httpbis-http2-11.txt
[7]
https://tools.ietf.org/rfcdiff?url1=draft-ietf-httpbis-http2-11.txt&url2=rfc7540

---

From: Andy Green <[email-masked]>
Date: Sat, 4 May 2019 15:29:02 +0100

On 04/05/2019 14:26, Patrick Meenan wrote:

> even weighting across streams, isn't it? If so and the content being 

I think you may have missed the point.

PRIORITY isn't implemented in lws, but it does implement tx credit more 
or less.  And you can simulate PRIORITY by the client using tx credit 
modulation to control how much of what can come at any given time.

> served is web pages to a browser it can be as much as an order of 
> magnitude slower than if the priorities were honored (Chrome mitigates 
> this a bit currently by holding back requests because...wait for 
> it....the current state of priority support is pretty bad).

Yeah... maybe there's a reason for that.

> If you have a typical page with a few blocking scripts and stylesheets 
> in the head, a bunch of images (some visible, some not) and more scripts 
> at the end of the document, delivering them in priority order lets the 
> browser start rendering the content by just loading the 
> scripts/stylesheets in the head (and then prioritizing the images).  If 
> you round-robin all of the streams the page will be blank until 
> everything finishes downloading and then it will finally display. Not 
> only is that an order of magnitude slower compared with HTTP/2 servers 
> that do support priorities, it's also MUCH slower than HTTP/1.1.

... order of magnitude eh...

> I'm sure there are cases where it doesn't matter (and maybe those are 
> the cases where libwebsockets is used) but it is absolutely critical for 

No it isn't.  Nothing breaks... everything works just fine without it. 
You can quite happily trade off its complexity and memory footprint by 
ignoring it and eating some reduction in speed the first time the site 
is visited.  There's a clientside cache in the browser case that can be 
populated once and will likely stay there a year or whatever.  Then much 
of this struggle and complexity turns out to be over "prioritizing" 
which few bytes of a 304 you get first after the first access.

In the cases there are advantages to controlling the ordering, PRIORITY 
and all points south like "stream reprioritization", priority trees, etc 
are artifacts and states that live in the client and can stay there, 
simply driving client decisions about stream WINDOW_UPDATE emission to 
get almost the same result implicitly.

> browser <-> server connections. There's also a good chance that your 
> users don't realize it isn't supported or don't realize it isn't working 
> (which is entirely believable given the current situation 
> <https://github.com/andydavies/http2-prioritization-issues#cdns--cloud-hosting-services>).

Lws operates at the very low end, down to things like esp32.  In that 
world, the expense of the tls tunnel is really high (even with mbedtls, 
 > 32KB per tunnel just for buffers on machines with < 200KB global 
heap).  The tunnels are slow to set up with large keys in the certs too. 
  Using h2 to mux inside one tls context is already a huge win.  In 
these case optimizing ordering is unimportant.

-Andy

> On Sat, May 4, 2019 at 1:23 AM Andy Green <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
> 
> 
>      > On Tue, Jan 29, 2019 at 3:12 PM Patrick McManus
>     <[email-masked] <mailto:[email-masked]>
>      > <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>      >
>      >
>      >     On Tue, Jan 29, 2019 at 2:56 PM Patrick Meenan
>     <[email-masked] <mailto:[email-masked]>
>      >     <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>      >
>     Someone

---

From: Patrick Meenan <[email-masked]>
Date: Sun, 5 May 2019 09:52:50 -0400

Using flow control from the client to try to schedule resource delivery
doesn't work. It's a bit like pushing on a string, causes HOL blocking and
there is no way to do it without a 1 RTT delay between each action. Every
case we've considered using flow control as a way to control the streams
has ended up being worse for performance. The latest that comes to mind was
to use it for scheduling image delivery on low-bandwidth connections by
pausing streams after the first few KB of each image is delivered but by
the time the WINDOW_UPDATE makes it to the server there is enough data
already in-flight to negate the benefit.

Assume the simple case of resources A, B and C that you want delivered in
that order.

1 - Client sends requests for A, B and C along with WINDOW_UPDATEs for B
and C to pause them
2 - Server responds with A
3 - Client sends WINDOW_UPDATE for B to start streaming it
4 - Server responds with B
5 - Client sends WINDOW_UPDATE for C to start streaming it
6 - Server responds with C

That's functionally identical to HTTP/1.1 where the request for A is sent
first followed by requests for B and C, all over the same connection with
HOL blocking for A and delays between each resource.

The same applies with sending a high-priority request (X) while other
low-priority responses are already in-flight.  If you pause the other
streams then you introduce whatever back-end delay there is in
generating/fetching X as a gap in the data transfer.  If you wait for X to
start streaming and then pause the other streams then you are introducing a
1 RTT delay in increasing the priority of X. In both cases, resuming the
other streams after X completes introduces another 1 RTT delay. If X can't
fully saturate the connection then you are also artificially limiting the
transfer.

To be able to fully saturate the connection, most clients open up pretty
large windows right at the start and just let TCP flow control take over on
the connection as a whole.  I'm sure there were use cases in mind when the
flow control was added to the spec but at least from a browser/web content
perspective it would be easier if it didn't exist in the HTTP/2 layer and
was just handled at the transport layer.

On Sat, May 4, 2019 at 10:29 AM Andy Green <[email-masked]> wrote:

> > <
> https://github.com/andydavies/http2-prioritization-issues#cdns--cloud-hosting-services
> >).
>

---

From: Ian Swett <[email-masked]>
Date: Tue, 7 May 2019 15:54:48 -0400

> protect the origin from having to serve too many CDN nodes directly.

I may have misunderstood what Patrick was trying to say.  I thought he was
imagining a case where multiple connections from the SAME client hit a
single CDN node.  After re-reading his text and reading your comments, it
appears you have a case where there are many different clients going to a
single origin server and you're trying to use priorities to provide equal
weighting instead of using multiple connections.

In that case, strict priorities within groups and round robin within
groups(aka placeholders) seems like a better design that's simpler than the
existing proposal.  And client-facing servers could indicate a max number
of placeholders of 0.

> should address that; in practice, we'll see.

I think you're supporting my point that we have essentially no deployment
experience with these being utilized as originally envisioned?

> two different connections, which seems potentially concerning from a

It was the single user aspect I was confused by.  I'll assume that was a
misunderstanding.

> relative priority of a given client's requests.

That makes sense now that I understand what Patrick had in mind.  Though
hopefully the cache hit ratio of your CDN is high enough that most
resources don't go back to the origin.

---

From: Ian Swett <[email-masked]>
Date: Tue, 7 May 2019 16:09:13 -0400

Patrick, I should have read your reply first, but this is what I was trying
to describe in my reply a few minutes ago.

I made the additional observation that browsers don't really need
weighting, so I'd expect most servers to set the max number of groups to
1.  This lowers the bar for simple servers and clients A LOT while giving
some flexibility to those who want to experiment with something fancier.

But I'd rather make some type of numerical priorities the default in the
spec and add one or more extensions to experiment with more complex
options, given I don't expect we'll get evidence that the extra complexity
has any benefits prior to standardizing HTTP/3.

On Sat, May 4, 2019 at 9:30 AM Patrick Meenan <[email-masked]> wrote:

>> >  Lastly, I don't think it would result in optimal loading.  If one
>> could do this, strict numerical priorities would likely work better,