Subject: Results from adopting HTTP/3 priority

From: Alan Frindell <[email-masked]>
Date: Mon, 26 Apr 2021 15:31:00 +0000

The Instagram app for Android recently adopted HTTP priorities for HTTP/3 requests and I wanted to share our positive results with this group.

So far, Instagram has applied priorities in two different use cases:

  1. The most critical API requests during startup are assigned a higher priority than other requests.  This change resulted in a user-visible startup time win for the app, which is one of the critical metrics they track.  Unfortunately, the experiment was several months ago and we no longer have the data showing the exact win.

  2. Requests for images and videos that are on screen are assigned a higher priority than those that are off screen.  High priority content loads 20-30% faster than low priority content, and 10-20% faster than the previous baseline.  We see a reduction in the number of video stalls and an improvement in the image rendering rate.

The team is currently experimenting with reprioritizing images as their on/off screen status changes, but the early results are mixed.  This is somewhat expected since the objects may be all or partially flushed to the network by the time the reprioritization signal arrives.

The results are very encouraging and other product teams are looking to see how they can also leverage priorities.

I’d like to emphasize that the simplicity of the new priority scheme relative to the HTTP/2 scheme was a key factor in enabling the product team to engage and test quickly.  The server side implementation is also much simpler and less error prone compared to the H2 scheme.  We also find implementing priorities directly in the QUIC layer is more effective at selecting the highest priority data to send compared to HTTP/2, where careful tuning of TCP_NOTSENT_LOWAT is required to achieve the best performance.

Thanks

-Alan

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 27 Apr 2021 02:19:35 +0100

Hi Alan,

On Mon, 26 Apr 2021, 16:35 Alan Frindell, <[email-masked]> wrote:

> The Instagram app for Android recently adopted HTTP priorities for HTTP/3
> requests and I wanted to share our positive results with this group.
>

Thanks for sharing your findings. This is some valuable feedback.

Since you mention HTTP/3, I have a question about retransmission. It's not
a part of HTTP priorities per se but does your implementation consider
priority level for scheduling retransmission?

Cheers
Lucas

---

From: Yang Chi <[email-masked]>
Date: Tue, 27 Apr 2021 15:22:54 +0000

Hi,

Yes we consider priority level for retransmitting lost bytes, that is, the bytes are marked as real loss. Fresh data and loss buffer are scheduled together in our QUIC transport with respect to streams’ priorities. For PTO, we don’t consider priority.

Regards,
Yang

From: Lucas Pardue <[email-masked]>
Date: Monday, April 26, 2021 at 9:19 PM
To: Alan Frindell <[email-masked]>
Cc: HTTP Working Group <[email-masked]>, Yang Chi <[email-masked]>
Subject: Re: Results from adopting HTTP/3 priority
Hi Alan,

On Mon, 26 Apr 2021, 16:35 Alan Frindell, <[email-masked]<mailto:[email-masked]>> wrote:
The Instagram app for Android recently adopted HTTP priorities for HTTP/3 requests and I wanted to share our positive results with this group.

Since you mention HTTP/3, I have a question about retransmission. It's not a part of HTTP priorities per se but does your implementation consider priority level for scheduling retransmission?

Cheers
Lucas

---

From: Yang Chi <[email-masked]>
Date: Tue, 27 Apr 2021 15:30:26 +0000

I should also note that part of the experiment result (the critical API experiment mentioned in Alan’s email) was done without scheduling fresh data and loss data together. Back then, loss data were scheduled separately, before fresh data, but still respect to priority. In other words, we used to send high pri loss data, then low pri loss data, then high pri fresh data, then low pri fresh data. Then we switched to this new scheduler, where it first sends high pri loss data, then high pri fresh data, then low pri loss data followed by low pri fresh data. I didn’t see any interesting metrics move since this change though.

Yang

From: Yang Chi <[email-masked]>
Date: Tuesday, April 27, 2021 at 11:22 AM
To: Lucas Pardue <[email-masked]>, Alan Frindell <[email-masked]>
Cc: HTTP Working Group <[email-masked]>
Subject: Re: Results from adopting HTTP/3 priority
Hi,

Regards,
Yang

Cheers
Lucas

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 27 Apr 2021 17:53:02 +0100

On Tue, Apr 27, 2021 at 4:30 PM Yang Chi <[email-masked]> wrote:

> I should also note that part of the experiment result (the critical API
> experiment mentioned in Alan’s email) was done without scheduling fresh
> data and loss data together. Back then, loss data were scheduled
> separately, before fresh data, but still respect to priority. In other
> words, we used to send high pri loss data, then low pri loss data, then
> high pri fresh data, then low pri fresh data. Then we switched to this new
> scheduler, where it first sends high pri loss data, then high pri fresh
> data, then low pri loss data followed by low pri fresh data. I didn’t see
> any interesting metrics move since this change though.
>

Thank you Yang, very interesting. There's a lot of potential factors, so
even if metrics are neutral that is a useful observation.

---

From: Robin MARX <[email-masked]>
Date: Wed, 28 Apr 2021 11:14:09 +0200

Hello Alan and Yang,

Thank you for the very interesting insights.

I had a few more detail questions that I hope you can answer:

1) Can you detail how often you use *the "incremental" flag* and for which
types of resources (and in which urgency "buckets")? And why?
    If not using much incremental, that might explain in part why the
metrics don't change when prioritizing retransmits.
    a) I also wonder if you have more data/insight on how much QUIC's HOL
blocking removal has helped in practice and how that interacts with the
incremental logic. Matt has mentioned this before, but didn't share details
as of yet.
    b) If you do incremental, do you still Round-Robin streams on a
per-packet basis or have you switched to larger bursts per stream? Have you
experimented with other approaches?
    c) How exactly do you deal with mixed incremental/non-incremental
resources in the same urgency bucket?
        Say a and b are not incremental, then c and d are, then e again is
not. Do you do a, b, mixed cd, e? or for example: a, b, e, mixed cd (i.e.,
non-incremental is a bigger indicator of priority than stream ID/request
order)
        The spec currently does not detail how best to handle this case
(and to be honest, I'm not sure what the best default approach would
be...), but I assume the second option is easier to implement (use 2 queues
per priority bucket, 1 sequential, 1 RR).

2) You mention "implementing priorities directly in the QUIC layer": do you
have insight in the *design of the API* between the H3 and the QUIC layer
for this?
     d) Does the QUIC part internally implement this as a direct analogue
to the H3 priorities (say a list of urgency buckets, with some more
complexity for incremental vs non-incremental subslices, see c)) or does
QUIC offer a simpler/more complex setup onto which you map the H3 signals?
     e) Is there a coupling between stream-level flow control and stream
priority? (e.g., high priority streams get higher allowances from the
client than low-priority streams, to make sure they're not FC-blocked?). I
assume client-side FC allowance is always high enough, but still :)
     You mention TCP_NOTSENT_LOWAT, but the concept of deep vs shallow
buffers still stands in QUIC as well:
     f) Do you prepare buffers with data to be sent or do you (only)
generate QUIC packets/DATA frames on the fly?
     g) In the first case, how large are the buffers? Do you flush them if
a new, higher priority request arrives?

3) With regards to *prioritizing retransmits*:
   Is this deployed on a CDN-style setup, where e.g., the edge has
low-priority (static) resources cached and starts sending them before
receiving higher priority responses from the "origin"?
   I would assume that in this setup, you'd have cases where sending
high-priority fresh data before low-priority retransmits does matter a lot,
especially for non-incremental resources (though it depends on many
factors). Do you have any insight on this scenario?
   h) if deployed in a CDN-style setup, do you use (persistent) H3/QUIC to
the origin as well? I mainly wonder about if (and how) you tweak priority
across requests from multiple clients coalesced onto a single "backend
connection" (as that's one of the use cases we "lose" by dropping the
dependency tree).

If you can even answer a few of these, I'll be a happy camper ;)

With best regards,
Robin

-- 

dr. Robin Marx
Postdoc researcher - Web protocols
Expertise centre for Digital Media

*Cellphone *+32(0)497 72 86 94

www.uhasselt.be
Universiteit Hasselt - Campus Diepenbeek
Agoralaan Gebouw D - B-3590 Diepenbeek
Kantoor EDM-2.05

---

From: Alan Frindell <[email-masked]>
Date: Thu, 29 Apr 2021 15:50:38 +0000

Hi Robin, thanks for the detailed questions.  Answers inline.

> 1) Can you detail how often you use the "incremental" flag and for which types of resources (and in which urgency "buckets")? And why? 

Prior to adopting the http-priority draft, the mvfst scheduler treated everything as incremental with equivalent urgency.  After adopting the draft, we set the default priority to "u=3, i" to match this behavior.

In the places where our clients specify priorities, media requests show better results with the incremental flag, and API requests show better results without the incremental flag. We think videos in particular can benefit from the incremental flag when both audio and video requests are over the same connection. It’s not clear why API doesn’t benefit from it, but it's possible (at least in the app where we tried priority), the application layer still needs to wait for the entire JSON response to be downloaded before parsing it.

>    a) I also wonder if you have more data/insight on how much QUIC's HOL blocking removal has helped in practice and how that interacts with the incremental logic. Matt has mentioned this before, but didn't share details as of yet. 

We don’t have any additional data demonstrating how removing HoL blocking impacts perf, with or without respect to priority.

>    b) If you do incremental, do you still Round-Robin streams on a per-packet basis or have you switched to larger bursts per stream? Have you experimented with other approaches? 

We still RR on a per-packet basis. 

>    c) How exactly do you deal with mixed incremental/non-incremental resources in the same urgency bucket? 
>       Say a and b are not incremental, then c and d are, then e again is not. Do you do a, b, mixed cd, e? or for example: a, b, e, mixed cd (i.e., non-incremental is a bigger indicator of priority than stream ID/request order)

We do use two queues per priority bucket, so would send: a, b, e, mixed cd.  We sort the non-incremental streams by stream ID, which is not perfect but it does work for the common case.  Notably this is imperfect because there are 3 ID spaces for the sender (client bidi, server bidi -- unused in vanilla HTTP, and unidirectional).  We prioritize control streams over all other streams, but server pushes could easily jump ahead of earlier requests of equivalent urgency.  I agree the spec is vague here; I’m curious to hear if anyone else is using a different approach, how costly it is to implement and how that has performed.

> 2) You mention "implementing priorities directly in the QUIC layer": do you have insight in the design of the API between the H3 and the QUIC layer for this? 
>      d) Does the QUIC part internally implement this as a direct analogue to the H3 priorities (say a list of urgency buckets, with some more complexity for incremental vs non-incremental subslices, see c)) or does QUIC offer a simpler/more complex setup onto which you map the H3 signals?

We have a direct analog of the HTTP API implemented in the transport. The API is 

setPriority(streamId, urgency, incrementalFlag)

Our original design (dating to 2017) of the API between mvfst and proxygen was set up to allow for the transport to call back into the application (HTTP) layer to perform prioritization, specifically because we didn’t want the ugly complexity of the HTTP priority tree inside the transport.  This design turned out to be very complicated, and never really worked the way it was intended.  We’re still in the process of ripping it out.

The urgency/incremental priority scheme is simple enough to implement and probably powerful enough to work for future non-HTTP applications.  There are some constructs that application developers would like to express that are harder in the new scheme however -- namely a strict ordering of resources within an urgency level.   The concrete example is a list of video segments: you want the first before the second, second before third, and so on.  If the list of segments is longer than 8, you can’t implement this exactly according to the specification.  We’re able to fake it using the same urgency level, non-incremental and ordering by stream ID, but as noted earlier, the spec is vague about how to order non-incremental resources at the same urgency level.

>      e) Is there a coupling between stream-level flow control and stream priority? (e.g., high priority streams get higher allowances from the client than low-priority streams, to make sure they're not FC-blocked?). I assume client-side FC allowance is always high enough, but still :) 

We don’t allocate connection flow control according to priority.  We pretty much have flow control set high enough on the client that it shouldn’t be much of a problem.

>      You mention TCP_NOTSENT_LOWAT, but the concept of deep vs shallow buffers still stands in QUIC as well: 
>     f) Do you prepare buffers with data to be sent or do you (only) generate QUIC packets/DATA frames on the fly? 

We run the priority algorithm to pick a stream when mvfst is ready to send a packet -- there is no buffering after prioritization.  The HTTP/3 DATA frames are pre-generated but the QUIC STREAM frames are not.

>     g) In the first case, how large are the buffers? Do you flush them if a new, higher priority request arrives? 

Not applicable since we don’t do it that way.

> 3) With regards to prioritizing retransmits:
>   Is this deployed on a CDN-style setup, where e.g., the edge has low-priority (static) resources cached and starts sending them before receiving higher priority responses from the "origin"?

Static (cacheable) resources and dynamic resources that always hit origin are fetched over different connections today.  The prioritization within these two subgroups are separate.  That said, it’s possible that within a static connection, low priority content is cached and high priority content is arrives at the edge later.

>   I would assume that in this setup, you'd have cases where sending high-priority fresh data before low-priority retransmits does matter a lot, especially for non-incremental resources (though it depends on many factors). Do you have any insight on this scenario? 

I don’t think we have an specific insight into how prioritizing the high priority data over low priority retransmits affects performance in this scenario, other than it seems like the right thing to do.

>   h) if deployed in a CDN-style setup, do you use (persistent) H3/QUIC to the origin as well? I mainly wonder about if (and how) you tweak priority across requests from multiple clients coalesced onto a single "backend connection" (as that's one of the use cases we "lose" by dropping the dependency tree).

We have a mix of persistent HTTP/2 and HTTP/3 connections between our edge nodes and origins.  We currently give all requests on these connections equal priority, with the incremental flag.  To my knowledge, no CDN has ever implemented any kind of priority scheme for multiplexed connections to origin.  At the last HTTP Workshop, it seemed that very few CDNs even used a multiplexed protocol for that hop.

-Alan