Subject: Fwd: New Version Notification for draft-kazuho-httpbis-priority-00.txt

From: Kazuho Oku <[email-masked]>
Date: Mon, 8 Jul 2019 21:56:52 +0900

Hi,

Today, Lucas and I have submitted the following draft, that defines an
HTTP header field for driving prioritization.

https://datatracker.ietf.org/doc/draft-kazuho-httpbis-priority/

In short, the draft defines an end-to-end HTTP header field called
"Priority" for transmitting prioritization hints in absolute values
that have meanings. It is much simpler than the prioritization scheme
of H2 that communicates and uses a "tree". Not only the client, but
also the server can send the prioritization hints, in order to improve
the way the responses are prioritized. The prioritization scheme is
independent to HTTP versions; it can be used on both H2 and H3 (and
also on H1 for providing hints to intermediaries).

For more detail, please refer to the draft.

Background: back in May in London, the QUIC WG had a discussion on if
porting the prioritization scheme of H2 to H3 is the way to go [1].

I think there were two major arguments for having something different:
* Prioritization scheme of H2 is complex, and porting it to H3
increases the complexity [2].
* With the H2 scheme, it is hard for the server to tweak the
prioritization tree, because clients build their trees in their own
ways [3][4].

The arguments against were:
* Redesigning prioritization for H3 is likely to delay the
standardization and time-to-market.
* Having something different in H3 is an act of "adding" complexity to
HTTP as a whole.

This discussion has led us to wonder if there could be a technical
solution that resolves the issues of the H2 scheme (see the pro
arguments), at the same time minimizing the downsides.

And we came up with this proposal. I think the key selling points of
the proposal are:

* Much simpler thanks to each request carrying an absolute priority
value. No need to synchronize a "tree".
* Because the priority value sent by the client indicates how each
request / response affects the use of other responses (e.g.,
"blocking", "non-blocking"), the server can understand the intent of
the client and further optimize the delivery order.
* Use of the HTTP header field as the conveyer of the priority value
allows an origin server to indicate hints to an intermediary that
terminates the H2/H3 connections from the client. For example, an
origin server can indicate higher precedence for some images that
matter more to the user, while giving lower precedence to others.
* Another benefit of using an HTTP header field instead of a frame is
that prioritization logic becomes independent from HTTP versions. That
would help both clients and servers improve the quality of their
implementation, as well as fostering website owners fine-tune the
prioritization through the use of the Priority response header.
* A header-based prioritization scheme can be improved as time goes,
as HTTP headers are extensible by their nature. It also means that a
header-based design can be rolled out at an earlier stage than a
frame-based design.

To paraphrase, the header-based design provides simplicity,
extensibility, room to evolve, and the possibility to roll out early.
This could be the only prioritization scheme for H3. Separating
prioritization into a version-independent approach simplifies H3,
taking some load away from QUIC WG.

Of course, H3 needs to hit the market with a prioritization scheme, we
need to agree on that particular scheme. But I think we might be able
to agree on the need for a header-based prioritization scheme, that a
server can tweak, that uses absolute priorities. If that is the case,
I think we have fair chance on agreeing on something pretty soon.

To summarize, with the proposed design, I think we have the chance of
making prioritization better as we roll out HTTP/3, without getting
the standardization process delayed.

Please let us know what you think. Thank you in advance.

[1] https://github.com/quicwg/wg-materials/blob/master/interim-19-05/priorities.pdf
[2] https://github.com/quicwg/base-drafts/issues/2739
[3] https://github.com/quicwg/base-drafts/issues/2740
[4] In https://lists.w3.org/Archives/Public/ietf-http-wg/2019JulSep/0008.html,
Robin points out that server intervention is necessary for best
performance.

---------- Forwarded message ---------
From: <[email-masked]>
Date: 2019年7月8日(月) 21:51
Subject: New Version Notification for draft-kazuho-httpbis-priority-00.txt
To: Kazuho Oku <[email-masked]>, Lucas Pardue <[email-masked]>

A new version of I-D, draft-kazuho-httpbis-priority-00.txt
has been successfully submitted by Kazuho Oku and posted to the
IETF repository.

Name:           draft-kazuho-httpbis-priority
Revision:       00
Title:          The Priority HTTP Header Field
Document date:  2019-07-08
Group:          Individual Submission
Pages:          9
URL:
https://www.ietf.org/internet-drafts/draft-kazuho-httpbis-priority-00.txt
Status:         https://datatracker.ietf.org/doc/draft-kazuho-httpbis-priority/
Htmlized:       https://tools.ietf.org/html/draft-kazuho-httpbis-priority-00
Htmlized:
https://datatracker.ietf.org/doc/html/draft-kazuho-httpbis-priority

Abstract:
   This document describes the Priority HTTP header field.  This header
   field can be used by endpoints to specify the absolute precedence of
   an HTTP response in an HTTP-version-independent way.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

-- 
Kazuho Oku

---

From: Robin MARX <[email-masked]>
Date: Mon, 8 Jul 2019 16:55:50 +0200

Hello Kazuho and Lucas,

Thank you for the draft. Most of this feedback I've given elsewhere as
well, but to keep everything in one place:

1) I really like the header-based approach. It's extensible, allows us to
make progress on H3, is easier to use/understand,
can be "backported" to H2, has other nice properties which you touch upon
in the text (intermediates, caching, etc.)

2) I really dislike the header-based approach. It makes re-prioritization a
mess. You don't touch upon this (yet) in the draft,
but the discussion on github (
https://github.com/kazuho/draft-kazuho-httpbis-priority/wiki/Roughing-out-reprioritization)
proposes
using a separate, special H3/H2 level frame to provide this. This just
feels -very- dirty to me. The problem is that I can't really think
of a (much) better solution, other than referring to re-prioritized
resource A in resource B's headers, which is a whole other can of worms.
I'm not
really opposed to using the separate frame if that's the only option, but
it still takes away some of the nice properties of 1)

3) I really like the switch to "absolute"/"stateless" priority
levels/semantics (as you're referred to them elsewhere), as opposed to
building the tree directly.
This really helps for (partial) server-side (re-)prioritization. It does
require the server (implementers) to know a bit more about how browsers
work, but
I don't really see that as a big issue (given that we provide guidance and
examples on proper options)

4) I don't feel the current defined priority fields+values cover the use
cases though. You touch upon this in section 5.2, but I disagree with you
there:
For me, these new priority primitives are the core of the proposal (not the
header-based approach. I would champion these new semantics in a
frame-based setup as well)
and imo these should be nailed down (semi-)completely before considering
this approach. There should probably be more degrees of "urgency" \
(e.g., as Patrick Meenan mentioned things like "deferred" and
"background'), and there should maybe be something like
"importance"/"weight"/"precedence"/...
to be more fine-grained within resources of the same "urgency" level. As
mentioned in
https://github.com/kazuho/draft-kazuho-httpbis-priority/issues/12, it might
be good
to look at the existing work on the Priority Hints spec, seeing as they
probably have already looked at much of this as well.

5) I am also not sure about the interpretation of the current fields. For
example, as noted by Patrick (
https://github.com/kazuho/draft-kazuho-httpbis-priority/issues/13),
blocking has a specific meaning and it doesn't map 100% to your current
text. Similarly, what does progressive mean exactly? IIUC JS and CSS can be
parsed/compiled
in a streaming fashion in modern engines as well, it's mainly their
application that is delayed until they are fully available. Similarly, is a
non-progressively encoded jpeg
counted as "progressive"? How does the browser know if a JPEG will be
progressive or not when making the request? I feel you've got the answers,
but the usage of these
specific terms can make it more difficult to actually use this scheme in
practice.

6) This is also clear a bit from the example in section 3 (switching image
from progressive to non-progressive). I -assume- the goal here is to do
something like a "hero" image,
which you want to send before other images. However, it feels to me that
"abusing" the progressive field for this is not the best way to go about
that.

7) Most of this comes together in the fact that I'm having a hard time
thinking of how to represent existing H2 logic/browser use cases in this
new scheme.
E.g., how would you derive Chrome's current "dynamic fifo" from this
metadata? How would you implement Patrick Meenan's proposed 'ideal' setup
from these directives?
Having a couple of concrete examples would help to understand your intents
and probably also to ferret out some missing pieces.

TL;DR: I see this draft as a combination of two proposals:
A) a new way to define priority levels/semantics (i.e., "how will the
resource be used " rather than "when should you send it") : all for this!
         Needs more work though, current draft doesn't (fully) support use
cases and imo, it should.
B) sending those new levels via headers instead of frames: many advantages,
and "feels right"
        Except for the re-prioritization bit...

With best regards,
Robin

On Mon, 8 Jul 2019 at 15:00, Kazuho Oku <[email-masked]> wrote:

> [1]
> https://github.com/quicwg/wg-materials/blob/master/interim-19-05/priorities.pdf
> [4] In
> https://lists.w3.org/Archives/Public/ietf-http-wg/2019JulSep/0008.html,
> Status:
> https://tools.ietf.org/html/draft-kazuho-httpbis-priority-00
> Htmlized:

-- 

Robin Marx
PhD researcher - web performance
Expertise centre for Digital Media

T +32(0)11 26 84 79 - GSM +32(0)497 72 86 94

www.uhasselt.be
Universiteit Hasselt - Campus Diepenbeek
Agoralaan Gebouw D - B-3590 Diepenbeek
Kantoor EDM-2.05

---

From: Kazuho Oku <[email-masked]>
Date: Tue, 9 Jul 2019 10:42:23 +0900

Hi Robin,

Thank you for the comments. My responses below.

> in the text (intermediates, caching, etc.)

I'm delighted to hear that.

> 2) I really dislike the header-based approach. It makes re-prioritization
> a mess. You don't touch upon this (yet) in the draft,

While I can see how you feel sad, I am not worried, because
reprioritization can also be HTTP-version-independent in terms of API. The
only difference would be how the prioritization hints (in text) are encoded
as frames.

:+1:

> For me, these new priority primitives are the core of the proposal (not
> the header-based approach. I would champion these new semantics in a
> to be more fine-grained within resources of the same "urgency" level.
>

I think that adding "deferred" is easy, assuming that we would agree on the
meaning. It would mean the responses that should be sent _after_ the
"non-blocking" responses.

Re "background," I think we need to discuss how we want to prioritize them.
Should we assign it a yet lower precedence (than "deferred")? Or should we
state that it should be given some amount of bandwidth regardless of other
responses?

As mentioned in
> https://github.com/kazuho/draft-kazuho-httpbis-priority/issues/12, it
> might be good
> text.

I assume this to have been fixed by
https://github.com/kazuho/draft-kazuho-httpbis-priority/pull/14. IIUC,
Patrick's concern was that "blocks the processing of the document" seems to
be applicable to JavaScript files loaded in <BODY>. In #14 we've changed
the text to "blocks using the document", and I think it's clearer that
urgency=blocking does not necessarily apply to such JavaScript files.

> Similarly, what does progressive mean exactly? IIUC JS and CSS can be
> parsed/compiled
> application that is delayed until they are fully available.
>

When used by a client, it is a signal that indicates if the client prefers
receiving the responses with the bandwidth being distributed among similar
responses, or if it prefers receiving the response one by one.

Similarly, is a non-progressively encoded jpeg
> specific terms can make it more difficult to actually use this scheme in

For images, a browser should set progressive to 1, when it can assume that
the image can be rendered progressively, as doing so improves user
experience.

Servers would benefit from setting progressive to 0 for baseline JPEG
images, assuming that sending the first few percent of the file does not
improve user experience.

Assuming that what I've stated here makes sense, I think it might be a good
idea to clarify these points using examples in the draft.

> "abusing" the progressive field for this is not the best way to go about

As stated above, that's not the intent.

> and probably also to ferret out some missing pieces.

I agree that talking about mappings is a good idea. I think it would be
something like:

HTML -> urgency=document, progressive=?1
JS, CSS in HEAD -> urgency=blocking, progressive=?0
images -> urgency=non-blocking, progressive=?1
async-loaded JS -> urgency=deferred, progressive=?0

There's some wiggle room for fonts and JS, CSS being used inside BODY.
Depending on how important they seem to be, a client can set urgency to
"blocking" (if it essentially prevents the document from being used), to
"document" (if it thinks that the resources are as important as characters
and tags inside the HTML document), to "non-blocking" (if it thinks that
they are not important in terms of using the document), or even to
"deferred" (consider the case of a script tag at the body of the HTML
loading some analytic script).

If we are to be fine with one particular approach, I think we can recommend
that. But we do not need to struggle to reach consensus on one particular
algorithm, because in the proposed approach, servers can correct the
precedence of mis-prioritized requests under the proposed scheme.

> B) sending those new levels via headers instead of frames: many
> advantages, and "feels right"

-- 
Kazuho Oku

---

From: Robin MARX <[email-masked]>
Date: Tue, 9 Jul 2019 15:55:58 +0200

Hello Kazuho,

Thanks for the comments, most of which I agree with. The clarification from
Lucas elsewhere that the frame could contain an "opaque encoded header"
makes things a bit more pleasant for me.
I have created some issues on github to further explain some of my points,
as I feel some of your comments don't really answer my other reservations.

5) and 7) On the semantics of the used header names and values:
https://github.com/kazuho/draft-kazuho-httpbis-priority/issues/27
    My point was more general that the semantics of what you have now do
not map (perfectly) to the semantics that are already in-place when talking
about browser resource loading and this can create confusion down the road.
    Simply clarifying what your "re-defined" blocking means does not help
prevent confusion all that much imo.
    With your examples at the end as well, for me you are "abusing" the
blocking indicator to manipulate sending behaviour, rather than
communicating the actual semantics of a blocking resource (a CSS in the
document is still render blocking, even if it is less important than one in
the <head>).
    Resolution to this can be as simple as re-naming the values to prevent
confusion.

6) On the array of use cases / semantics that can be represented/should be
representable:
https://github.com/kazuho/draft-kazuho-httpbis-priority/issues/28
    You say for example that promoting an image to a hero image is not the
intent of the example. I can then ask: how would you indicate a hero-image
server to client then? There doesn't seem to be a way in the current setup
to (properly) do this.
    You could say that the server simply uses its internal knowledge and
sends that image first, but then the whole "this is usable by
intermediates"-argument becomes less powerful.
    I feel we should communicate as much metadata as possible back to the
server to allow it to make complex decisions, if it so chooses. The server
can still ignore most of the metadata if it decides to implement a simpler
scheme.
    In essence, the metadata communicated in the header should, imo, be
relatively de-coupled from the concrete implementations at the server (and
the current HTTP/2 setups)

With best regards,
Robin

On Tue, 9 Jul 2019 at 03:42, Kazuho Oku <[email-masked]> wrote:

>> I don't really see that as a big issue (given that we provide guidance
>> and examples on proper options)
>>
>
> :+1:
>
>
>>
> I think that adding "deferred" is easy, assuming that we would agree on
> the meaning. It would mean the responses that should be sent _after_ the
> Re "background," I think we need to discuss how we want to prioritize
> them. Should we assign it a yet lower precedence (than "deferred")? Or
> should we state that it should be given some amount of bandwidth regardless
> of other responses?
>
> As mentioned in
> Assuming that what I've stated here makes sense, I think it might be a
> good idea to clarify these points using examples in the draft.
>
>
>>
>> 6) This is also clear a bit from the example in section 3 (switching
>> image from progressive to non-progressive). I -assume- the goal here is to
>> do something like a "hero" image,
>> Having a couple of concrete examples would help to understand your
>> intents and probably also to ferret out some missing pieces.
>>
>
> If we are to be fine with one particular approach, I think we can
> recommend that. But we do not need to struggle to reach consensus on one
> particular algorithm, because in the proposed approach, servers can correct
> the precedence of mis-prioritized requests under the proposed scheme.
>
>
>>
>>          Needs more work though, current draft doesn't (fully) support
>> use cases and imo, it should.

-- 

Robin Marx

www.uhasselt.be

---

From: Kazuho Oku <[email-masked]>
Date: Wed, 10 Jul 2019 10:25:44 +0900

Hi Robin,

2019年7月9日(火) 22:56 Robin MARX <[email-masked]>:

> Hello Kazuho,
>
> Thanks for the comments, most of which I agree with. The clarification
> from Lucas elsewhere that the frame could contain an "opaque encoded
> header" makes things a bit more pleasant for me.

By saying "communicating the actual semantics of a blocking resource", I
assume that you are suggesting to send something like "this is a request
initiated by a style tag in body." Generally speaking, I think signaling
that sort of signal is a good idea.

OTOH, that's going to be a research project. I am open to defining such
signals alongside "urgency" that's being proposed by the document. OTOH,
"switching" to that approach would mean that it'd be less likely that we'd
have an alternative prioritization scheme adopted when we ship H3.

Therefore, it is my view that what we should do now is encode the priority
levels that the browsers use today, at the same time assigning meanings to
each of the priority levels.

We need to assign meanings so that servers can tweak the prioritization
scheme, because the server needs to know what type of resource is assigned
to each level.

Consider the case where a server wants to send HTML before CSS (it's not a
terrible idea, that's what Chrome suggests using the H2 scheme now). That'd
be only possible when the client uses a signal like "document", "blocking".
If the signals were named like "highest" or "medium" (or "5" or "4"), and
without the knowledge of to which of the two HTML and CSS will be
associated to, it would be impossible for a server to prioritize HTML above
CSS (or in the opposite order).

>     Resolution to this can be as simple as re-naming the values to prevent

We can discuss about the names.

> server to client then? There doesn't seem to be a way in the current setup

You can set the urgency. For example, if the server believes that an image
is as important as the document itself in terms of "using" the document, it
can set the value of the response header to "Priority: urgency=document."
That would be the instruction to the H2/H3 terminator that the response
should be given the same priority level as the HTML.

>>> 1) I really like the header-based approach. It's extensible, allows us
>>> to make progress on H3, is easier to use/understand,
>>> can be "backported" to H2, has other nice properties which you touch
>>> upon in the text (intermediates, caching, etc.)
>>>
>>
>>> 2) I really dislike the header-based approach. It makes
>>> re-prioritization a mess. You don't touch upon this (yet) in the draft,
>>> really opposed to using the separate frame if that's the only option,
>>> but it still takes away some of the nice properties of 1)
>>>
>>
>>> 5) I am also not sure about the interpretation of the current fields.
>>> For example, as noted by Patrick (
>> When used by a client, it is a signal that indicates if the client
>> prefers receiving the responses with the bandwidth being distributed among
>> similar responses, or if it prefers receiving the response one by one.
>>
>> For images, a browser should set progressive to 1, when it can assume
>> that the image can be rendered progressively, as doing so improves user
>> experience.
>>

-- 
Kazuho Oku

---

From: Robin MARX <[email-masked]>
Date: Wed, 10 Jul 2019 11:19:01 +0200

Hello Kazuho,

I agree that more extensive signaling would be research project with a
medium/high turn around time.
I think that there might have been a mis-interpretation from my end on that
count though: I interpreted this draft as a way to be able to postpone the
prioritization to after H3 / not make finishing H3 dependent on it.
My assumption was that we would ship H3 with the draft-21 changes (aka:
very similar to H2's setup) and then augment/extend that with this approach
e.g., 6 months afterwards.

Instead, if I understand you correctly, it is the intent to finish this
together with H3 and expect browsers/clients to implement this together
with H3 as its default (and only?) standardized prioritization option.
As such, it is my fear that we would be trying to rush this, to have
"something" ready by the deadline, without properly testing or
"researching" how it would work or if it covers all use cases. Afaict,
that's going dangerously close to how we got the H2 setup in the first
place...
If it's the intent to have something simpler for H3 that allows for
rudimentary tie-ins for server-side prioritization and that works well with
existing browser setups, I feel Patrick Meenan's original proposal is far
superior and more flexible than the current proposal in the draft for this
purpose, while being nearly as simple/straightforward.
To be clear: we could still use the "HTTP header" approach, just not the
"urgency" and "progressive" aspects then (e.g., priority= level=63,
concurrency=3).
Later work after the "research project" is done could then add additional
header values to allow more use cases / semantics etc.

With best regards,
Robin

On Wed, 10 Jul 2019 at 03:25, Kazuho Oku <[email-masked]> wrote:

>> I have created some issues on github to further explain some of my
>> points, as I feel some of your comments don't really answer my other
>> reservations.
>>
>>     Resolution to this can be as simple as re-naming the values to
>> prevent confusion.
>>
>
>> 6) On the array of use cases / semantics that can be represented/should
>> be representable:
>>     You say for example that promoting an image to a hero image is not
>> the intent of the example. I can then ask: how would you indicate a
>> hero-image server to client then? There doesn't seem to be a way in the
>> current setup to (properly) do this.
>>
>
>>>> This really helps for (partial) server-side (re-)prioritization. It
>>>> does require the server (implementers) to know a bit more about how
>>>> browsers work, but
>>>> 4) I don't feel the current defined priority fields+values cover the
>>>> use cases though. You touch upon this in section 5.2, but I disagree with
>>>> you there:
>>>> and imo these should be nailed down (semi-)completely before
>>>> considering this approach. There should probably be more degrees of
>>>> "urgency" \
>>>> specific terms can make it more difficult to actually use this scheme
>>>> in practice.
>>>>
>>>
>>>> which you want to send before other images. However, it feels to me
>>>> that "abusing" the progressive field for this is not the best way to go
>>>> about that.
>>>>
>>>

-- 

Robin Marx

www.uhasselt.be

---

From: Kazuho Oku <[email-masked]>
Date: Thu, 11 Jul 2019 12:46:30 +0900

2019年7月10日(水) 18:19 Robin MARX <[email-masked]>:

> I think that there might have been a mis-interpretation from my end on
> that count though: I interpreted this draft as a way to be able to postpone
> the prioritization to after H3 / not make finishing H3 dependent on it.

Yes. That is the intent.

My view is that if we are to have something other than the H2-based design,
it's beneficial to do it before or as the H3 hits the market. This is
because then we could use H3 as the vehicle for providing something better,
at the same time easing the pain of people implementing the complex
H2-based scheme.

> "urgency" and "progressive" aspects then (e.g., priority= level=63,

I think that there is a slight misunderstanding here.

Assuming that the Patrick's proposal your are referring to is
https://github.com/pmeenan/http3-prioritization-proposal, concurrency is a
boolean parameter. In terms of functionality, it is identical to the
"progressive" parameter of the Priority header proposal.

Therefore, the only difference between the two proposals regarding how the
priorities are expressed are:
* if each urgency has a meaning
* the number of the urgency levels (3 vs. 64)

That said, I can see that just having 3 levels would be too restrictive for
browsers.

If we are to propose the header-based approach as "the" priority scheme for
H3, I think we should try to provide a way to transplant the urgency levels
(that are currently internal to the browsers) to the "urgency" parameter.

>>>     Simply clarifying what your "re-defined" blocking means does not
>>> help prevent confusion all that much imo.
>> Therefore, it is my view that what we should do now is encode the
>> priority levels that the browsers use today, at the same time assigning
>> meanings to each of the priority levels.
>>
>> Consider the case where a server wants to send HTML before CSS (it's not
>> a terrible idea, that's what Chrome suggests using the H2 scheme now).
>> That'd be only possible when the client uses a signal like "document",
>> "blocking". If the signals were named like "highest" or "medium" (or "5" or
>> "4"), and without the knowledge of to which of the two HTML and CSS will be
>> You can set the urgency. For example, if the server believes that an
>> image is as important as the document itself in terms of "using" the
>> document, it can set the value of the response header to "Priority:
>> urgency=document." That would be the instruction to the H2/H3 terminator
>> that the response should be given the same priority level as the HTML.
>>
>>
>>>     I feel we should communicate as much metadata as possible back to
>>> the server to allow it to make complex decisions, if it so chooses. The
>>> server can still ignore most of the metadata if it decides to implement a
>>> simpler scheme.
>>>>> For me, these new priority primitives are the core of the proposal
>>>>> (not the header-based approach. I would champion these new semantics in a
>>>>> to look at the existing work on the Priority Hints spec, seeing as
>>>>> they probably have already looked at much of this as well.
>>>>>
>>>>> blocking has a specific meaning and it doesn't map 100% to your
>>>>> current text.
>>>>>
>>>>

-- 
Kazuho Oku

---

From: Robin MARX <[email-masked]>
Date: Thu, 11 Jul 2019 12:15:16 +0200

Hello Kazuho,

Yes, you are right, my apologies. Patrick's proposal recently changed to
the approach proposed by Ian Swett in
https://github.com/quicwg/base-drafts/pull/2700 and I was still thinking of
the old version.

I see that you have a PR that splits urgency up into about 8 numerical
levels (https://github.com/kazuho/draft-kazuho-httpbis-priority/pull/34),
each with their own semantics and logic.
This feels a bit like SPDY priorities-on-steroids and is indeed a nice
middle ground between Patrick's 64 levels and your original 3. I still
don't like the naming of the groups, but to settle that I guess we will
need to resort to a duel at dawn ;)
I've left some notes on the PR adding some additional examples to the text
to further clarify intended usage. With something like that in, I feel it's
a workable proposal.

With best regards,
Robin

On Thu, 11 Jul 2019 at 05:46, Kazuho Oku <[email-masked]> wrote:

> My view is that if we are to have something other than the H2-based
> design, it's beneficial to do it before or as the H3 hits the market. This
> is because then we could use H3 as the vehicle for providing something
> better, at the same time easing the pain of people implementing the complex
> H2-based scheme.
>
>
> https://github.com/pmeenan/http3-prioritization-proposal, concurrency is
> a boolean parameter. In terms of functionality, it is identical to the
> That said, I can see that just having 3 levels would be too restrictive
> for browsers.
>
> If we are to propose the header-based approach as "the" priority scheme
> for H3, I think we should try to provide a way to transplant the urgency
> levels (that are currently internal to the browsers) to the "urgency"
> parameter.
>
>
>>>>     My point was more general that the semantics of what you have now
>>>> do not map (perfectly) to the semantics that are already in-place when
>>>> talking about browser resource loading and this can create confusion down
>>>> the road.
>>>>     You could say that the server simply uses its internal knowledge
>>>> and sends that image first, but then the whole "this is usable by
>>>>>> Thank you for the draft. Most of this feedback I've given elsewhere
>>>>>> as well, but to keep everything in one place:
>>>>>>
>>>>>> 1) I really like the header-based approach. It's extensible, allows
>>>>>> us to make progress on H3, is easier to use/understand,
>>>>>> using a separate, special H3/H2 level frame to provide this. This
>>>>>> just feels -very- dirty to me. The problem is that I can't really think
>>>>>> I don't really see that as a big issue (given that we provide
>>>>>> guidance and examples on proper options)
>>>>>>
>>>>>
>>>>> :+1:
>>>>>
>>>>>
>>>>>>
>>>>> I think that adding "deferred" is easy, assuming that we would agree
>>>>> on the meaning. It would mean the responses that should be sent _after_ the
>>>>>> https://github.com/kazuho/draft-kazuho-httpbis-priority/issues/12,
>>>>>> it might be good
>>>>> https://github.com/kazuho/draft-kazuho-httpbis-priority/pull/14.
>>>>> IIUC, Patrick's concern was that "blocks the processing of the document"
>>>>> seems to be applicable to JavaScript files loaded in <BODY>. In #14 we've
>>>>> changed the text to "blocks using the document", and I think it's clearer
>>>>> that urgency=blocking does not necessarily apply to such JavaScript files.
>>>>>
>>>>>
>>>>>> 7) Most of this comes together in the fact that I'm having a hard
>>>>>> time thinking of how to represent existing H2 logic/browser use cases in
>>>>>> this new scheme.
>>>>> I agree that talking about mappings is a good idea. I think it would
>>>>> be something like:
>>>>>
>>>>>>          Needs more work though, current draft doesn't (fully)
>>>>>> support use cases and imo, it should.

-- 

Robin Marx

www.uhasselt.be

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 12 Jul 2019 09:32:30 +0900

Hi Robin,

2019年7月11日(木) 19:15 Robin MARX <[email-masked]>:

> https://github.com/quicwg/base-drafts/pull/2700 and I was still thinking
> of the old version.
>

I'm glad to hear that, and more importantly, thank you for the detailed
feedback. I think that PR #34 is becoming much more mature thanks to your
comments, and I'm feeling more confident that it is the correct approach.

>> Therefore, the only difference between the two proposals regarding how
>> the priorities are expressed are:
>>> Later work after the "research project" is done could then add
>>> additional header values to allow more use cases / semantics etc.
>>>
>>> With best regards,
>>> Robin
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>>>     With your examples at the end as well, for me you are "abusing"
>>>>> the blocking indicator to manipulate sending behaviour, rather than
>>>> By saying "communicating the actual semantics of a blocking resource",
>>>> I assume that you are suggesting to send something like "this is a request
>>>> Consider the case where a server wants to send HTML before CSS (it's
>>>> not a terrible idea, that's what Chrome suggests using the H2 scheme now).
>>>>> 6) On the array of use cases / semantics that can be
>>>>> represented/should be representable:
>>>>>     In essence, the metadata communicated in the header should, imo,
>>>>> be relatively de-coupled from the concrete implementations at the server
>>>>> (and the current HTTP/2 setups)
>>>>>
>>>>> With best regards,
>>>>> Robin
>>>>>
>>>>>
>>>>>
>>>>>>> really opposed to using the separate frame if that's the only
>>>>>>> option, but it still takes away some of the nice properties of 1)
>>>>>>>
>>>>>>
>>>>>>> 5) I am also not sure about the interpretation of the current
>>>>>>> fields. For example, as noted by Patrick (
>>>>>>> Similarly, what does progressive mean exactly? IIUC JS and CSS can
>>>>>>> be parsed/compiled
>>>>>>> counted as "progressive"? How does the browser know if a JPEG will
>>>>>>> be progressive or not when making the request? I feel you've got the
>>>>>>> answers, but the usage of these
>>>>>>> specific terms can make it more difficult to actually use this
>>>>>>> scheme in practice.
>>>>>>>
>>>>>>
>>>>>> Assuming that what I've stated here makes sense, I think it might be
>>>>>> a good idea to clarify these points using examples in the draft.
>>>>>>
>>>>>>
>>>>>>>
>>>>>> There's some wiggle room for fonts and JS, CSS being used inside
>>>>>> BODY. Depending on how important they seem to be, a client can set urgency
>>>>>> to "blocking" (if it essentially prevents the document from being used), to
>>>>>>> A) a new way to define priority levels/semantics (i.e., "how will
>>>>>>> the resource be used " rather than "when should you send it") : all for
>>>>>>> this!

-- 
Kazuho Oku

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 12 Jul 2019 11:42:31 +0900

Hello Tom,

Thank you for your feedback. My comments inline.

2019年7月12日(金) 5:23 Tom Bergan <[email-masked]>:

> I agree with Robin's feedback. Echoing some of that feedback, plus adding
> my own:
>
> 1. It doesn't make much sense to have one mechanism for initial
> prioritization (HTTP headers) and another for reprioritization (H2/H3
> frames).
>

I feel somewhat strongly that these priorities should be carried by an HTTP
header field. Let me explain.

The first reason is that the hints are end-to-end. H2 priorities are
hop-to-hop, because the priority of each request is in fact a leaf of a
priority tree, which is a property of a connection. In contrast, our aim is
to move to "absolute" priorities, in which every request has an independent
associated value.

This associated value does not change hop-by-hop. Consider the case of an
HTTP request for a CSS file that is referred to from the HEAD section of an
HTML. The fact that the request originated from that section does not
change as it traverses through intermediaries. Such information could be
useful when prioritizing requests on a connection between the intermediary
and the server. Note than in H1, the only way of sending metadata is
through the use of header fields. I'd also point out that, typically,
caches contain only the header fields and the content. Therefore, for the
signal to traverse though H1 or to be cached, it has to be defined as a
header field.

Second reason is that the servers want to tweak priorities. As Robin points
out in his research [1], the optimal prioritization strategy differs
between websites. Therefore it is beneficial to provide the knobs for
tweaking.

In fact, proprietary examples already exist. Cloudflare defines a custom
header field to tweak H2 priorities [2]. We try to drive prioritization
based on content-type if we think that the client is not providing a good
priority tree [3]. However, lack of a standardized method to communicate
the hints from web applications to H2 terminators severely limits the
applicability and the granularity of server-driven tweaking.

This Priority header proposal is an attempt to solve that headache through
standardization.

Even on the client-side, the way to communicate prioritization has been a
problem. I think that Priority Hints is a good first step. By defining
priorities as absolute values, that can be represented as a string, we'd be
possible to communicate finer grained signal, even including
server-specific extensions. I'd argue that use of headers would be the
solution that is most easily and widely adoptable, because it can be
transmitted using the existing pipes for transmitting HTTP messages. This
is not just about browsers.

It's true that reprioritization needs to be communicated using a frame in a
HTTP-version-specific way. However, I do not see that as an issue, because
we can use one unified way of representing the hints, by using the
definition of the header value. What I mean is that H2/H3 frames can
transmit the header field string as-they-are, when reprioritization
happens. We have precedent for this approach: AltSvc frame [4].

> 2. Why not make urgency an sh-integer? This would avoid endless debates
> about how many priority levels are needed. If a server doesn't want to
> support 15 decimal digits
> <https://tools.ietf.org/html/draft-ietf-httpbis-header-structure-10#section-3.6> of
> priority, it can collapse that integer however it wishes. There is no MUST
> requirement to strictly follow the requested ordering.
>

Thank you for the suggestion. I think we are leaning towards having 8
urgency levels. As Robin has pointed out, we've been discussing that in
https://github.com/kazuho/draft-kazuho-httpbis-priority/pull/34.

Though I'd note that each urgency level needs to have a well-defined
meaning to let the server intervene with the priorities.

> 3. I'm not sure how I feel about Section 3, specifically "[...] a server
> is permitted to send a 'Priority' response header field.  When used, the
> parameters found in this response header field override those specified by
> the client." It seems quite brazen for the server to tell the client that
> they have prioritization wrong, when the client is the one who will
> actually use the resource.
>

It's about an origin server (or a cached content) correcting the H2/H3
terminator's understanding of how responses should be prioritized. I do not
think we state that the end client's (in this case the web browser's) view
is updated, but maybe we need some editorial work to avoid misunderstanding
here.

> 4. I think the "progressive" parameter is too closely tied to "progressive
> images". This was called "concurrency" in Pat's proposal
> <https://github.com/pmeenan/http3-prioritization-proposal#bandwidth-splitting>,
> which is a better name IMO. Concurreny=0 means "FIFO preferred" and
> concurrency=1 means "any order". Concurrency=1 is useful any time you don't
> care about the ordering, not only when the responses "can be processed
> progressively".
>

Thank you for the suggestion. How about "incremental"? I think "concurrent"
might sound like an expected action, rather than a property of how a
response is likely to be processed.

We have an issue that discusses this naming.
https://github.com/kazuho/draft-kazuho-httpbis-priority/issues/35.

>
> 5. In an ideal world, the concurrency parameter is not necessary at all.
>

I disagree.

If a resource should be concurrently served depends on if the client has
the possibility of incrementally processing the response, and if the
content that is served by the server is actually incrementally processable
by the the client.

Consider the case of an image. In case of a progressive JPEG file, it makes
sense to serve it incrementally, because providing rough image at an early
moment improves user experience. But if the image was in fact a baseline
(i.e. non-progressive) JPEG file, there is much less reason to serve it
incrementally, because having the ability to view a few pixels on top does
not help. In such case, with the proposed approach, the server, the only
actor with the knowledge of the file format, can correct the behavior of
the H2/H3 terminator for better performance.

[1] https://lists.w3.org/Archives/Public/ietf-http-wg/2019JulSep/0008.html
[2]
https://blog.cloudflare.com/better-http-2-prioritization-for-a-faster-web/
[3]
https://h2o.examp1e.net/configure/http2_directives.html#http2-reprioritize-blocking-assets
[4] https://tools.ietf.org/html/rfc7838#section-4

Resources with the same urgency would be served in an arbitrary order,
> possibly interleaved. Ordering would be expressed through urgency levels
> only. The concurrency parameter was introduced because it's difficult to
> encode lengthy sequential orderings in very few bits (7 bits in Pat's
> proposal). But with 15 digits, or ~53 bits, you can mimic Pat's proposal by
> using the 7 upper bits for urgency and the lower ~46 bits for the FIFO
> order. That is likely good enough.
>
> On Thu, Jul 11, 2019 at 3:19 AM Robin MARX <[email-masked]> wrote:
>
>> Hello Kazuho,
>>
>> I've left some notes on the PR adding some additional examples to the
>> text to further clarify intended usage. With something like that in, I feel
>> it's a workable proposal.
>>
>> With best regards,
>> Robin
>>
>>>> To be clear: we could still use the "HTTP header" approach, just not
>>>> the "urgency" and "progressive" aspects then (e.g., priority= level=63,
>>>> concurrency=3).
>>>>
>>>
>>> https://github.com/pmeenan/http3-prioritization-proposal, concurrency
>>> is a boolean parameter. In terms of functionality, it is identical to the
>>>>>> Thanks for the comments, most of which I agree with. The
>>>>>> clarification from Lucas elsewhere that the frame could contain an "opaque
>>>>>> encoded header" makes things a bit more pleasant for me.
>>>>> OTOH, that's going to be a research project. I am open to defining
>>>>> such signals alongside "urgency" that's being proposed by the document.
>>>>> OTOH, "switching" to that approach would mean that it'd be less likely that
>>>>> we'd have an alternative prioritization scheme adopted when we ship H3.
>>>>>
>>>>> We need to assign meanings so that servers can tweak the
>>>>> prioritization scheme, because the server needs to know what type of
>>>>> resource is assigned to each level.
>>>>>
>>>>>>     You say for example that promoting an image to a hero image is
>>>>>> not the intent of the example. I can then ask: how would you indicate a
>>>>>>>> can be "backported" to H2, has other nice properties which you
>>>>>>>> touch upon in the text (intermediates, caching, etc.)
>>>>>>>>
>>>>>>>
>>>>>>>> 4) I don't feel the current defined priority fields+values cover
>>>>>>>> the use cases though. You touch upon this in section 5.2, but I disagree
>>>>>>>> with you there:
>>>>>>> Re "background," I think we need to discuss how we want to
>>>>>>> prioritize them. Should we assign it a yet lower precedence (than
>>>>>>> "deferred")? Or should we state that it should be given some amount of
>>>>>>> bandwidth regardless of other responses?
>>>>>>>
>>>>>>> As mentioned in
>>>>>>> For images, a browser should set progressive to 1, when it can
>>>>>>> assume that the image can be rendered progressively, as doing so improves
>>>>>>> user experience.
>>>>>>>
>>>>>>> Servers would benefit from setting progressive to 0 for baseline
>>>>>>> JPEG images, assuming that sending the first few percent of the file does
>>>>>>> not improve user experience.
>>>>>>>
>>>>>>>> 6) This is also clear a bit from the example in section 3
>>>>>>>> (switching image from progressive to non-progressive). I -assume- the goal
>>>>>>>> here is to do something like a "hero" image,
>>>>>>>> E.g., how would you derive Chrome's current "dynamic fifo" from
>>>>>>>> this metadata? How would you implement Patrick Meenan's proposed 'ideal'
>>>>>>>> setup from these directives?

-- 
Kazuho Oku

---

From: Tom Bergan <[email-masked]>
Date: Thu, 11 Jul 2019 13:23:28 -0700

<https://tools.ietf.org/html/draft-ietf-httpbis-header-structure-10#section-3.6>
of

3. I'm not sure how I feel about Section 3, specifically "[...] a server is
permitted to send a 'Priority' response header field.  When used, the
parameters found in this response header field override those specified by
the client." It seems quite brazen for the server to tell the client that
they have prioritization wrong, when the client is the one who will
actually use the resource.

On Thu, Jul 11, 2019 at 3:19 AM Robin MARX <[email-masked]> wrote:

>>>>>> 2019年7月8日(月) 23:56 Robin MARX <[email-masked]>:
>>>>>>
>>>>>>> Hello Kazuho and Lucas,
>>>>>>>
>>>>>> the precedence of mis-prioritized requests under the proposed scheme..
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>> porting the prioritization scheme of H2 to H3 is the way to go [1]..
>>>>>>>>

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 12 Jul 2019 23:01:57 +0900

Hello all,

Thank you very much for all the discussion here and elsewhere.

Based on the feedback, Robin, Lucas and I have expanded the number of
urgency levels from 3 to 8, including a "background" level. From what we
have heard, this would be sufficient; 5 levels is what some browsers use
now (and we now provide 5 + background to them).

I hope that this gives them enough space to start with (we can subdivide
existing levels later), at the same time associating enough meanings to
each of the urgency levels so that the servers can tweak the prioritization
scheme.

We can't submit -01 now, but the up-to-date draft can be found at:

https://kazuho.github.io/draft-kazuho-httpbis-priority/draft-kazuho-httpbis-priority.html

The repository and the issue list can be tracked from
https://github.com/kazuho/draft-kazuho-httpbis-priority/.

2019年7月12日(金) 11:42 Kazuho Oku <[email-masked]>:

> I feel somewhat strongly that these priorities should be carried by an
> HTTP header field. Let me explain.
>
> Second reason is that the servers want to tweak priorities. As Robin
> points out in his research [1], the optimal prioritization strategy differs
> It's true that reprioritization needs to be communicated using a frame in
> a HTTP-version-specific way. However, I do not see that as an issue,
> because we can use one unified way of representing the hints, by using the
>> 4. I think the "progressive" parameter is too closely tied to
>> "progressive images". This was called "concurrency" in Pat's proposal
> Thank you for the suggestion. How about "incremental"? I think
> "concurrent" might sound like an expected action, rather than a property of
> how a response is likely to be processed.
>
> Consider the case of an image. In case of a progressive JPEG file, it
> makes sense to serve it incrementally, because providing rough image at an
> early moment improves user experience. But if the image was in fact a
> baseline (i.e. non-progressive) JPEG file, there is much less reason to
> serve it incrementally, because having the ability to view a few pixels on
> top does not help. In such case, with the proposed approach, the server,
> the only actor with the knowledge of the file format, can correct the
> behavior of the H2/H3 terminator for better performance.
>
>
>>> https://github.com/quicwg/base-drafts/pull/2700 and I was still
>>> thinking of the old version.
>>>
>>>>> My assumption was that we would ship H3 with the draft-21 changes
>>>>> (aka: very similar to H2's setup) and then augment/extend that with this
>>>>> approach e.g., 6 months afterwards.
>>>>>
>>>>> Instead, if I understand you correctly, it is the intent to finish
>>>>> this together with H3 and expect browsers/clients to implement this
>>>>> together with H3 as its default (and only?) standardized prioritization
>>>>> option.
>>>>>
>>>>
>>>> Yes. That is the intent.
>>>>
>>>>>>>     My point was more general that the semantics of what you have
>>>>>>> now do not map (perfectly) to the semantics that are already in-place when
>>>>>> By saying "communicating the actual semantics of a blocking
>>>>>> resource", I assume that you are suggesting to send something like "this is
>>>>>> a request initiated by a style tag in body." Generally speaking, I think
>>>>>> signaling that sort of signal is a good idea.
>>>>>>
>>>>>>>     I feel we should communicate as much metadata as possible back
>>>>>>> to the server to allow it to make complex decisions, if it so chooses. The
>>>>>>>>> Thank you for the draft. Most of this feedback I've given
>>>>>>>>> elsewhere as well, but to keep everything in one place:
>>>>>>>>>
>>>>>>>>> 1) I really like the header-based approach. It's extensible,
>>>>>>>>> allows us to make progress on H3, is easier to use/understand,
>>>>>>>>> of a (much) better solution, other than referring to
>>>>>>>>> re-prioritized resource A in resource B's headers, which is a whole other
>>>>>>>>> can of worms. I'm not
>>>>>>>>> This really helps for (partial) server-side (re-)prioritization.
>>>>>>>>> It does require the server (implementers) to know a bit more about how
>>>>>>>>> browsers work, but
>>>>>>>> I think that adding "deferred" is easy, assuming that we would
>>>>>>>> agree on the meaning. It would mean the responses that should be sent
>>>>>>>> _after_ the "non-blocking" responses.
>>>>>>>>
>>>>>>>>> in a streaming fashion in modern engines as well, it's mainly
>>>>>>>>> their application that is delayed until they are fully available.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Assuming that what I've stated here makes sense, I think it might
>>>>>>>> be a good idea to clarify these points using examples in the draft.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> which you want to send before other images. However, it feels to
>>>>>>>>> me that "abusing" the progressive field for this is not the best way to go
>>>>>>>>> about that.
>>>>>>>>>
>>>>>>>>
>>>>>>>> I agree that talking about mappings is a good idea. I think it
>>>>>>>> would be something like:
>>>>>>>>

-- 
Kazuho Oku

---

From: "Roy T. Fielding" <[email-masked]>
Date: Fri, 12 Jul 2019 11:53:53 -0700

> On Jul 12, 2019, at 7:01 AM, Kazuho Oku <[email-masked]> wrote:
> 
> Hello all,
> 
> Based on the feedback, Robin, Lucas and I have expanded the number of urgency levels from 3 to 8, including a "background" level. From what we have heard, this would be sufficient; 5 levels is what some browsers use now (and we now provide 5 + background to them).
> 
> I hope that this gives them enough space to start with (we can subdivide existing levels later), at the same time associating enough meanings to each of the urgency levels so that the servers can tweak the prioritization scheme.
> 
>     https://kazuho.github.io/draft-kazuho-httpbis-priority/draft-kazuho-httpbis-priority.html <https://kazuho.github.io/draft-kazuho-httpbis-priority/draft-kazuho-httpbis-priority.html>
> 
> The repository and the issue list can be tracked from https://github.com/kazuho/draft-kazuho-httpbis-priority/ <https://github.com/kazuho/draft-kazuho-httpbis-priority/>.
> 

Hi Kazuho,

This is starting to sound a lot like Waka's request priority (hi, lo, HiLo) and request context
[slide 9 of http://tools.ietf.org/agenda/83/slides/slides-83-httpbis-5.pdf <http://tools.ietf.org/agenda/83/slides/slides-83-httpbis-5.pdf> ]. I agree that defining
this as an e2e header field is a good way forward. HiLo is progressive with high priority for the
first block and then regular priority for the rest.

Note that there are other (security and privacy) reasons for explicitly communicating the context
of a request, since it can be used to detect some forms of fraud, phish, or transclusion.
However, that would need to be in a non-script-settable field.

We should also consider non-browsing priorities, such as indicating intentionally long-lived
responses that might have a low priority but do require some form of unbuffered regularity
(e.g., pongs). I don't think progressive is an independent parameter unless we intend
to communicate some sort of block size; otherwise, it is just saying that HoL blocking
isn't desired, which is not necessary to communicate beyond not setting "background".

In general, I like the approach, but I think the structured header syntax is the wrong choice.
Why send 26 bytes to communicate one (or maybe two) bytes of information?

....Roy

---

From: Tom Bergan <[email-masked]>
Date: Fri, 12 Jul 2019 12:12:34 -0700

Some more feedback inline.

> I feel somewhat strongly that these priorities should be carried by an
HTTP header field.
> The first reason is that the hints are end-to-end. H2 priorities are
hop-to-hop, because the
> priority of each request is in fact a leaf of a priority tree, which is a
property of a connection.
> In contrast, our aim is to move to "absolute" priorities, in which every
request has an
> independent associated value.

Prioritization is fundamentally hop-to-hop. If you have a request with two
hops, say, browser -> CDN -> origin, it's not clear how the
browser-declared priority should affect the CDN -> origin connection,
especially when that connection is shared by multiple browsers. I get that
the CDN -> origin connection might want to incorporate browser priorities
in some way, but the decision of how to do that should be made by the
*CDN*, not the *browser*.

> I think we are leaning towards having 8 urgency levels. ... Though I'd
note that each urgency
> level needs to have a well-defined meaning to let the server intervene
with the priorities.

If the latter sentence is the only rationale for having just 8 urgency
levels, then I disagree strongly. See below.

>> 3. I'm not sure how I feel about Section 3, specifically "[...] a server
is permitted to send a
>> 'Priority' response header field.  When used, the parameters found in
this response header
>> field override those specified by the client." It seems quite brazen for
the server to tell the
>> client that they have prioritization wrong, when the client is the one
who will actually use
>> the resource.
terminator's
> understanding of how responses should be prioritized.
> ...
> servers want to tweak priorities. As Robin points out in his research
[1], the optimal
> prioritization strategy differs between websites. Therefore it is
beneficial to provide the
> knobs for tweaking.

Peeking through Robin's paper, I believe the key idea is:

"Could we not make [an] argument that the server (usually) already has all
the resources for the web page and thus has a good overview from the start?
... Concretely, the hero images are given a higher server-side priority
than what they would normally receive from the client"

This is a fine argument. However, I believe this information belongs in a
web platform spec (such as HTML), not in an HTTP spec. Priority is not a
property of a resource in isolation. Priority is defined relative to
*other* resources that the client needs for its current task. The proposed
Priority header does not provide any of that context, therefore it's not
possible for the server to know how resources should be prioritized, except
in very specific cases.

There are many ways your proposal can go wrong. For example, suppose a
JavaScript file is used synchronously in <head> by web pages on some
domain. The server knows this and responds by using a high priority. Then
some new pages are added in which the script is still loaded in <head>, but
with <script async> or <script defer>. Now the server-specified priority is
wrong! When looking at the script request in isolation, the server does not
know how the script will be used, therefore it does not know how to
prioritize it.

>> 4. I think the "progressive" parameter is too closely tied to
"progressive images". This
>> was called "concurrency" in Pat's proposal, which is a better name IMO.
Concurreny=0
>> means "FIFO preferred" and concurrency=1 means "any order".
Concurrency=1 is useful
>> any time you don't care about the ordering, not only when the responses
"can be
>> processed progressively".
>
"concurrent" might sound
> like an expected action, rather than a property of how a response is
likely to be processed.
> https://github.com/kazuho/draft-kazuho-httpbis-priority/issues/35.

Thanks, I will comment on the issue.

> If a resource should be concurrently served depends on if the client has
the
> possibility of incrementally processing the response, and if the content
that is
> served by the server is actually incrementally processable by the the
client.

I think I did a poor job explaining my argument.

Your current proposal has 4 urgency bits and one concurrency bit (aka
"progressive" or "incremental"). I am saying that we can default to
concurrency=1 given enough urgency bits. For example, given ~53 bits (using
an sh-integer), we can define the following translation:

func TranslatePriority(urgency4bits int, progressive bool) uint64 {
  if progressive {
    return urgency4bits << 49
  } else {
    seq := nextSeq[urgency4bits]  // where nextSeq[x] is initialized to
2^49-1
    nextSeq[urgency4bits]--
    return (urgency4bits << 49) | seq
  }
}

This is almost equivalent to your proposal, except for caveats about how
(urgency=X, progressive=false) is ordered with respect to (urgency=X,
progressive=true). Those caveats seem unimportant in practice.

---

From: Patrick Meenan <[email-masked]>
Date: Fri, 12 Jul 2019 15:36:03 -0400

I'm somewhat opposed to defaulting to concurrency=1 and using a lot of bits
for a priority to handle sequencing because it introduces the problem of
needing to know what other requests are in-flight and their priorities to
set the appropriate sequence (the state that I was largely trying to get
rid of when we implemented Cloudflare's prioritization).

By using absolute priorities tied to the request/response itself the
browser rendering engine, the CDN back-ends and even the origin server
application code, all of which are well outside of the connection itself,
can influence and adjust the priorities of resources at an absolute level
and still trust that they will be delivered in a reasonable order when it
gets to the code that is actually managing the connection. Otherwise you're
going to end up having an absolute scheme defined on either end (like the 5
levels internal in Chrome and Cloudflare's header-based scheme) and doing a
translation at the wire layer anyway.  If we encode an absolute scheme
directly then the clients on either end can eventually migrate to using it
directly.

It's also not necessarily strictly point-to-point.  At my origin I may
decide to implement a query-param-based scheme for prioritization for
resources that are always the same (template code in the head of my pages
for example) where the server is configured to always set the priority for
those as "critical script", independent of what the browser requested. The
CDN in the middle doesn't need to know the application-specific knowledge
that I created between the front-end code and the origin, it just needs to
cache the priority headers along with the resources and respect whatever
the origin responded with.

For the security contexts and other uses for knowing WHY a specific
resource is important, I'd strongly encourage not trying to overload the H3
priority scheme with that data and leave that to the fetch metadata spec
that Mike West is working on.

---

From: Tom Bergan <[email-masked]>
Date: Fri, 12 Jul 2019 13:26:41 -0700

On Fri, Jul 12, 2019 at 12:36 PM Patrick Meenan <[email-masked]> wrote:

> I'm somewhat opposed to defaulting to concurrency=1 and using a lot of
> bits for a priority to handle sequencing because it introduces the problem
> of needing to know what other requests are in-flight and their priorities
> to set the appropriate sequence (the state that I was largely trying to get

There's state on the client, but it's a simple table. There's no state on
the server, unless you're expecting the server to reverse-translate the
priorities, which seems unnecessary (see below). That said, I don't feel
too strongly about this.

> directly then the clients on either end can eventually migrate to using it

I feel more strongly about this: If the server wants to suggest a
prioritization for subresources, I don't follow why the best place to do
that is in the HTTP response for the subresource itself, rather than in the
preceding HTML response. For example, we might define link elements like:

Link: rel=priority href=foo.js priotity=whatever

The HTML's response header can include link elements like this one. The
"whatever" can be some globally-understood thing, where the browser can
translate "whatever" to underlying H2/H3 priorities however it wants.

> the origin responded with.

It's very roundabout to inject a priority into HTML, route that through a
CDN, to an origin server (via a subresource query param), then communicate
that back to the CDN via a response header. Not to mention:

- What about <link rel=prefetch>? You can't use different query params,
because that would break the prefetch.
- What if your page is loaded in an iframe within some other page? Your
server-assigned priority might be correct in the iframe context, but not in
the whole-page context if the browser has lowered the iframe's priority.
- What if your page is loaded in a background tab? Maybe the browser wants
to lower the priority of background tabs. But it can't, if the server will
override that decision.

The Link proposal above appears to be simpler and more flexible. The
browser might want this information anyway, for other reasons, such as to
throttle cache lookups.

---

From: Lucas Pardue <[email-masked]>
Date: Sat, 13 Jul 2019 00:27:29 +0100

Hi Tom,

Thanks for your sharing your thoughts. I'd like to respond in more direct
detail but fear I might tread on the conversation toes.

To talk around a general topic of discussion, I've been noodling the
philosophy of priorities today. One thing that emerges to me is that a
client parsing a document has a _vague idea_ what the resource is, it can
determine some information from link relation, element, file extension etc
but it is only a partial picture. Clients typically assign their H2
priorities at this stage, based on the intended usage. What they can't know
exactly is the selected representation [1], which is determined by the
server. For example, whether a JPEG image is encoded as progressive or
baseline. The nature of the selected representation may affect how the
client would use it, and it is challenging for a server to reflect this
back into all the flavours of H2 priority tree that exist. And although a
client could perhaps apply reprioritisation to the in-flight response
(after it learns information), chances are that it will happen too late.

Since the selected representation is chosen during the lifetime of an
end-to-end connection, it is difficult to define a static signal that can
work for all. So my opinion is that the proposal is less about one side
running roughsod over the other, but all sides attempting to paint a fuller
picture using a common set of accessible terms. Of course this might go
askew, your example of a background priority getting "upgraded" has come up
- there is space in the spec for this to be addressed IMO.

Kinds regards
Lucas

[1] https://tools.ietf.org/html/rfc7231#section-3

---

From: Kazuho Oku <[email-masked]>
Date: Mon, 15 Jul 2019 21:35:02 +0900

Roy, thank you for your comments. My responses inline.

2019年7月13日(土) 3:54 Roy T. Fielding <[email-masked]>:
> The repository and the issue list can be tracked from https://github.com/kazuho/draft-kazuho-httpbis-priority/.
>
> [slide 9 of http://tools.ietf.org/agenda/83/slides/slides-83-httpbis-5.pdf ]. I agree that defining

Thank you for sharing the slides. It's reassuring to know that you had
thought about something similar, and still think that it's preferable.

HiLo is interesting. Am I correct in assuming that the intended
primary use case is to transmit the size of the images so that the
browsers can determine the layout at an earlier moment?

While I think that such signal is useful to some extent, I am not sure
how useful it is.

Partly because it is not the best way of notifying the client the
metadata at the earliest moment. In case of images, we typically embed
the size of the image either in the CSS or the IMG tag, so that the
client would not need to wait for the first few bytes of the image.

The other reason is that HiLo does not communicate the reason the
client thinks the first few bytes are important. If we are to have
something like HiLo, I think it would be better to have a richer
signal that tells the server why and / or what needs to be sent ASAP.

Considering these aspects and the fact that the H2 prioritization does
not have this type of signal, I think we can start without having HiLo
in the Priority header.

> However, that would need to be in a non-script-settable field.

I agree that such filtering rule would help us to some extent. Though
it would be fragile regardless of the header field defined as
non-script-settable, considering the fact that clients that do not
implement the Priority header extension could pass through the header
field set by the application.

> (e.g., pongs).

I'd argue that one of the non-background urgencies should be used for
a request that require a timely response (e.g., ping-pong).

Maybe the issue in the current draft is that, as others have pointed
out, the terms "non-background" urgencies are too browser-centric. It
might be a good idea to consider renaming "document" to "default" at
least.

> I don't think progressive is an independent parameter unless we intend

I'd argue that the progressive parameter is not related to HoL
blocking. It's rather a way to signal if the user experience would
improve by interleaving the response with others. The intended
use-case is for images that support progressive rendering, as IIUC
generally assumed that interleaving between the responses that carry
images that can be rendered progressively improves user experience.

> Why send 26 bytes to communicate one (or maybe two) bytes of information?

Honestly speaking I wouldn't mind choosing something else than
Structured Headers as long as the alternative is extensible. At the
same time, I do not think 26 bytes is an issue because I'd assume that
it would be compressed into just a few bytes with HPACK / QPACK.

-- 
Kazuho Oku