Subject: Re: Comments on Extensible Prioritization Scheme for HTTP

From: Glenn Strauss <[email-masked]>
Date: Sat, 9 Apr 2022 07:25:57 -0400

On Thu, Mar 31, 2022 at 05:41:09PM +0200, Willy Tarreau wrote:
> On Thu, Mar 31, 2022 at 11:15:29AM -0400, Glenn Strauss wrote:
> > On Thu, Mar 31, 2022 at 03:20:50PM +0100, Lucas Pardue wrote:
> > > Instead, an HTTP/2 server that send SETTINGS_NO_RFC7540_PRIORITIES=1
> > > is most likely going to support Extensible Priorities because there
> > > is nothing else.
> > 
> > I think the logic of that statement is the strongest argument that
> > the setting name is indirect and suboptimal.

I continue to strongly disagree with using the flag as an implication.

Rather than SETTINGS_NO_RFC7540_PRIORITIES=1, I think a more logical
approach is to use affirmative signals, as suggested in
https://datatracker.ietf.org/doc/html/draft-lassey-priority-setting-00
which proposed SETTINGS_ENABLE_HTTP2_PRIORITIES.

I think the above should apply to SETTINGS_ENABLE_PRIORITY_UPDATE.

Should PRIORITY_UPDATE prove to be more widely implemented than
PRIORITY, then use of PRIORITY by clients will phase itself out without
further action, and even without the need for something like
SETTINGS_ENABLE_HTTP2_PRIORITIES.

Prioritization is flexible, and the server may ignore the priority
signals provided by the client.

I am an implementer.  My server (lighttpd) parses PRIORITY frames
to comply with RFC7540 (HTTP/2), but otherwise ignores them.
lighttpd plans to continue to ignore PRIORITY frames.

For widest client compatibility, lighttpd also plan to omit sending
SETTINGS_NO_RFC7540_PRIORITIES=1

https://mikebishop.github.io/http-misc-extensions/draft-bishop-httpbis-grease.html
"HTTP/2 [HTTP2] frame types and settings employ a similar mechanism of
 ignoring unknown values. This makes HTTP/2 a good candidate to employ
 grease on connections. The need for such a technique was demonstrated
 recently by an HTTP/2 implementation which closed the connection upon
 receipt of an unknown setting."

*If* the proposal for PRIORITY_UPDATE included an affirmative signal,
such as the client sending SETTINGS_ENABLE_PRIORITY_UPDATE=1, then the
server could be more confident that the client would likely understand
the same setting SETTINGS_ENABLE_PRIORITY_UPDATE (0/1) sent by the
server back to the client.

Currently, SETTINGS_NO_RFC7540_PRIORITIES=1 is proposed for client to
tell the server that the server need not keep RFC7540 PRIORITY state,
but existing clients do not send that, and many also do not utilize
PRIORITY.  If PRIORITY gets deprecated, then over time, the sending of
SETTINGS_NO_RFC7540_PRIORITIES=1 will end up being wasted bytes.
However, the proposed use of SETTINGS_NO_RFC7540_PRIORITIES=1 overloads
the setting with an *indirect* meaning to use PRIORITY_UPDATE, and I
disagree with this overloading.

> > then why not have the setting be to communicate directly that the
> > server supports PRIORITY_UPDATE?
> 
> Probably because it was considered that the vast majority of
> implementations would advertise the two at once anyway, and that the
> extra cost for those willing to implement nothing is close to zero.

One data point: as justified above, lighttpd plans to omit advertising
what I do not think needs to be advertised.

> > On Thu, Mar 31, 2022 at 04:04:15PM +0200, Willy Tarreau wrote:
> > >>    Similarly, if the client receives SETTINGS_NO_RFC7540_PRIORITIES with
> > >>    value of 0 or if the settings parameter was absent, it SHOULD stop
> > >>    sending PRIORITY_UPDATE frames (Section 7.1), since those frames are
> > >>    likely to be ignored.
> > >
> > >I'm personally not seeing a problem here. What it says is that if the
> > >peer announces that it doesn't want to abandon the default behavior,
> > >the one seeing this must respect this and not switch to the alternate
> > >one.
> > 
> > I got confused with the indirect logic.
> > 
> > As above, because of the overloading and inverted logic, there is no way
> > for the server to communicate exactly what it supports if the server
> > supports both.
> 
> Maybe the essential point you forgot is that once the server sends its
> settings, it's too late for the client to make a different choice because
> usually the client will send preface + settings + one or a few requests,
> then all the server can do is confirm the client's choices. As such the
> client will not wait for the server's response before starting to send.
> I guess that if the client sends the setting to 1 and the server doesn't
> send anything or sends it to zero, most likely the client will just
> refrain from sending PRIORITY frames (since it promised not to via the
> initial setting) and it will not send PRIORITY_UPDATE frames either since
> the server didn't mention accepting them.

BTW, during an initial connection open, a client can send PRIORITY
information as part of HEADERS by setting the PRIORITY flag.  There is
little need to send PRIORITY frames that early in the connection.
Also, during an initial connection, the client can open request streams
in (basic) order of priority, should the client have more than one
stream to open that early in the connection.

A client which is optimizing for bandwidth might omit sending
PRIORITY_UPDATE frames with the initial connection until after
receiving server connection preface, unless the client has previously
connected to the server and the server previously indicated
affirmative support for PRIORITY_UPDATE (indirect and not required
with the current proposal from Pardue).

(SETTINGS_NO_RFC7540_PRIORITIES=1 is not required, since clients and
 servers should ignore unknown settings)

> > A client might or might not send SETTINGS_NO_RFC7540_PRIORITIES.
> 
> Then in this case the client wants to stay on the RFC7540 behavior.

That is not optimal for the server (if the server implements PRIORITY),
since the server might not want to unnecessarily keep the priority
state if the client does not plan to use it.

PRIORITY is not widely implemented according to Pardue's proposal.
A client could simply *not send* SETTINGS_NO_RFC7540_PRIORITIES
and *not send* PRIORITY frames.  The server is then left keeping
unnecessary state.  Then again, if PRIORITY is deprecated, the
server might choose to ignore PRIORITY.

tl;dr: if PRIORITY is not widely implemented, then
SETTINGS_NO_RFC7540_PRIORITIES=1 would be better named
SETTINGS_ENABLE_PRIORITY_UPDATE=1 for an affirmative signal
that the client intends to use PRIORITY_UPDATE.  The server
might (optionally) confirm with SETTINGS_ENABLE_PRIORITY_UPDATE=1.

? Do other implementers think that PRIORITY_UPDATE frames will be sent
frequently enough to warrant SETTINGS_ENABLE_PRIORITY_UPDATE=1?
Might these settings simply be omitted, and PRIORITY deprecated?
(and SETTINGS_NO_RFC7540_PRIORITIES=1 omitted, too?) Since PRIORITY is
not widely implemented, does there need to be a mechanism to disable it?
Perhaps a future revision to HTTP/2 can simply deprecate PRIORITY.

Cheers, Glenn

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 9 Apr 2022 14:37:56 +0200

> signals provided by the client.

Absolutely, which is why for us as servers, this doesn't change much.
What really matters is how to provide the information the client needs
to act optimally.

> SETTINGS_NO_RFC7540_PRIORITIES=1

HAProxy doesn't implement priority either and currently sends nothing.
However once this draft becomes standard, I plan to send this setting
to inform compatible clients that they do not have to expect any
particular behavior from it on this point. It's not a matter of client
compatibility, H2 agents are required to ignore unkonwn settings, so
those unaware of this setting will not change anything, while those
aware of it will benefit from it.

> server back to the client.

But for what purpose ? Why would the server which does not implement
priority care about what the client thinks ? *If* there would be any
benefit for a client in knowing that a server doesn't implement either
scheme, I would agree with you and would rather ask for two settings
or for a versioned one (e.g. advertise what scheme is supported). But
it doesn't seem like client implementers have requested that level of
distinction and many of them are represented in this working group.
Thus as a server implementer I'm just trusting them, if they think a
single setting is fine I'm not going to impose them a second one.

> PRIORITY.

The most important is that many clients *do* support priority, that
these ones are already deployed and working fine, and that you cannot
suddenly break them by telling them "ah, you didn't read the news, the
default value recently changed and while you used to comply with the
standard now you violate it".

> If PRIORITY gets deprecated, then over time, the sending of

Yes that's true. And probably that advertising the desired priority
scheme number instead would be better, with 0=not supported,
1=default=RFC7540, 2=priority_update, and so on. At least these bytes
would be re-usable for future schemes. But you cannot change the default
that works fine between a non-negligible set of clients and servers.

> disagree with this overloading.

It could indeed be split or turned into an enum for more long-term
viability, but I can definitely understand why implementers do not feel
the need for two settings for the same switch in their code.

> what I do not think needs to be advertised.

But this is where you're getting it wrong in my opinion. Just like me with
haproxy, you're having an INCOMPLETE IMPLEMENTATION. We both have some work
left to be done. This extension offers us an easy path to advertise that
we did not implement that part, instead of having to implement it, and
you're declining the offer. That's not consistent, I'm afraid. Yes I would
like to see a slightly different approach (e.g. implementation type as I
suggested above) but quite frankly, if I can already avoid to implement
all that non-optimal scheme at the expense of sending one byte, I will
do it, regardless of the setting's name! And for the long term I would
like to implement the new scheme (or at least parts of it) because it
looks much better to me.

For me what matters is what client implementers think. If a client tells
us that they would benefit from knowing the total lack of support versus
support for the new scheme, then a distinction would be better. That's up
to them.

>  servers should ignore unknown settings)

It's not "should ignore" but "MUST ignore", and that's exactly what
encourages to advertise what you want to do, so that the peer that
know the setting are totally informed. That setting alone conveys two
information, one being "I will not send you PRIORITY frames nor require
that you build the dependency tree" and the other one being "I may
however send you some PRIORITY_UPDATES". It doesn't mean that it will
send them, just that it may.

> state if the client does not plan to use it.

That's precisely the reason for sending this setting.

> PRIORITY is not widely implemented according to Pardue's proposal.

"not widely implemented" is a proof of being suboptimal and the reason
for proposing a different approach, in hope that it will be much more
widely implemented. It doesn't mean we should punish all those who did
their job well (i.e. major browsers and servers) by suddenly declaring
them non-compliant or at least making them misbehave when facing an
updated peer!

> server might choose to ignore PRIORITY.

The way you present it makes me feel like you imagine that all clients
and servers are replaced atomically in the whole world. That doesn't
work like this and is the reason why the working group focuses so much
on interoperability. The proposal here makes sure that a compliant
client talking to a compliant server can be upgraded in any order
without causing trouble to each other. Your proposal would for example
cause trouble to some applications if servers are upgraded before
clients, thus would force servers to update very late, and would
needlessly waste resources in non-upgraded servers for all upgraded
clients.

With the current proposal, instead, servers can be upgraded very
quickly, not changing the behavior with existing clients, then each
upgraded client benefits from this knowledge and at the same time can
save some server resources.

That's why one must never change default values once adopted, and why
the default set must be well thought before a standard is released!

Regards,
Willy

---

From: Glenn Strauss <[email-masked]>
Date: Sat, 9 Apr 2022 09:58:16 -0400

> aware of it will benefit from it.

It would be trivial for lighttpd to unconditionally send
SETTINGS_NO_RFC7540_PRIORITIES=1 with server connection preface.

However, I am choosing not to do so (at least for now) due to:
https://mikebishop.github.io/http-misc-extensions/draft-bishop-httpbis-grease.html
"The need for such a technique was demonstrated recently by an HTTP/2
 implementation which closed the connection upon receipt of an unknown
 setting."

I do not know which client this was or how widely it is used, but it is
mentioned in the grease draft as a good reason supporting greasing
HTTP/2 SETTINGS.

I have completed implementing PRIORITY_UPDATE on my public dev branch.
I want to avoid a lighttpd upgrade breaking an existing client when
PRIORITY_UPDATE is not yet standardized.

> standard now you violate it".

Prioritization signals may be ignored by the server.  Clients will not
"break".  SETTINGS_NO_RFC7540_PRIORITIES=1 is not standardized, so it is
to be expected that most/all current clients do not send that setting,
and it is unknown when they will be upgraded, and when upgraded,
unknown if this will be a required setting.  For server implementations
which implement PRIORITY, this may be a missing signal.

Will a future revision of HTTP/2 include this signal, saying that a
client SHOULD send SETTINGS_NO_RFC7540_PRIORITIES=1 in the client
connection preface if the client does not intend to use RFC7540
PRIORITY?  Or will a future revision of HTTP/2 deprecate PRIORITY?

> that works fine between a non-negligible set of clients and servers.
...
> the default set must be well thought before a standard is released!

I agree with you that this setting might be better as an advertisement
of bitflags, supporting up to 32 different prioritization schemes,
with 0 indicating none supported.  (bitflags in one SETTING would be an
alternative to separate SETTINGs to enable PRIORITY, PRIORITY_UPDATE.)

That would suggest the server send something akin to
SETTINGS_PRIORITIZATION_SCHEMES
  0x0 none
  0x1 PRIORITY_UPDATE
  0x2 PRIORITY
  0x4 ???
  0x8 ...
and the flags can be OR'd together.

If the client sent SETTINGS_PRIORITIZATION_SCHEMES, that would be a hint
to the server, especially if the bitflag for PRIORITY were omitted so
that the server knows affirmatively that the server need not keep
PRIORITY state.

Absense of the setting from from server connection preface would
indicate RFC7540 defaults or could be affirmatively indicated with
  SETTINGS_PRIORITIZATION_SCHEMES=2

In the above
  SETTINGS_PRIORITIZATION_SCHEMES=1 == SETTINGS_NO_RFC7540_PRIORITIES=1
but 
  SETTINGS_PRIORITIZATION_SCHEMES=0 indicates no prioritization,
    as opposed to SETTINGS_NO_RFC7540_PRIORITIES=0 which indicates
    RFC7540 defaults, and could be equally achieved by omitting
    SETTINGS_NO_RFC7540_PRIORITIES=0.

Willy, I like your proposal to make the setting a more bona fide
advertisement and hope that the PRIORITY_UPDATE proposal authors will
take another look at SETTINGS_NO_RFC7540_PRIORITIES

https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-priority-12#section-2.1
   The value of SETTINGS_NO_RFC7540_PRIORITIES MUST be 0 or 1.  Any
   value other than 0 or 1 MUST be treated as a connection error (see
   Section 5.4.1 of [HTTP2]) of type PROTOCOL_ERROR.  The initial value
   is 0.

Cheers, Glenn

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 9 Apr 2022 18:20:22 +0200

> mentioned in the grease draft as a good reason supporting greasing

It can happen from time to time on early implementations, which is why
greasing is good. But usually such implementations don't last long.
Several times we noticed trouble with websocket a long time ago and H2
more recently that were caused in fact by anti-malware programs plugged
into browsers and inspecting the stream. Having the user disable them
was enough to fix the trouble, which was likely sufficient to report
the issue to the vendor and have it fixed soon. It turns out that the
whole web mostly runs on very few implementations (both clients and
servers), and if you have just 2-3 of them that induce breakage with a
non-compliant one, that last one is urged to get fixed otherwise it will
become so much unusable that it will disappear.

> PRIORITY_UPDATE is not yet standardized.

You shouldn't worry about this, really. Implementations that break with
lighttpd will have trouble visiting many sites, and would probably have
trouble with some sites already. It's more difficult when you deal with
embedded clients (software updates from cars, IoT devices, payment
terminals etc) that are not updated often, because you may have to add
an option to disable this. But quite frankly, you'd rather implement
what you need to improve your users' experience than live in the past by
fear of breaking already broken clients. I'm personally very careful about
not breaking stuff, and despite this we continue to move forward. A few
times we've met trouble with extremely popular clients that were very
quickly resolved, either on our side if we were faulty, or on the browser
side when it was their turn, and in the end users were very pleased by
the reactivity and already don't remember about this.

The real risk you're facing by implementing something that is not standardized
is to have to change some frame IDs or the behavior id these were to change
later. That's a risk I'm willing to take during development but not in LTS
versions, even though it's extremely low.

> "break".

They can notice a serious difference. I've seen this exactly once. One
hosting provider hosts hundreds of applications among which one has much
worse load time in H2 than H1 through haproxy, and they verified that when
joining the server directly, priorities would address this. Thus they
disabled H2 for this specific application and that was all. That's what
I mean by "breaking". If the application is designed to firmly rely on
priorities, it may be degraded. And in my opinion it's not acceptable
to cause that trouble with perfectly complying software.

>  SETTINGS_NO_RFC7540_PRIORITIES=1 is not standardized, so it is

Sure but if the client sends this it also stops expecting the regular
priorities to be used. That's the point.

> PRIORITY?  Or will a future revision of HTTP/2 deprecate PRIORITY?

We don't really care. Clients and servers *are* expected to support
PRIORITY. It's not an option in the protocol, it's not negotiated.
Without a specific extension like this draft, both sides are expected
to implement and support it. It's just that it's easy to silently ignore
and complex to implement with little return on investment and many of
us delayed that implementation to the point of no more intending to
implement it.

> alternative to separate SETTINGs to enable PRIORITY, PRIORITY_UPDATE.)

Yes I also thought about a bit flag that would do it fine to let both
sides decide which one to agree on.

> and the flags can be OR'd together.

I would personally swap 1 & 2 to keep them in chronological and/or
preference order. I.e. AND the two peers' flags and pick the topmost
bit. But that's a detail.

> that the server knows affirmatively that the server need not keep

Definitely.

> take another look at SETTINGS_NO_RFC7540_PRIORITIES

I agree that it would be more durable and a bit cleaner, but quite
frankly, I don't expect this to be that big a game changer, because I
wouldn't be surprised if in 10 years we'll only have H1 and H3 (or H4),
i.e. legacy vs low-latency, thus I suspect that this setting will not
evolve further.

>    Section 5.4.1 of [HTTP2]) of type PROTOCOL_ERROR.  The initial value

Indeed this one could be changed to allow other values, in the spirit
of forward compatibility and greasing.

Willy

---

From: Lucas Pardue <[email-masked]>
Date: Sat, 9 Apr 2022 17:29:51 +0100

Hey,

I agree with the majority of Willy says here.

I want to reiterate that the draft-ietf-httpbis-prioirty isn't a proposal,
it has WG and IETF consensus, and IESG approval to be published as an RFC.
It will be published any day soon. I do not believe the discussion points
here are substantial enough, or provide sufficient new information, to
warrant reopening the design aspects with the working group and running
through the entire process all over again.

7540bis has deprecated stream priorities; see
https://httpwg.org/http2-spec/draft-ietf-httpbis-http2bis.html#section-5.3.2.
This too has WG and IETF consensus, and approval to be published. 7540bis
along with a large number of other HTTP-related documents are in late
stages of such as AUTH48. Revisiting Extensible Priorities risks adding
more delays and I don't think there is reasonable motivation to do that.

Extensible Priorities does not prevent a subsequent draft being made that
invents a priority scheme negotiation mechanism that will work with HTTP/2
and HTTP/3. Proponents of doing so can run the process of gathering
interest and seeking adoption. Personally, I'm pessimistic such effort is
worth it until there is an actual alternative scheme that would need it.

Cheers
Lucas