Subject: Priority Placeholders in HTTP/2 and HTTP/QUIC

From: Mike Bishop <[email-masked]>
Date: Tue, 25 Jul 2017 19:18:16 +0000

This regards issue 441 in HTTP/QUIC.  For background, HTTP/2 allows the use of closed streams in the priority tree.  While not formally specified, a pattern has emerged in HTTP/2 of using placeholders – implicitly-closed streams, RFC 7540 section 5.1.1 – as root nodes for priority groups.  This is less-easily done in HTTP/QUIC, since the use of streams is supposed to be in sequence, and has no implicit-close rule.  (QUIC really can’t have an implicit-close rule, since STREAM frames might arrive out of order.  Instead, QUIC currently has an implicit-open rule, though that too is currently under discussion.)  It gets even worse that you can chain up to/through actual requests that have already been fulfilled, and you don’t know how long you need to maintain those states.  If the server discards state the client was expecting to keep, the peers can get into really divergent views of what the priority tree is.

I see two paths forward, depending how aggressive we want to be.  If we want to just keep the HTTP/2 model as-is, we could make it explicitly legal for a client to close a stream with no data.  Implementations can informally prioritize maintaining priority state for these as likely placeholders.  Leaves the same potential state explosion, but since it’s all advisory, servers can still discard any of these that aren’t being used if the client goes overboard.

We discussed this briefly in Prague, and concluded that we didn’t want to do a wholesale replacement of the priority scheme, but were interested in a solution of some kind.  There was also some sadness expressed about the unbounded state commitment in the above situations.  We talked about allowing the server to specify a number of placeholders it’s willing to permit the client, and add a flag to the PRIORITY frame saying that the ID given is a placeholder ID rather than a Stream ID.  This is more of a departure, but bounds the state explosion and allows the server to be more aggressive about pruning closed streams.  Our principle, reaffirmed in Prague, was that departures from HTTP/2 which aren’t forced by QUIC need to come from the HTTPbis WG.

I wrote a quick draft casting the second option as an HTTP/2 extension.  Because HTTP/2 implies the stream to be prioritized by sending the PRIORITY frame on that stream, I needed to mint a separate frame to prioritize the placeholders themselves.  Not coincidentally, it’s the HTTP/QUIC PRIORITY frame.  😊  The HTTP/QUIC incarnation of this would add an additional flag to the PRIORITY frame indicating whether the prioritized stream is a placeholder, just as this extension adds to the dependency.

What do folks think?

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Tuesday, July 25, 2017 11:58 AM
To: Mike Bishop <[email-masked]>
Subject: New Version Notification for draft-bishop-httpbis-priority-placeholder-00.txt

A new version of I-D, draft-bishop-httpbis-priority-placeholder-00.txt
has been successfully submitted by Mike Bishop and posted to the IETF repository.

Name:		draft-bishop-httpbis-priority-placeholder
Revision:	00
Title:		Priority Placeholders in HTTP/2
Document date:	2017-07-25
Group:		Individual Submission
Pages:		7
URL:            https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Finternet-drafts%2Fdraft-bishop-httpbis-priority-placeholder-00.txt&data=02%7C01%7Cmichael.bishop%40microsoft.com%7C6ace0f1cf1594ec919ac08d4d38f0770%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636366058645292697&sdata=qNdBmBlsbjHS2e9LLNSB8NbwVz9H%2B7HK8TaA%2BHpixTA%3D&reserved=0
Status:         https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-bishop-httpbis-priority-placeholder%2F&data=02%7C01%7Cmichael.bishop%40microsoft.com%7C6ace0f1cf1594ec919ac08d4d38f0770%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636366058645292697&sdata=ieLs1GA0dVTtSAhazJ%2BC3OeEPvaYXN0ptOZL%2FmyrZcI%3D&reserved=0
Htmlized:       https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftools.ietf.org%2Fhtml%2Fdraft-bishop-httpbis-priority-placeholder-00&data=02%7C01%7Cmichael.bishop%40microsoft.com%7C6ace0f1cf1594ec919ac08d4d38f0770%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636366058645292697&sdata=4IQRlgVAemvZSw0TnUA%2FZNDAHCRmCUo1%2FQRf1aLerSc%3D&reserved=0
Htmlized:       https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-bishop-httpbis-priority-placeholder-00&data=02%7C01%7Cmichael.bishop%40microsoft.com%7C6ace0f1cf1594ec919ac08d4d38f0770%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636366058645292697&sdata=VGEStTY8Gi3t3Hs230J6TmuIxi5FOLycje91etAcRJs%3D&reserved=0

Abstract:
   [RFC7540] defines HTTP/2, including a method for communicating
   priorities.  Some implementations have begun using closed streams as
   placeholders when constructing their priority tree, but this has
   unbounded state commitments and interacts poorly with HTTP/QUIC
   ([I-D.ietf-quic-http]).  This document proposes an extension to the
   HTTP/2 priority scheme for both protocols.

Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: Martin Thomson <[email-masked]>
Date: Wed, 26 Jul 2017 11:02:16 +1000

This seems to be phrased in terms of addressing the unbounded state
problem, but I don't see that as possible.  Removing closed streams
from the tree removes information in a way that can affect
prioritization, so you can't ever avoid accumulating priority state.
(An example of this is in RFC 7540.)

A more fundamental change in the design would be required to bound the
state.  Right now, I don't know what that change would look like.

As for fixing the problem in QUIC, why not add flags to the PRIORITY
frame so that you can designate a "stream ID" as a placeholder
instead?  It's a much smaller change overall.

---

From: Mike Bishop <[email-masked]>
Date: Wed, 26 Jul 2017 17:23:55 +0000

Yes, the version in HTTP/QUIC would be two flags instead of a flag and a frame, but still the same server-side setting and logic.  HTTP/2 needs a new frame because the "Prioritized Stream ID" is implicit.  I don't particularly see that as "much smaller," since the new frame is essentially PRIORITY-with-the-flag-set.

The piece that helps reduce unbounded state is giving both sides clear guidance about what should be removed and roughly when; the example in 7540 is still true, except that now both sides expect it.  If you care about the separation, use a placeholder to make the distinction.

But you can't tighten the logic without first defining placeholders, and I wanted to at least raise the possibility of defining placeholders in HTTP/2 first.

-----Original Message-----
From: Martin Thomson [mailto:[email-masked]] 
Sent: Tuesday, July 25, 2017 6:02 PM
To: Mike Bishop <[email-masked]>
Cc: [email-masked]
Subject: Re: Priority Placeholders in HTTP/2 and HTTP/QUIC

This seems to be phrased in terms of addressing the unbounded state problem, but I don't see that as possible.  Removing closed streams from the tree removes information in a way that can affect prioritization, so you can't ever avoid accumulating priority state.
(An example of this is in RFC 7540.)

A more fundamental change in the design would be required to bound the state.  Right now, I don't know what that change would look like.

As for fixing the problem in QUIC, why not add flags to the PRIORITY frame so that you can designate a "stream ID" as a placeholder instead?  It's a much smaller change overall.

On 26 July 2017 at 05:18, Mike Bishop <[email-masked]> wrote:
> Subject: New Version Notification for 
> draft-bishop-httpbis-priority-placeholder-00.txt
>
>

---

From: Patrick McManus <[email-masked]>
Date: Thu, 27 Jul 2017 14:50:31 +0000 (UTC)

On Tue, Jul 25, 2017 at 9:02 PM, Martin Thomson <[email-masked]>
wrotee.

> instead?  It's a much smaller change overall.
That's what I have envisioned as most pragmatic.