Subject: Design Issue: Life-cycle of a Stream

From: James M Snell <[email-masked]>
Date: Wed, 1 May 2013 10:03:45 -0700

The current spec is rather vague on the complete life-cycle of a
stream within a session. Aside from discussing how the Mommy Endpoint
and the Daddy Endpoint get together, we ought to take just a bit of
time to make sure we're all on the same page...

In a separate thread, several of us discussed stream creation...
specifically, how is a stream initiated. Let's see if we can formalize
the process and the language. I've written the following out as
statements, if you disagree with the statement or see it a different
way, lemme know! Some of this is obvious and well established, I'm
documenting it here just to make sure we have the complete picture
captured. I've marked the items that appear to me to be the most
controversial with an asterisk.

1. Streams are initialized or opened by sending a HEADERS or
HEADERS+PRIORITY frame that uses a previously unused stream
identifier.

2. Only HEADERS and HEADERS+PRIORITY frames are used to open a stream.

3. PUSH_PROMISE frames reserve a stream identifier but do not open the stream.

4. Promised streams are only opened by sending a subsequent HEADERS or
HEADERS+PRIORITY frame that uses the reserved stream identifier.

5. All streams are either fully open, half-closed by the client,
half-closed by the server, or fully closed.

6. All promised streams are associated with an originating "parent" stream.

7. A stream is half-closed by sending a frame with the FINAL bit set.

8. A stream becomes fully-closed when either: a) both endpoints send
frames with the FINAL bit set or b) either endpoint sends an RST_FRAME
with that streams identifier.

9*. Promised streams are automatically half-closed for the recipient.
That is, when the server sends a PUSH_PROMISE to a client, that
streams becomes automatically half-closed for the client. The only
frame the client can send back to the server for that stream are
RST_STREAM frames.

  -- or --

9*. Promised streams are handled just like regular streams. The server
opens the stream and half-closes. The stream does not fully close
until either the client half-closes it or a RST_STREAM frame is sent
for that stream.

10*. Sending an RST_FRAME for a parent stream fully closes that stream
as well as all associated promised streams.

  -- or --

10*. Sending an RST_FRAME for a parent stream only closes that stream
and has no affect on associated promised streams.

11*. Fully closing a parent stream using symmetric half-closes will
fully close all associated promised streams.

  -- or --

11*. Fully closing a parent stream using symmetric half-closes has no
affect on associated promised streams.

12. Half or fully closing a promised stream has no impact on the
parent stream or any associated sibling promised streams.

Ok, so that covers the lifecycle.. let's talk about processing.. this
is largely a reiteration of my previous note on this but I want to
make sure I've hit all the major points in how this is categorized.

Tier 1: Includes the following...

A. Reception and basic parsing of frames. "Basic parsing" here means
looking at the header bytes to determine the frame type, making note
of the flags, frame size and stream identifier, and performing any
necessary header block decompression and state management.

B. Handling of all session frames (stream id #0). This means that
upgrade negotiation, SETTINGS, PING, flow control and session level
error handling are all Tier 1 processes.

C. Basic lifecycle handling of stream frames. This includes
determination of whether a stream is open or closed, whether or not
additional frames are allowed for a stream, validation of proper
stream id use, reservation of stream id's with push_promise, and
handling of the FINAL flag.

Tier 2: Includes the following...

A. Frame-type specific processing. Validation and handling of frame
payloads and frame-specific flags, not including header block
processing (which is tier 1)

B. Additional stream-level error handling not related to stream-lifecycle

Tier 3: Includes all application-level handling (HTTP Semantics)

So, when we say things like "an endpoint must be prepared to continue
receiving frames on a closed stream", we mean to say that "an endpoint
must be prepared to do tier 1 processing on all frames, even those
sent for closed streams."

Please weigh in on whatever items y'all feel are controversial. I'll
work up proposed spec edits based on whatever feedback is received.

- James

---

From: William Chan (陈智昌) <[email-masked]>
Date: Wed, 1 May 2013 14:36:28 -0300

Thanks for sending this email. This is great. I'll chime in with my
personal interpretations.

> 6. All promised streams are associated with an originating "parent" stream.

For HTTP/2 I think this totally makes sense and this is the right
interpretation. Again, if we assume the framing layer is separate from
HTTP/2 usage, then this might not be good to require.

> frame the client can send back to the server for that stream are

I think we agreed on this interpretation in the last email thread you
raised on PUSH_PROMISE.

> and has no affect on associated promised streams.

I forget if we settled this issue or not. But I'm inclined to prefer your
second interpretation here. I'd rather explicitly reset streams.

> affect on associated promised streams.

The latter. I believe the spec only says we have to promise the streams
before the parent stream closes. We don't have to even open the streams
yet. This is probably ideal since the parent stream is likely the main
document, and the main document is likely to contain the most important
data. One might say just send all the data and delay the sending of a
0-byte DATA frame with the FINAL flag set, but I see no reason to require
that. Speaking as an implementer, it's easier for the browser to receive
the FINAL flag ASAP (there are some implementation complications with
multiple readers and single writer, where multiple "readers" are requesting
a resource at the same URL, and are sharing the same response...we stream
the response to a single reader and only when we know it to be complete do
we stream to all other readers. don't ask me why, it's complicated and
relates to our cache implementation).

> make sure I've hit all the major points in how this is categorized.

I don't really want to start calling this Tier Number. I call things by the
relevant layer. Everyone else at the in person meetings has done the same.
I know jpinner@ has organized his code accordingly to his view of the
layers. I'd like us to standardize terminology on that.

---

From: Martin Thomson <[email-masked]>
Date: Wed, 1 May 2013 10:49:24 -0700

> 2. Only HEADERS and HEADERS+PRIORITY frames are used to open a stream.

I don't see why 1 & 2 are necessary.  It's just arbitrary.  Sending
any frame should suffice.

> 3. PUSH_PROMISE frames reserve a stream identifier but do not open the stream.

Good.

> HEADERS+PRIORITY frame that uses the reserved stream identifier.

Again, promised streams are opened by sending frames.

> half-closed by the server, or fully closed.

I think of this differently.  Streams aren't bi-directional
constructs, they are a loose binding of two unidirectional constructs.
 Each unidirectional component has its own state.

FINAL ends a stream in the direction that it appears on.
RST_STREAM ends a stream in the direction that it appears on and
requests the closure of the opposite direction.

> 6. All promised streams are associated with an originating "parent" stream.

That binding is a problem.  The reason for the stream association in
push promise is to obtain request headers.  That relationship is a
function of the HTTP usage of streams, not an inherent function of the
streaming layer.

> 7. A stream is half-closed by sending a frame with the FINAL bit set.

See above, FINAL ends a stream for the direction in which it appears.

> with that streams identifier.

RST_STREAM.

> until either the client half-closes it or a RST_STREAM frame is sent

Neither works particularly well for me.  If you consider independent
lifecycles for each direction of a stream, this sort of distinction is
unnecessary.

> and has no affect on associated promised streams.

The latter, please.

> affect on associated promised streams.

The latter, please.

> parent stream or any associated sibling promised streams.

Of course.

> error handling are all Tier 1 processes.

You probably need more than one dimension in your taxonomy to get this
right.  I don't see this as a function of the thing that processes
frames.

> handling of the FINAL flag.

I'm assuming that there is a stream controller that takes all
stream-related frames and does things to the stream state.  That
controller looks at FINAL and consumes things like RST_STREAM.  But
that operates at what I consider to be Tier 2.  It also has its own
layering because flow control exists here.

---

From: Mike Bishop <[email-masked]>
Date: Wed, 1 May 2013 18:04:45 +0000

#5 vs. #1 raises a potential issue:  A stream is opened by sending a HEADERS or HEADERS+PRIORITY, but none of the states in #5 match a stream which hasn’t been opened yet.  To say that it’s closed to begin with suggests that a stream which has been half-closed by both sides could be re-opened later, which I don’t think any of us have suggested is a healthy option.

I’m inclined to say that there is an uninitialized state in which a stream has not been opened yet.  Receipt of HEADERS/HEADERS+PRIORITY (or any frame, if we want to minimize the HTTP semantics in the framing layer) moves the stream from uninitialized to open.  Receipt of a PUSH_PROMISE referencing the stream ID moves the client from uninitialized to half-closed, since the client’s side of the exchange is inferred from what’s already sent.

I agree with Will on the 9-11 options.  Each stream lives or dies on its own, and we should keep their lifecycles as distinct as possible.  They need to be linked, because you have to not have discarded the parent stream’s state in order to infer the state for the push stream; once that state has been copied, I think it should be a fully distinct stream.

(Though #9 might need to be revisited if we allow pushed streams to have priority in a future discussion.)

From: [email-masked] [mailto:[email-masked]] On Behalf Of William Chan (???)
Sent: Wednesday, May 1, 2013 10:36 AM
To: James M Snell
Cc: [email-masked]
Subject: Re: Design Issue: Life-cycle of a Stream

Thanks for sending this email. This is great. I'll chime in with my personal interpretations.

On Wed, May 1, 2013 at 2:03 PM, James M Snell <[email-masked]<mailto:[email-masked]>> wrote:

For HTTP/2 I think this totally makes sense and this is the right interpretation. Again, if we assume the framing layer is separate from HTTP/2 usage, then this might not be good to require.

I think we agreed on this interpretation in the last email thread you raised on PUSH_PROMISE.

  -- or --

  -- or --

I forget if we settled this issue or not. But I'm inclined to prefer your second interpretation here. I'd rather explicitly reset streams.

  -- or --

The latter. I believe the spec only says we have to promise the streams before the parent stream closes. We don't have to even open the streams yet. This is probably ideal since the parent stream is likely the main document, and the main document is likely to contain the most important data. One might say just send all the data and delay the sending of a 0-byte DATA frame with the FINAL flag set, but I see no reason to require that. Speaking as an implementer, it's easier for the browser to receive the FINAL flag ASAP (there are some implementation complications with multiple readers and single writer, where multiple "readers" are requesting a resource at the same URL, and are sharing the same response...we stream the response to a single reader and only when we know it to be complete do we stream to all other readers. don't ask me why, it's complicated and relates to our cache implementation).

I don't really want to start calling this Tier Number. I call things by the relevant layer. Everyone else at the in person meetings has done the same. I know jpinner@ has organized his code accordingly to his view of the layers. I'd like us to standardize terminology on that.

- James

---

From: James M Snell <[email-masked]>
Date: Wed, 1 May 2013 16:21:27 -0700

Ok, based on the feedback given so far, we seem to be in agreement
that promised streams ought to have an independent lifecycle from the
parent stream. Closing the parent, even with an RST_STREAM, ought not
affect any of the associated promised streams, and that the only
association that truly exists between the original and the promised is
that the promised stream inherits headers from the original. Does that
seem accurate? (The header inheritance does raise a number of other
questions but I'll get to those later on). This resolves items 10 and
11.

Items #2 and #4 are still controversial and we need more input to figure it out.

Item #9 is also controversial. Currently the group appears to be
leaning towards requiring that pushed streams are always
unidirectional and automatically half-closed for the receiver.
However, there is the opposing point of view (which I share) that we
ought to allow pushed streams to be optionally bidirectional, even if
http/2 semantics do not make direct use of the bidirectionality. Some
subprotocol later on might.

All of the other items in the lifecycle list appear to be non-controversial.

As for the Tier discussion, we can use the "layer" terminology
instead. How about, "Tier 0" == "Session Layer", "Tier 1" == "Framing
Layer", "Tier 2" == "HTTP/2 Layer". Also, it is suggested that Item C
from my tier 1 in the original note ought to be part of tier 1. That
works for me.

Does that appear to be an accurate assessment?

- James