Subject: Issue re. HTTP2 STREAM and HEADER block use same end bit polarity

From: David Morris <[email-masked]>
Date: Sat, 15 Jun 2013 16:14:18 -0700 (PDT)

This issue:

https://github.com/http2/http2-spec/issues/129

describes my concern that the polarity is reversed between STREAM FINAL
and HEADER CONTINUES which are both flag bits used to manage continuation.

I think this will introduce confusion to folks analyzing wire level bits
as well as reading code.

I do acknowledge the the current flag names match the sense of the 
polarity so the names probably should change.

Dave Morris

---

From: Mike Belshe <[email-masked]>
Date: Sat, 15 Jun 2013 16:57:58 -0700

I agree on the consistency issue Dave presents.  I also like Dave's
suggestion to use 1 to mean final everywhere.

Mike

process question:  is it valuable to reply in github?  or is the list
preferred?

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sat, 15 Jun 2013 17:05:23 -0700

I don't particularly care. I just want to point out that the reason it
"natural" to do it the way it's already done, is FINAL and CONTINUES are
the exceptional cases. So to the degree that it's nicer to by default have
no flags set, the current approach is better. I don't have any paint to
waste on this bike shed though.

> process question:  is it valuable to reply in github?  or is the list

Always the list. If you see much discussion on github, yell at them to
bring it to the list. And any commits/issues/updates on github should
reference the rough consensus from the mailing list.

---

From: James M Snell <[email-masked]>
Date: Sat, 15 Jun 2013 17:13:26 -0700

+1... consistency makes the most sense.

---

From: James M Snell <[email-masked]>
Date: Sat, 15 Jun 2013 17:15:25 -0700

And fwiw, I already had a note for this in my list of todos following the
interim.

---

From: Roberto Peon <[email-masked]>
Date: Sat, 15 Jun 2013 17:49:08 -0700

no objections here with the proposal.

---

From: Amos Jeffries <[email-masked]>
Date: Sun, 16 Jun 2013 16:45:27 +1200

From an implementation point of view a default value of 0 for both 
flags is easiest of all to implement, with polarity of 1 being set for 
the exceptional cases.

The bulk of frames in a stream will *not* be the FINAL frame. Likewise 
the bulk of headers delivered will likely be small enough to *not* have 
a CONTINUES necessary.

So to me the existing polarity seems to be correct. I propose renaming 
the CONTINUES flag to "EXTEND" (extended headers present) or "LARGE" 
(large header set) or "MULTI" (multiple header blocks) or something 
else  avoiding the unfortunate wording overlap with the FINAL semantics 
description.

Amos

On 16/06/2013 12:49 p.m., Roberto Peon wrote:
> On Sat, Jun 15, 2013 at 5:15 PM, James M Snell <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     And fwiw, I already had a note for this in my list of todos
>     following the interim.
>
>     On Jun 15, 2013 5:13 PM, "James M Snell" <[email-masked]
>     <mailto:[email-masked]>> wrote:
>
>         On Jun 15, 2013 5:06 PM, "William Chan (陈智昌)"
>         <[email-masked] <mailto:[email-masked]>> wrote:
>
>             I don't particularly care. I just want to point out that
>             the reason it "natural" to do it the way it's already
>             done, is FINAL and CONTINUES are the exceptional cases. So
>             to the degree that it's nicer to by default have no flags
>             set, the current approach is better. I don't have any
>             paint to waste on this bike shed though.
>
>             On Sat, Jun 15, 2013 at 4:57 PM, Mike Belshe
>             <[email-masked] <mailto:[email-masked]>> wrote:
>
>                 I agree on the consistency issue Dave presents.  I
>                 also like Dave's suggestion to use 1 to mean final
>                 everywhere.
>
>                 Mike
>
>                 process question:  is it valuable to reply in github?
>                  or is the list preferred?
>
>
>             Always the list. If you see much discussion on github,
>             yell at them to bring it to the list. And any
>             commits/issues/updates on github should reference the
>             rough consensus from the mailing list.
>
>
>
>
>                 On Sat, Jun 15, 2013 at 4:14 PM, David Morris
>                 <[email-masked] <mailto:[email-masked]>> wrote:
>
>
>
>                     This issue:
>

---

From: Jeff Pinner <[email-masked]>
Date: Tue, 18 Jun 2013 11:39:25 -0700

A suggestion for the HEADERS frame flags that incorporates "FINAL",
"MSG_DONE", and changes the polarity of "CONTINUES:"

0x1: END_STREAM - indicates that this frame is the last the endpoint will
send on the identified stream.
0x2: END_MESSAGE - indicates that this frame comprises a message boundary.
0x4: END_HEADERS - indicates that this frame marks the end of the encoded
header block.

For compression, the text can then read something like "a header block is
compressed and encoded according to <link header compression spec> and
serialized in a sequence of HEADERS frames... frames that comprise an
encoded header block must be written sequentially and cannot be interleaved
with other frames... the final frame in the sequence must be identified by
setting the END_HEADERS flag" (re-written by our editors to make it easier
to understand)

On Sat, Jun 15, 2013 at 9:45 PM, Amos Jeffries <[email-masked]> wrote:

> From an implementation point of view a default value of 0 for both flags
> is easiest of all to implement, with polarity of 1 being set for the
> exceptional cases.
>
> The bulk of frames in a stream will *not* be the FINAL frame. Likewise the
> bulk of headers delivered will likely be small enough to *not* have a
> CONTINUES necessary.
>
> So to me the existing polarity seems to be correct. I propose renaming the
> CONTINUES flag to "EXTEND" (extended headers present) or "LARGE" (large
> header set) or "MULTI" (multiple header blocks) or something else  avoiding
> the unfortunate wording overlap with the FINAL semantics description.
>
> Amos
>
>
>> On Sat, Jun 15, 2013 at 5:15 PM, James M Snell <[email-masked]<mailto:
>> [email-masked]>> wrote:
>>
>>         <[email-masked] <mailto:[email-masked]>**> wrote:
>>
>>                     https://github.com/http2/**http2-spec/issues/129<https://github.com/http2/http2-spec/issues/129>
>>

---

From: Martin Thomson <[email-masked]>
Date: Tue, 18 Jun 2013 11:53:57 -0700

> 0x4: END_HEADERS - indicates that this frame marks the end of the encoded

I like this generally, however I'm struggling with the distinction
between END_MESSAGE and END_HEADERS.

I can appreciate the logical purity of the argument for a distinction,
but I can't see a practical benefit to it.  END_HEADERS implies
concatenation, but END_MESSAGE just ...ends?  I'm not sure then what
you do with a sequence of frames that form a message, if you aren't
concatenating them. Websockets APIs concatenate.

If this is about the hard lock-out that we have when HEADERS are
continued, that doesn't need a special signal, just some text.

---

From: Jeff Pinner <[email-masked]>
Date: Tue, 18 Jun 2013 13:32:59 -0700

I am going to make a completely contrived example to try to illustrate my
thoughts:

Let's say I have a message based protocol that tries to send the following
messages over stream 1:

Content-Type: text
Message 1

Content-Type: json
{message_id: 2}

This could be done by encoding the following frames on stream 1:

Stream-Id 1 HEADERS(END_HEADERS): Content-Type: text
Stream-Id 1 DATA(END_MESSAGE): Message1
Stream-Id 1 HEADERS(END_HEADERS): Content-Type: json
Stream-Id 1 DATA(END_STREAM,END_MESSAGE): {message_id: 2}

Now for the sake of argument lets say that the frame length was so small
that "Content-Type" couldn't fit completely in the frame. Then the client
would have to send the following:

Stream-Id 1 HEADERS: Content-
Stream-Id 1 HEADERS(END_HEADERS): Type: text
Stream-Id 1 DATA(END_MESSAGE): Message1
Stream-Id 1 HEADERS: Content-
Stream-Id 1 HEADERS(END_HEADERS): Type: json
Stream-Id 1 DATA(END_STREAM,END_MESSAGE): {message_id: 2}

In any case -- the point here is to illustrate that concatenation of header
blocks for decoding is completely orthogonal to the possible existence of
messages and their boundaries.

- Jeff

---

From: Roberto Peon <[email-masked]>
Date: Tue, 18 Jun 2013 13:42:18 -0700

END_HEADERS indicates the end of a header block.
END_MESSAGE indicates the end of a message, which could be metadata only,
or it could be metadata+data.
END_MESSAGE is necessary because END_HEADERS is insufficient to indicate
the end of a message. It would only be sufficient if we knew that messages
were only headers.

END_MESSAGE should not be allowed on any headers frame where END_HEADERS is
not true.

I see Jeff replied before I got this email sent.
What he said too :)
-=R

---

From: Jeff Pinner <[email-masked]>
Date: Tue, 18 Jun 2013 14:09:32 -0700

https://github.com/http2/http2-spec/pull/136 to reverse the polarity of
CONTINUES and rename FINAL -> END_STREAM and CONTINUES -> END_HEADERS

On Tue, Jun 18, 2013 at 1:42 PM, Roberto Peon <[email-masked]> wrote:

> END_MESSAGE should not be allowed on any headers frame where END_HEADERS
> is not true.
>
> On Tue, Jun 18, 2013 at 11:53 AM, Martin Thomson <[email-masked]
> > wrote:
>

---

From: Martin Thomson <[email-masked]>
Date: Tue, 18 Jun 2013 15:13:05 -0700

On 18 June 2013 13:42, Roberto Peon <[email-masked]> wrote:
> END_MESSAGE indicates the end of a message, which could be metadata only, or
> it could be metadata+data.

The term "message" has not been defined.  I believe that is the
problem here.  (We actually removed that word from the draft for that
precise reason).

I never imagined a definition of message that resembled that, since
HTTP message =~ all the on-stream frames.  I know that you have
something in mind for websockets, but I'm not sure that I like the
reuse of HEADERS for the purpose of providing WS message/frame
headers, but that's a whole new discussion.

I understand what you are doing now.  I don't think that we should add
the flag until we can properly explain what it means.

---

From: James M Snell <[email-masked]>
Date: Tue, 18 Jun 2013 16:48:48 -0700

+1 ... we especially do not need END_MESSAGE in the spec right now..
at least for this initial implementation draft. We ought to pick this
one up a bit later once we get into the discussion about extensions.

---

From: James M Snell <[email-masked]>
Date: Tue, 18 Jun 2013 16:49:32 -0700

+1 to END_STREAM and END_HEADERS, +1 to reserving 0x2 for END_MESSAGE
later on but -1 to including it in this implementation draft.

---

From: Jeff Pinner <[email-masked]>
Date: Tue, 18 Jun 2013 18:02:37 -0700

reserved: https://github.com/http2/http2-spec/pull/137

On Tue, Jun 18, 2013 at 4:49 PM, James M Snell <[email-masked]> wrote:

> the
> the

---

From: Martin Thomson <[email-masked]>
Date: Tue, 18 Jun 2013 21:52:47 -0700

And merged.

On 18 June 2013 18:02, Jeff Pinner <[email-masked]> wrote:
>> >> flags