Subject: Re: h2 frame layout

From: Stefan Eissing <[email-masked]>
Date: Mon, 1 Sep 2014 10:14:41 +0200

+1 on issues addressed by Roy.

Having good alignment on protocol data seems to be good for machines and humans both.

//Stefan (from the back benches of another building across town)

Am 01.09.2014 um 04:44 schrieb Greg Wilkins <[email-masked]>:

> 
> On 1 September 2014 11:57, Matthew Kerwin <[email-masked]> wrote:
> If the framing machine were frame-type-agnostic,
> 
> +1.  That is my main point.
>  
> all frame types could carry the END_STREAM flag
> 
> that is one way of achieving it.    Roy's proposal B is another.
> 
> I'm a bit agnostic as to which is best.   The flag is probably simplest and more conventional, but would mean that we need to review silly combinations like RST_STREAM with no END_STREAM.    But maybe in a frame-type agnostic world, we don't need RST_STREAM and GO_AWAY frames?   Isn't RST_STREAM just a GO_AWAY with a non zero stream ID?
> 
> Maybe we can replace the END_STREAM flag and both GO_AWAY and RST_STREAM frames with a single END_STREAM frame:
> 	•   END_STREAM(SID==0, CODE==0),  normal graceful go away
> 	•   END_STREAM(SID==0, CODE!=0),  error connection shutdown
> 	•   END_STREAM(SID!=0, CODE==0),  normal stream shutdown
> 	•   END_STREAM(SID!=0, CODE!=0),  reset stream style error
> cheers
> 
> 
> 
> 
> 
>   
> 
> 
> 
> 
> -- 
> Greg Wilkins <[email-masked]> 
> http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
> http://www.webtide.com  advice and support for jetty and cometd.

<green/>bytes GmbH
Hafenweg 16, 48155 Münster, Germany
Phone: +49 251 2807760. Amtsgericht Münster: HRB5782

---

From: Martin Thomson <[email-masked]>
Date: Tue, 2 Sep 2014 10:36:54 -0700

On 29 August 2014 15:49, Roy T. Fielding <[email-masked]> wrote:
> I should be shocked that folks are "not concerned" about the use
> of a 9-octet header and the resulting effect on data alignment

We're not doing anything to pad frames and maintain alignment, so any
attempt to align would require that you pad very carefully.  I don't
see this as a valid argument.

>   2) The length is the most random of these fields and used last.

That depends on the structure of your software.  In one world view,
the delineation of frames comes before determining their type.

The other arguments (efficiency, etc...) are valid.  I'll let others
speak to those.

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 2 Sep 2014 11:34:56 -0700

> see this as a valid argument.

I don't need to pad a 64bit fixed frame preamble, so it isn't difficult
at all after my changes are made.  The difficulty (or waste) of 72bits is
really in how I read data off the network (into buffers) and how I write
data to the network (out of buffers) and the hardware tries to compensate
for non-aligned data.  The other inefficiency occurs when using non-network
representations of the protocol to do application routing.  I don't need
a guarantee of alignment.  What I want is for it to be less statistically
likely on every request/response.

However, I do agree that this is minor compared to the fact that those
extra eight bits aren't even necessary, so sending them in every frame
is pretty stupid for a protocol that is supposed to be efficient.

> the delineation of frames comes before determining their type.

In a world view where the number indicates the total length of the
frame, yes.  But that isn't what is given here.  It is the length of
the frame payload, which comes after the fixed frame preamble.
A smarter design would have allowed some of the standard frames to use
those bits for data instead of payload length (e.g., PING, PRIORITY,
RST_STREAM, and WINDOW_UPDATE), but there is also value in frame
consistency.

The value of starting the preamble with a very consistent number,
like the stream-id, is that it makes visual debugging of framing
errors a whole lot easier.

....Roy

---

From: Patrick McManus <[email-masked]>
Date: Tue, 2 Sep 2014 17:25:31 -0400

Hi Roy,

On Fri, Aug 29, 2014 at 6:49 PM, Roy T. Fielding <[email-masked]> wrote:

> This comment is in reference to section 4.1 of
>
>   http://tools.ietf.org/id/draft-ietf-httpbis-http2-14.txt
>
> > 4.1.  Frame Format
> >
> >    All frames begin with a fixed 9-octet header followed by a variable-
> >    length payload.
> [..]
>
> I think this has become a bit moth-eaten over time as the WG has
> adjusted field lengths and perhaps added or removed a few.
>

That time was quite recent:  from -13 to -14 as the consensus was to
increase the payload length from 16 to 24 bits maintaining a requirement
for a fixed 10 0 bits unless otherwise negotiated via settings.

I was never a fan of that - if we were to decide that a concise 64 bits is
necessary then I think we should revert that length field increase rather
than giving up the type and flag space extensions may want to use. The
extended length users could operate via extension.

But before anyone gets too excited, overall I personally don't think the
need to either align or pack the header into 64bits is very important given
the rather unaligned nature of the payloads and so I don't favor reopening
the issue. The 8 vs 9 byte header was part of the issue discussion already
- as can be seen here
http://lists.w3.org/Archives/Public/ietf-http-wg/2014JulSep/0706.html when
a proposal reallocating 8 bits from stream ID instead of type and flags so
I don't feel like this is unconsidered information when making that
decision.

-Patrick

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 02 Sep 2014 21:36:26 +0000

--------
In message <[email-masked]>
, Patrick McManus writes:

>the issue.

For what it is worth, I don't think trying to align the frames makes
any sense, because there is no way we can guarantee that they will
be aligned into underlying TCP and IP packets in the first place.

Things like keeping the :headers first, maybe even in strictly
defined order like HTTP/1.1 does it will be much more important
for high performance implementations.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Martin Thomson <[email-masked]>
Date: Tue, 2 Sep 2014 14:54:25 -0700

> for high performance implementations.

Yeah, we did that because we could when we removed the reference set
from HPACK.  A fixed order (or even format, so you avoid the need for
labels) might have been better in that regard, though it would have
complicated things a little more.

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 3 Sep 2014 11:55:30 +1000

In the spirit of getting less meta....

I've no objections to proposal A, but I don't think it gains much either.

I like the direction that Proposal B goes, ie getting rid of general flags
within the frame header, but there were several optional parts in that
proposal that I think need to be fleshed out.    In general, I'm not that
keen on END_STREAM and END_HEADERS being new frame types as I think the
extra  2*64 bytes for a minimal HTTP message is significant.

So here is a sub-proposal of proposal B where I expand some of the options
that Roy outlined:

*Proposal B0*

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |E|                 Stream Identifier (31)                      |
 +---------------+-----------------------------------------------+
 | Frame Type(8) |   Payload Length (24)                         |
 +=+=============================================================+
 |                   Frame Payload (0...)                      ...
 +---------------------------------------------------------------+

E == END_STREAM

So the currently defined flags would be handled as:

   -   END_STREAM:  using E bit (was the reserved bit)
   -   END_HEADERS: Use END_META_DATA frame (see below)
   -   PADDED: send in padding frame (see other thread)
   -   PRIORITY: send in priority frame (see other thread)
   -   Settings ACK: SETTING_ACK frame type
   -   Ping ACK: PONG frame type.

With this proposal, instead of carrying headers and trailers as

  HEADER CONTINUATION*

We would effectively carry them as

  CONTINUATION* HEADER

ie the different frame type is used to indicate the last frame and not the
first (which
depends on the proposal to move optional priority out of the HEADERS frame).

However, since headers can be sent as trailers, I would optionally propose
that we
rename to END_META_DATA and META_DATA, so a complete HTTP message is

    META_DATA* END_META_DATA DATA* (META_DATA* END_META_DATA)?

For PUSH_PROMISE frames, we would need to remove the header block from the
pp frame itself and instead send in a following frame:

    PUSH_PROMISE META_DATA* END_META_DATA

*Proposal B1*
I think there is merit in generalising the distinction between META_DATA
and END_META_DATA, as if we ever wanted to revive end segment semantics,
it would not really be sufficient to come up with END_DATA and DATA,
because
the semantics of a segment would not be understood by intermediaries

So to generalise further B1 uses 1 bit of the frame type as a generic
END_SEGMENT:

  0                   1                   2                   3
  0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 |E|                 Stream Identifier (31)                      |
 +---------------+-----------------------------------------------+
 |S|Frame Type(7)|   Payload Length (24)                         |
 +=+=============================================================+
 |                   Frame Payload (0...)                      ...
 +---------------------------------------------------------------+

E == END_STREAM
S == END_SEGMENT

So the currently defined flags would be handled as:

   -   END_STREAM:  using E bit (was the reserved bit)
   -   END_HEADERS: Use S (see below)
   -   PADDED: send in padding frame (see other thread)
   -   PRIORITY: send in priority frame (see other thread)
   -   Settings ACK: SETTING_ACK frame type
   -   Ping ACK: PONG frame type.

A HTTP message sequence would then become:

    META_DATA+ DATA* META_DATA*

and a push promise:

    PUSH_PROMISE META_DATA+

Note also that strictly speaking an end point need not look at the
END_SEGMENT
flag to extract HTTP semantics, as first DATA frame or END_STREAM indicates
the end of the metadata.  However it is better to require the END_SEGMENT
bit to
be set so that Intermediaries can be written to forward any frame with the
END_SEGMENT flag set in a timely fashion, without having to know about
HTTP semantics.

regards

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 3 Sep 2014 09:22:41 +0300

I’ve raised this as <https://github.com/http2/http2-spec/issues/603>.

This one is a bit difficult to pick apart, but it seems like one of the core arguments is that type and flags waste bits. Again, how do people feel about cutting these down?

Regarding putting the stream identifier first — what concrete benefits will that bring?

This issue (again) seems to be a matter of preference / judgement; I don’t see any interoperability problems highlighted here, so making a change here is going to require good consensus.

Regards,

On 30 Aug 2014, at 1:49 am, Roy T. Fielding <[email-masked]> wrote:

>>    |                 Length (24)                   |
>>    +---------------+---------------+---------------+
>>    |   Type (8)    |   Flags (8)   |
>>    +-+-+-----------+---------------+-------------------------------+
>>    |R|                 Stream Identifier (31)                      |
>>                               Frame Layout
> I also find the use of a bit array with magic gaps to be editorially
> bizarre, but this message is about the actual bits.
> 
> of a 9-octet header and the resulting effect on data alignment, but it
> is true that it won't matter much to character-oriented Web traffic.
> Nevertheless, I'd expect HTTP/2 to at least make an effort here
> given the number of APIs that don't process character traffic.
> 
> 
> Looking at the current spec, I think we can do better.
> 
>  1) There are only 10 defined frame types and 2 useful flags.
>  3) The stream identifier is the least random and used first.
>  4) Why are we reserving the high bit of an unsigned field?
> 
> My suggestion is that the frame layout be changed to either
> 
>  0                   1                   2                   3
> |                   Stream Identifier (32)                      |
> |Type(4)|Flag(4)|   Payload Length (24)                         |
> or
> |                   Stream Identifier (32)                      |
> Rationale for both:
>  11% reduction in per-frame overhead
>  A fixed 64bit layout is faster to parse than 72bits.
>  A sender doesn't have to worry about 64bit alignment.
>  A receiver is likely to partition based on stream id first.
> 
> Rationale for A:
> 
>  If we need more than 6 additional frame types, we might as well
>    skip to a new protocol version.
>  If we need more than 4 flags per frame, we should use another type.
>  The useful flags are END_STREAM and END_HEADERS and they could be
>    defined in general (independent of frame type).
> 
> Rationale for B:
> 
>  The current usage of frame flags in spec-14 is frightening.
>  In all cases, they can and should be separate frame types.
> 
>  END_STREAM:  Orthogonal. Could be moved to the reserved bit or made
>    another frame type (an extra 8 bytes at end of stream is better
>    than the extra 8 flag bits per frame).
> 
>  END_HEADERS: Orthogonal. Could be just another frame type or moved
>    to the beginning of the frame payload in Headers, Continuation,
>    and Push_Promise.
> 
>  PADDED: Padding is just a form of ignored data and should
>  be a PADDING type.  We can then remove those optional fields from the
>  other frame types.  Optional fields at the beginning of otherwise fixed
>  frames is parser hell.
> 
>  Header PRIORITY: What do headers have to do with frame priority?
>  Why here and not on DATA?  We should instead have different
>  frame types for MY_PRIORITY (this is how I am sending/reading) and
>  YOUR_PRIORITY (this is how I want you to prioritize sending/reading).
>  We can then remove those optional fields from the other frame types.
> 
>  Settings ACK: send an ACK frame type instead (and remove the additional
>  complexity of processing requirements in the Settings type).
> 
>  Ping ACK: send a PONG type.
> 
> 
> Alternative A would just make the protocol more efficient and faster to
> parse at the cost of some extensibility. B would significantly simplify the
> protocol and the spec.  Some combination of the two is also possible
> (e.g., 6 bits for type and 2 flag bits for END_STREAM and END_HEADERS).
> 
> Alternatively, if END_STREAM is moved to the reserved bit E, then it
> would look like:
> 
>  0                   1                   2                   3
> Yes, I know this suggestion comes late in the process.  I was busy.
> 
> Cheers,
> 
> Roy T. Fielding                     <http://roy.gbiv.com/>
> Senior Principal Scientist, Adobe   <http://www.adobe.com/>
> 
> 

--
Mark Nottingham   http://www.mnot.net/

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 03 Sep 2014 06:49:02 +0000

--------
In message <[email-masked]>, Mark Nottingham wri
tes:

>Regarding putting the stream identifier first - what concrete benefits 
>will that bring?

There are no performance benefits.

Given a transport with packets larger than the frame header
(ie: all but async RS-232), the only case where the order can
matter is if the frame header is split across packets.

For plain HTTP/TCP/IP/Ethernet that happens with a probability
of 9/1480 = 0.6%.

In reality the probability is a lot lower because frames will end
up at front of new packets after inter-packet delays.

The best "gedanken-experiment" I can come up with is a hyperintelligent
ethernet controller which uses the stream-id to route the packet to
a particular CPU-core.

However, such an ethernet controller would have to also react to the
length field or it wouldn't know where the next frame starts, so to
order of the fields wont matter.

(For a relevant piece of prior art in this space, see the contortions
the Intel '599 ethernet controller goes through to support timestamping
of PTP packets.  It's around page 700-ish of the 1000+ page datasheet.)

But the same cannot be said of the question about how we tell that
a stream has ended.

Such an intelligent ethernet controller would benefit if there were
a marker bit in the frame header that says "this is the last
frame on this stream" so that it can free any resources associated
with the processing of that stream.