Subject: HTTP 2 Header Tables and Priority

From: Steve Davis <[email-masked]>
Date: Sun, 4 Aug 2013 11:42:20 -0500

If I understand correctly from reading -http2-04 and -compression-01, then the following is true:

1) There is only one set of request/response header tables per connection that serves all streams.

2) It is essential that the header table indexes and values are agreed by both peers in order to interpret and process future header frames correctly. i.e. The respective tables must be in the same state when interpreting a new header frame as it was on the peer when constructing the header frame.

3) However, streams can be prioritized for processing order according to the implementation.

Given the above, there does not seem to be an ordering of header frame processing that will guarantee that the header tables of the connected peers will remain synchronized. Is this a mis-read, or a genuine issue?

A few smaller observations:

1) regarding -compression-01 is that substitution and increment indices are represented by (index + 1) to avoid collision with the literal prefix. Would it not be simpler, less bug-prone, and generally more consistent to simply define the start of the header index tables at index 1?

2) regarding -http2-04, there's a slightly inconsistent wording wrt the frame flag bits, e.g in section 6.2: 
 END_STREAM (0x1):  Bit 1 being set indicates that this frame is the
      last that the endpoint will send for the identified stream
Since bit numbering in the rest of the document number is decimal starting from 0, "bit 1" suggests 0x40. I am assuming 0x1 is the correct interpretation here which would appear in the decimal ordering as used in the rest of the document as "7"... minor nit-pick, I know.

3) in -compression-01 the whole idea of "integer encoding" seems overly-complex in order to save a few bits. Given the overall saving provided already by indexing, I question whether this additional saving really worth the effort? I guess that's a call between network saving and cpu savings... in this case I'd say simplicity would be more worthwhile than the savings for the network -- but that's just an opinion. 

regs,
/s

---

From: Roberto Peon <[email-masked]>
Date: Sun, 4 Aug 2013 18:53:01 +0200

Compression should always be the last step before serializing-to-the-wire
for HEADERS, else you will have problems when doing any muxing (as you've
indicated). Since HTTP/2 is currently predicated on using TCP, so long as
this rule is followed, all is well.

-=R

On Sun, Aug 4, 2013 at 6:42 PM, Steve Davis
<[email-masked]>wrote:

> 2) It is essential that the header table indexes and values are agreed by
> both peers in order to interpret and process future header frames
> correctly. i.e. The respective tables must be in the same state when
> interpreting a new header frame as it was on the peer when constructing the
> header frame.
>
> Given the above, there does not seem to be an ordering of header frame
> processing that will guarantee that the header tables of the connected
> peers will remain synchronized. Is this a mis-read, or a genuine issue?
>
> 1) regarding -compression-01 is that substitution and increment indices
> are represented by (index + 1) to avoid collision with the literal prefix.
> Would it not be simpler, less bug-prone, and generally more consistent to
> simply define the start of the header index tables at index 1?
>
> Since bit numbering in the rest of the document number is decimal starting
> from 0, "bit 1" suggests 0x40. I am assuming 0x1 is the correct
> interpretation here which would appear in the decimal ordering as used in
> the rest of the document as "7"... minor nit-pick, I know.
>
> 3) in -compression-01 the whole idea of "integer encoding" seems
> overly-complex in order to save a few bits. Given the overall saving
> provided already by indexing, I question whether this additional saving
> really worth the effort? I guess that's a call between network saving and
> cpu savings... in this case I'd say simplicity would be more worthwhile
> than the savings for the network -- but that's just an opinion.
>
> regs,
> /s
>

---

From: Fred Akalin <[email-masked]>
Date: Sun, 4 Aug 2013 18:59:50 +0200

> peers will remain synchronized. Is this a mis-read, or a genuine issue?

You have to do your compression "just in time", i.e. right before you write
the frame to the network.

Note that you have to do something similar for assigning stream IDs to
guarantee monotonicity.

---

From: Steve Davis <[email-masked]>
Date: Sun, 4 Aug 2013 12:38:30 -0500

OK now I see how to meet the requirement for ordering...

Since I'm using a language with good concurrent support (erlang), I was looking to spin out processing of each incoming tcp binary over to different processes according to a quick check of the streamId in the frame header. It rather looks like I missed the fact that the frame decode/encode has to be done sequentially to reconstruct the request before passing it to/from the relevant stream process.

Thanks very much Fred and Roberto for the pointers.

regs,
/s

On Aug 4, 2013, at 11:59 AM, Fred Akalin <[email-masked]> wrote:

> You have to do your compression "just in time", i.e. right before you write the frame to the network. 
> 
> Note that you have to do something similar for assigning stream IDs to guarantee monotonicity.
>

---

From: Martin Thomson <[email-masked]>
Date: Mon, 5 Aug 2013 07:20:36 +0200

On 4 August 2013 19:38, Steve Davis <[email-masked]> wrote:
> It rather looks like I missed the fact that the frame decode/encode has to
> be done sequentially to reconstruct the request before passing it to/from
> the relevant stream process.

That's the safe way.  Ultimately, there will be a single thread that
sequences writes into the TCP stream, so there's limited gain from
concurrent processing for compression.  That said, you might be able
to come up with a clever scheme that allows multiple threads to do the
bulk of the compression logic concurrently.  I'm not sure that it
would be worthwhile at this early stage, however, because it's
unlikely to give any significant gains in contrast, though it
certainly would be a complex piece of code that probably wouldn't
survive changes.  As I understand it, the current spec isn't highly
likely to be the final one.