Subject: Static Table Entries

From: "Simpson, Robby (GE Energy Management)" <[email-masked]>
Date: Thu, 7 Aug 2014 20:11:39 +0000

I've searched around and cannot find any guidelines for what does and does
not go into the static table.

Looking at the current entries, I believe it would be useful to add at
least the following:
- :method PUT
- :method DELETE
- :method HEAD
- :status 201

And perhaps:
- :method
- :path (even this would save ~4 bytes IIUC, vs. the first occurrence)
- :status

I'm coming from a constrained device perspective, so the more in the
static table (code) vs. the header table (RAM), the better, as far as I
can tell.

Is there a reason these should not be added?

Also, I find it odd that common content types are not in the static table
(e.g., "text/html", "image/jpeg", "image/gif", "image/png") for
content-type and perhaps accept.  And perhaps content-encoding for gzip
and deflate.

- Robby

Robby Simpson, PhD
System Architect
GE
Digital Energy

---

From: Martin Thomson <[email-masked]>
Date: Thu, 7 Aug 2014 13:26:24 -0700

> Looking at the current entries, I believe it would be useful to add at

The values we have there are based on a frequency analysis provided by
Akamai.  The methods and status codes that we have account for some
ridiculously large proportion of requests.  And note that every entry
makes the header table larger, which increases the number of bytes
needed to reference the header table.

> - :path (even this would save ~4 bytes IIUC, vs. the first occurrence)

Bare values aren't necessary.  You can reference an entry with a value
and provide a different value.

> I find it odd that common content types are not in the static table

See above.  There just isn't enough commonality in these header fields
to justify adding the extra entries.  We just don't have enough good
data.  And getting good data is only a small part of the problem;
we're basically done here, so changes like this require overpowering
justification.

---

From: "Kulkarni, Saurabh" <[email-masked]>
Date: Thu, 7 Aug 2014 16:27:01 -0500

Martin,

Since now we have switched the static table to the lower indices, we
should try to see if we can populate the entries if possible with their
most-popular values. For e.g. Œcontent-encoding: gzip¹ is better than
Œcontent-encoding: <empty>¹. Having an empty value made sense when we
always added entries to header table before, but now it just seems like we
are missing an opportunity to further get some compression thru populating
all the headers.

- Saurabh

---

From: Martin Thomson <[email-masked]>
Date: Thu, 7 Aug 2014 14:50:20 -0700

> are missing an opportunity to further get some compression thru populating

I just went through the list; there really isn't much that we can add
that is unambiguously good like that.

Content-Encoding seems like the only good example.  Cache-Control: no-cache too.

But see previous comments regarding changes.

---

From: "Kulkarni, Saurabh" <[email-masked]>
Date: Thu, 7 Aug 2014 16:59:34 -0500

Well even if say content-type: text/html is used 10% of the time, isn’t it
good to have one of these rather than having an empty entry. I am not
proposing enumerating all the combinations, its just to replace the
content-type: empty-string with an entry with non-empty string value. So
for values different from text/html you can still use the index with a
different value, but for text/html you can just refer to the entry.

- Saurabh

---

From: <[email-masked]>
Date: Thu, 7 Aug 2014 22:59:43 +0000

> On 07 Aug 2014, at 23:54, "[email-masked]" <[email-masked]> wrote:

I think we should at least keep a list (wiki page) of these micro optimizations so that we could discuss further if the draft doesn't make it out if last call for some reason.

This email message is intended only for the use of the named recipient. Information contained in this email message and its attachments may be privileged, confidential and protected from disclosure. If you are not the intended recipient, please do not read, copy, use or disclose this communication to others. Also please notify the sender by replying to this message and then delete it from your system.

---

From: Martin Thomson <[email-masked]>
Date: Thu, 7 Aug 2014 16:31:45 -0700

> I think we should at least keep a list (wiki page) of these micro optimizations so that we could discuss further if the draft doesn't make it out if last call for some reason.

Last call doesn't mean that we are forbidden from discussing changes.
It's not special in that regard.  It's primarily time that causes the
threshold to be raised.  If implementations have to go back and make
other changes, then it definitely would be easier to throw in a few
small optimizations like this, even larger ones (like reworking the
HPACK opcode sequences), but I predict that any change, no matter how
small, will get some resistance.

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 8 Aug 2014 11:10:00 +1000

On 8 August 2014 09:31, Martin Thomson <[email-masked]> wrote:

> > I think we should at least keep a list (wiki page) of these micro
> optimizations so that we could discuss further if the draft doesn't make it
> out if last call for some reason.
>

I did some rough frequency analysis on the test data and came up with extra
values for:

"accept-language","en-US,en;q=0.5"
"accept-ranges","bytes"
"accept","image/png,image/*;q=0.8,*/*;q=0.5"
"age","0"
"allow","GET"
"cache-control","no-cache"
"content-disposition","attachment"
"content-encoding","gzip"
"content-language","en-US"
"content-length","0"
"content-type","image/jpeg"
"vary","Accept-Encoding"

Adding these values improved compression of the test data by 1.1%

However, removing anything that looks like linguistic imperialism and
making a few more neutral choices gives:

"accept-ranges","bytes"
"accept","*/*"
"age","0"
"allow","GET"
"cache-control","no-cache"
"content-disposition","attachment"
"content-encoding","gzip"
"content-length","0"
"content-type","application/x-javascript"
"vary","Accept-Encoding"

This still achieved 0.9% saving.

However I'd like to see some rigour applied to selecting the most frequent
value to apply, and I don't think the test data is large enough nor
representative enough for this.

So I think this is a good change to make  IFF we are breaking the protocol
for another reason, but it's probably not good enough to justify breaking
on it's own (unless someone can produce hard numbers).

So I'm happy to put this one on a list of breaking changes to make if we
are going to make breaking changes.

cheers

-- 
Greg Wilkins <[email-masked]>
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.

---

From: Martin Thomson <[email-masked]>
Date: Thu, 7 Aug 2014 19:38:28 -0700

Thanks Greg, I opened https://github.com/http2/http2-spec/issues/587 ,
which I will mark as non-blocking.
On Aug 7, 2014 6:10 PM, "Greg Wilkins" <[email-masked]> wrote:

> I did some rough frequency analysis on the test data and came up with
> extra values for:
>

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 08 Aug 2014 06:11:31 +0000

--------
In message <[email-masked]>

>representative enough for this.

There is at least one header where raw statistics will mislead us:
Accept-Encoding is subject to stylistic variance with no subtantive
difference in effect.

The three main values I see right now are:

	Accept-Encoding: gzip
	Accept-Encoding: deflate, gzip
	Accept-Encoding: gzip,deflate
	Accept-Encoding: gzip, deflate

To which there is no substantive difference at the end of the day.

Adding one of them to HPACK means that most implementations can
shave 10-ish bytes of the first request without any substantive
change to their codebase.

As author of bikeshed.org I really don't care if it is "gzip,deflate"
or "deflate,gzip" but statistics could tip the scales for just
"gzip" as the most common subset.

So if you have time Greg, I would appreciate numbers for two tests:

A) fold all A-E's containing both "gzip" and "deflate" to the same
   static entry, leave the rest alone.

B) fold all A-E's containing "gzip" to the same static entry, leave
   the rest alone.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 8 Aug 2014 17:59:33 +1000

> B) fold all A-E's containing "gzip" to the same static entry, leave

I can do experiments, but the data set I have only has "gzip, deflate" and
no other variations, so the results will be pretty meaningless.
So I really need a better data set.

Note that if we end up doing this, I'm not opposed to picking something
that is not the most frequent in order to encourage convergence and less
stylistic variations.  ie if we don't want deflate anymore, let's pick
"gzip" even if "gzip, deflate" is more frequent.

cheers

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 08 Aug 2014 08:21:13 +0000

--------
In message <CAH_y2NH0EiPSrn3o+3AqQQ=[email-masked]>

>"gzip" even if "gzip, deflate" is more frequent.

+1

---

From: "Martin Nilsson" <[email-masked]>
Date: Fri, 08 Aug 2014 13:15:40 +0200

> needed to reference the header table.

Just because it is frequent doesn't mean we should support it. I think  
that the :status 400, :status 404 and :status 500 should go away. We don't  
have to fail efficiently, and response headers are less interesting to  
optimize than request headers.

/Martin Nilsson

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 08 Aug 2014 11:23:28 +0000

--------

>optimize than request headers.

"We don't have to fail efficiently" runs counter to everything I have
ever read, learned or experienced about high performance computing.

---

From: "Simpson, Robby (GE Energy Management)" <[email-masked]>
Date: Fri, 8 Aug 2014 19:46:45 +0000

>needed to reference the header table.

Understood.  The suggested entries above are from a different perspective.
 While I concede that Web is the primary use case, and that I'm certain
Akamai has provided great data for that, the above suggestions come from
my experience in the IoT space.

These actually account for a great deal of the traffic there, and often
headers are a large part of the overall message payload.

>and provide a different value.

That's great.  My apologies - I did not catch that from my read of hpack.
Then I would certainly agree with others that having values for all of the
entries is a no-brainer.

>we're basically done here, so changes like this require overpowering

Understood.  My fear is that the static table is immutable going forward,
so we may have to live with it for a long time.

One other issue I've found in the static table entries:  I've noted that
they are all in alphabetical order, which makes searching much easier.
However, there appears to be one outlier -- "accept."  Since "/" comes
before "/index.html", shouldn't "accept" come before "accept-charset",
etc.?

---

From: Martin Thomson <[email-masked]>
Date: Fri, 8 Aug 2014 13:04:26 -0700

> before "/index.html", shouldn't "accept" come before "accept-charset",

Ordering is another area where we didn't have good enough data.

And yes, accept is lexically before accept-charset.  See previous
comments. I'm sure that will annoy some people, but I'm honest enough
to admit that I get a small amount of enjoyment from annoying the
obsessive-compulsive.

---

From: "Simpson, Robby (GE Energy Management)" <[email-masked]>
Date: Fri, 8 Aug 2014 20:17:26 +0000

>Ordering is another area where we didn't have good enough data.

What kind of data would you be looking for there?  Search times?

>to admit that I get a small amount of enjoyment from annoying the

Not sure the tradeoff in your enjoyment with annoying those with disorders
outweighs efficient implementation of searching the static table.

Granted, my code is not yet polished, but it seems to now require
additional entries/pointers for the index (if kept in alphabetical order
for searching) or searching the entire list (if kept in index order).  I'm
also guessing that some may not catch this while implementing.

Once again, not necessarily worth stopping the presses, but something we
should track if we have the chance to make changes.

---

From: Martin Thomson <[email-masked]>
Date: Fri, 8 Aug 2014 14:04:03 -0700

> What kind of data would you be looking for there?  Search times?

Actually, I imagine that searching will be basically the same no
matter how you distribute the entries.  But if you recall, we used to
have static tables at the end, so having common values appear early in
the table would have been a real compression gain. Smaller values
avoid needing extra bytes to encode the index.  This is still a small
thing, but only for the first 14 entries.