Subject: HTTP2: Header Field Name.

From: "Felix" <[email-masked]>
Date: Tue, 03 Oct 2017 07:12:33 +0000

Hi.
I was writing a h2 server lately.
I'm failed to find a good reason from rfc7540 or rfc7541 to necessarily convert header field name to lowercase.

There are some disadvantages for a h2-to-h1 proxy about it:
1. Before forward a header stream to h1 endpoint. It may need to compare every header name. If the name is an iana registered name, convert it to upper camel-case. For example, convert "content-type" to "Content-Type". Apparently, this is not very efficient.
2. User could add a custom header entry with a name "MyParameter" to headers. If sender's sdk return error about it, that just make people think http2 is a step backward compare to http. If sender's sdk converts the name to lowercase then send it. The h1 receiver will not get "MyParameter" value, cause it's named "myparameter" now.

If we remove the lowercase rule. There is an advantage:
I don't know why header names are dash-line connected nowadays.
But if we will add some new feature with a new header to http2. We can use upper camel-case name. Compare CoolFeature to Cool-Feature, that's one byte saved. If we only allows lowercase, the dash-line is required. Without dash-line, it's not readable.

If we remove the lowercase rule. We need remove rules described in section 8.1.2, 8.1.2.6 in rfc7540. Turn Static Table Entries back to upper camel-case in rfc7541.

This change will affect all http2 implementations. It's not too late since http2 is not widely used yet.

---

From: Daniel Stenberg <[email-masked]>
Date: Mon, 9 Oct 2017 09:14:12 +0200 (CEST)

On Tue, 3 Oct 2017, Felix wrote:

> 1. Before forward a header stream to h1 endpoint. It may need to compare 
> every header name. If the name is an iana registered name, convert it to 
> upper camel-case. For example, convert "content-type" to "Content-Type". 
> Apparently, this is not very efficient.

Well, you shouldn't have to. HTTP/1.1 has case insensitive header names so you 
can pass lower case versions of them to the server and it should work.

I say *should* because I know this is a common reason for problems with 
upgrading to h2 with legacy code that makes wrong header case assumptions. 
Such code should be fixed.

> http2 is not widely used yet.

I'd say HTTP/2 is widely used. Over 17% of the top 10 million sites and almost 
35% of the top 1000:

   https://w3techs.com/technologies/comparison/ce-http2

-- 

  / daniel.haxx.se

---

From: Amos Jeffries <[email-masked]>
Date: Tue, 10 Oct 2017 01:19:58 +1300

> 1. Before forward a header stream to h1 endpoint. It may need to compare every header name. If the name is an iana registered name, convert it to upper camel-case. For example, convert "content-typ…

This is wrong. As Daniel said already HTTP/1 header names are case 
insensitive. So the h2 name is what the proxy should output.

If your proxy has a registry of HTTP/1 names it should make some minimal 
effort to produce the defacto standard casing, but is not required to.

> I don't know why header names are dash-line connected nowadays.

They are because they are. The dash is a character the author(s) 
defining a header get to choose from just like the numbers and alphabet 
characters. Some choose to use it, some dont.

> But if we will add some new feature with a new header to http2. We can use upper camel-case name. Compare CoolFeature to Cool-Feature, that's one byte saved.

Only in the human-readable format.

HTTP/2 is binary protocol. Having case sensitivity increases the machine 
versions size through increased static/dynamic table entry counts, 
increases the risk of permutation attacks, and adds lots of small 
incidental issues with case management have to be avoided for it all to 
work right.

> If we only allows lowercase, the dash-line is required. Without dash-line, it's not readable.

Consider it this way:

"CoolFeature" and "COOlfEAtUrE" are the same name in HTTP (both 1.x and 
2). As are all other possible permutations of upper and lower case 
characters.

Now how does the HPACK encoder encode all possible case *insensitive* 
permutations use "the one true name" when it is case-sensitive and has 
no list of all future defined headers casing?

Also, the lower casing is not new in HTTP/2. It exists implicitly in 
HTTP/1 as well through the case-insensitive definition. HTTP/2 makes 
naive agents simpler as the case sensitivity no longer has to be 
accounted for and provides a nice big stick to get bad implementations 
fixed when converting to h2.

> This change will affect all http2 implementations. It's not too late since http2 is not widely used yet.

On the contrary; As Daniel said, HTTP/2 is already widely used and 
growing quite fast.

Amos

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 9 Oct 2017 21:44:14 +0200

On Tue, Oct 03, 2017 at 07:12:33AM +0000, Felix wrote:
> But if we will add some new feature with a new header to http2. We can use
> upper camel-case name.

What an horror! CamelCase was invented only to make editors add
auto-completion. Every time you use an acronym (ie: almost always in
network environments), you never know how the words have to be spelled.
People love to write "TransmissionControlProtocol", then later shorten
it to "TCP" when it's mixed with some other words ("NewTCP") but then
when considered as TCP and not a shortened form as the above, it becomes
"Tcp" so you'll have "TcpClose" and such stuff. I've yet to see code
where camel case remains consistent for acronyms!

> Compare CoolFeature to Cool-Feature, that's one byte saved.

That's wrong, because in HTTP "Cool-Feature" is the same as "cool-feature"
so you save one bit per character = 12 bits total by having the dash instead
of upper cases, compared to the former where each character must have a bit
to indicate the case. As long as you have on average less than one dash per
8 caracters, the dash wins. This counts when you start to use certain
compression algorithms such as Huffman.

> dash-line, it's not readable.

That's why there's the dash. In HTTP header field names, just like for file
names, config keywords, composed first names, the dash is used to associate
words together to form a larger non-breakable word.

Another thing to consider is that going case-sensitive in a text-oriented
protocol supporting intermediaries creates security issues, because you can
do whatever you want, you'll always find implementations not properly checking
the case for such a widely deployed protocol. These ones can be attacked by
bypassing the intermediaries. And it has already been the case 15 years ago
where some load balancers could not properly parse "content-length:" or
"cookie:". So better make this the norm (and save bits on the wire).

> http2 is not widely used yet.

No, for sure, it's just compatible with HTTP/1.0 and 1.1 which have been
around since 1996 or so, so probably not that many implementations will
notice if you change it after 21 years of unimaginable use cases :-)

Willy

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 10 Oct 2017 07:36:14 +0200

On Tue, Oct 10, 2017 at 01:19:03PM +0800, Felix wrote:
> Why you need extra bit to indicate the case? They are ascii character, one
> octet each no matter lowercase or uppercase without Huffman Encoding.
> Even by using Huffman Encoding.
> CoolFeature is 64(7+5+5+6+7+5+5+5+6+6+5+2) bits including padding bits.
> cool-feature is 72(5+5+5+6+6+6+5+5+5+6+6+5+7) bits including padding bits.

I'm not talking about the static huffman table we have to encode anything
including URLs and values, I'm talking about encoding in general. What
I'm saying is that (to make things simple) you need ~4.81 bits per character
to encode any letter, a dash and an end marker (28 symbols). You need ~5.73
bits per character to encode any letter with its case and an end marker (53
symbols). So at minima you'll need 68.76 bits for the camel case version,
against 62.40 for the dash version. That's one less octet for the dash
version. You can turn it however you want, you won't change this because
it's a matter of alphabet complexity.

If you want to turn it back to plain h2 encoding, then cool-feature could
very well be one or two bytes by being in the table (static or dynamic).
And there its presence in lower case makes it much easier and cheaper to
lookup existing entries using hashes, trees, dichotomy or even perfect
hashes.

Willy

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 10 Oct 2017 08:11:28 +0200

On Tue, Oct 10, 2017 at 01:56:57PM +0800, Felix wrote:
> Hi. 
> I'm just curious about how's this conclusion of "~4.81 bits per character
> to encode any letter" coming from?

Simply log2(28). (28 symbols for 26 letters + dash + end).

Willy

---

From: "Felix" <[email-masked]>
Date: Tue, 10 Oct 2017 05:20:06 +0000

> compression algorithms such as Huffman.

Why you need extra bit to indicate the case? They are ascii character, one octet each no matter lowercase or uppercase without Huffman Encoding.
Even by using Huffman Encoding.
CoolFeature is 64(7+5+5+6+7+5+5+5+6+6+5+2) bits including padding bits.
cool-feature is 72(5+5+5+6+6+6+5+5+5+6+6+5+7) bits including padding bits.
Still one byte less. Not to mention a character less to be decoded by endpoint to save some of CPU/Memory resources.

---

From: "Felix" <[email-masked]>
Date: Tue, 10 Oct 2017 05:57:33 +0000

Hi. 

------------------ Original ------------------
From:  "Willy Tarreau";<[email-masked]>;
Date:  Tue, Oct 10, 2017 01:36 PM
To:  "Felix"<[email-masked]>;
Cc:  "ietf-http-wg"<[email-masked]>;
Subject:  Re: HTTP2: Header Field Name.

Willy