Subject: Permitted characters in HTTP/2 fields

From: "Martin Thomson" <[email-masked]>
Date: Tue, 27 Apr 2021 15:55:00 +1000

At our last interim, we discussed potential ways in which HTTP/2 was probably too strict about characters (octets really) in field names and values.

The conclusion then was to loosen the restriction and mandate only a small set of checks.  This should match what implementations already do.

https://github.com/httpwg/http2-spec/pull/846 proposes the following rules:

Field names and values can't contain CR, LF, or NUL.
Field names and values can't start or end with SP or HTAB.
Field names can't include uppercase ASCII (A-Z) or COLON (with an exemption for pseudo-headers).

This loosens the previous rule that made a message malformed if the field value was not valid according to the core ABNF definitions.  With the change, messages are only malformed if the processing rules of a field explicitly say that the message is malformed.

This differs in one way from what was recorded from our previous discussion: colons.  I realized that the main reason for the rules was to avoid things like request smuggling attacks, for which missing colons could be problematic.  Most of these rules exist to avoid conversions from HTTP/2 to HTTP/1.1 resulting in smuggling, so it makes sense to be a little more comprehensive in the approach.

The idea here is that this makes intermediary responsibilities clear: the above checks are mandatory because they have direct security and interoperability consequences, but that is all that HTTP/2 requires of implementations.  Beyond that, the core rules of HTTP apply.

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 27 Apr 2021 14:38:19 +0200

Hi Martin,

On Tue, Apr 27, 2021 at 03:55:00PM +1000, Martin Thomson wrote:
> At our last interim, we discussed potential ways in which HTTP/2 was probably
> too strict about characters (octets really) in field names and values.
> 
> The conclusion then was to loosen the restriction and mandate only a small
> set of checks.  This should match what implementations already do.
> 

I'm having a minor concern with this last one because these are permitted
for values in HTTP/1, it's just that these SP/HT are not part of the
value, and I suspect that by being too strict on this we could cause
some breakage where there is a direct H1->H2 conversion; I don't know
in fact if anyone carefully strips leading/trailing whitespaces from
H1 before passing them to H2 right now, but we may end up declaring
some existing implementations suddenly non-compliant.

>  Beyond that, the core rules of HTTP apply.

Just to be sure (as I'm not certain I'm interpreting that last sentence
correctly). You mean that the all other restrictions from the core rules
still apply (in which case the current subset of characters remains
limited), that's it, or anything else maybe ?

Thanks,
Willy

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 28 Apr 2021 08:00:16 +1000

I really Really don't like rules that are fuzzy or only sometimes enforced.

By all means change the permissible character in fields, but do it with
precise ABNF. If there is to be a difference between what SHOULD be sent
and what MUST be checked, then have two sets of ABNF.

If we have precise ABNF then some non normative text that says it is ok to
violate that ABNF, then it just makes for a very poor standard.

Ultimately implementor should be able to rely on ABNF to let them implement
precisely correct parser and generators.