Subject: SETTINGS frames as header blocks?

From: Alek Storm <[email-masked]>
Date: Sun, 13 Apr 2014 16:49:14 -0700

Hi all,

I have a radical suggestion for the SETTINGS frame: rather than serializing
it as a sequence of (id, value) pairs, where id and value are 8- and 32-bit
integers, respectively, use a header block, as in HEADERS/PUSH_PROMISE/
CONTINUATION. This offers the following benefits:

   -

   Extensibility: By allowing setting names to be arbitrary strings, users
   can introduce custom settings without worrying about collisions in a
   255-element namespace, and without having to maintain some sort of registry.
    -

   Expressiveness: String values allow complex settings to be encoded -
   this has been widely, and successfully, used in HTTP headers (e.g. the Link
   header <http://tools.ietf.org/html/rfc5988#section-5.5> and the
cache-control
   headers <http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9>
   ).
    -

   Simplicity: Making SETTINGS values more expressive would allow the new
   ALTSVC frame to become a setting, rather than a separate frame, since it
   confers only fire-and-forget connection-level metadata. In fact, it could
   follow the format of the Alt-Svc HTTP
header<http://tools.ietf.org/html/draft-ietf-httpbis-alt-svc-01#section-3>.
   Other extensions to the protocol can be added the same way, just as
   Alt-Svc was added to HTTP/1.1 - in contrast, "extension" frame types are
   a non-starter, since implementations are required to error out when
   receiving an unrecognized frame type.

There is also a semantic argument to be made: just as HEADERS frames
express request/response-level metadata, SETTINGS frames express
connection-level metadata (in fact, HTTP/1.1, which lacks the latter
mechanism, has made use of header fields on individual requests to express
properties about the connection as a whole, e.g. Alt-Svc and the hop-by-hop
headers, like Connection and Upgrade).

Adding header-table-size, enable-push, max-concurrent-streams, and
initial-window-size to the initial static HPACK table (or even maintaining
a separate compression context and static table) should allow us to achieve
a final SETTINGS frame size comparable to the current serialization format.
I've been contributing to hyper <https://github.com/lukasa/hyper>, a Python
client implementation, lately, and I'd be happy to give this proposal a
whirl in a branch and report back with before-and-after benchmarks.

Apologies if the above has been proposed before - I wasn't able to find
anything in a search of the archives, but I may have missed something.

Best,
Alek

---

From: Amos Jeffries <[email-masked]>
Date: Mon, 14 Apr 2014 14:16:21 +1200

>    headers <http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9>

This point is a *downside* of textual representations. The purpose of
HTTP/2 is to rollout a fast and efficient binary protocol. The SETTINGS
frame is a mandatory to parse+process piece of that protocol.
 Forcing all implementations to contain a textual headers parser just
for SETTINGS re-introduces one of the major problems of HTTP/1 -
complexity and difficulty handling all possible textual input. Note the
original proposals for HTTP/2 were 2:1 against having HEADERS frames in
textual format as well.

>    receiving an unrecognized frame type.

If the editors choose to make ALTSVC a SETTINGS field it would be *far*
simpler to define ALTSVC as a setting in the binary form with an opaque
N-byte value following it. Textual SETTINGS content adds nothing but
complexity for this use-case.

> headers, like Connection and Upgrade).

HTTP/2 being binary protocol represents them as fixed binary fields in
appropriate places. Allowing simpler parsers with zero chance of parsing
errors from decoding text values into binary form - as seen routinely in
HTTP/1 traffic.

> a final SETTINGS frame size comparable to the current serialization format.

Trading simplicity and speed which SETTINGS can be both created and
concumed for extra complexity and difficulty writing implementations.
While simultaneously increasing the state all implementations are
required to store long-term.

Also, your proposal earlier noted that complex and flexible textual
values would be a benefit. This contradicts the idea of having fixed
values in the static HPACK table. One cannot put textual names or values
that are unknown into the static HPACK table.

> anything in a search of the archives, but I may have missed something.

FYI: In one of the original proposals (network-friendly draft) we
started with simply separating the HTTP/1 end-to-end an hop-by-hop
headers into two frame types equivalent to SETTINGS and HEADERS. It was
even then intended to convert to binary forms similar to what is seen in
SETTINGS for most of the reasons I mention above and probably a lot of
others.

Amos

---

From: Alek Storm <[email-masked]>
Date: Sun, 13 Apr 2014 21:13:14 -0700

Thanks for the detailed reply! Comments below.

On Sun, Apr 13, 2014 at 7:16 PM, Amos Jeffries <[email-masked]> wrote:

> >    headers <
> http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9>
> >    ).
>

> > header<
> http://tools.ietf.org/html/draft-ietf-httpbis-alt-svc-01#section-3>.

That's an excellent point. As I understand it, HPACK encodes an opaque
series of octets as a header value. Therefore, it should be possible to use
the binary format of the current ALTSVC frame as the value of the proposed
`alt-svc` SETTINGS field. Would you be amenable to that solution?

> errors from decoding text values into binary form - as seen routinely in

Agreed.

> required to store long-term.

I think the complexity argument is subjective, or at least different in
each implementation. For example, in hyper, it's quite trivial to reuse the
HPACK code to populate SETTINGS - I believe it would be fewer lines of code
than the custom SETTINGS format, in fact. And I don't think there's any
additional state - it's the same compression context, the size of which is
bounded by `header-table-size`.

> that are unknown into the static HPACK table.

I don't know what you mean here - unless I've fundamentally misunderstood
HPACK, the static table is only an optimization for commonly-used
names/values. HPACK can certainly encode other name-value pairs; my point
was just that for the settings that have already been defined in the spec,
HPACK encoding is comparably efficient. Custom extensions will certainly
incur additional overhead. Basically, the total size of a SETTINGS frame in
the current format and an ALTSVC frame is roughly equal to the size of an
HPACK-encoded SETTINGS frame with an `alt-svc` setting in binary format.

> SETTINGS for most of the reasons I mention above and probably a lot of

Thanks again for your response! FWIW, I completely agree with your primary
objection (that textual representations are hard to parse), so I'd like to
modify my proposal to specify that all SETTINGS values will retain their
current binary format (including `alt-svc`). Would love to get your
thoughts on the updated version.

Best,
Alek