Subject: reserved flags and extensions

From: Willy Tarreau <[email-masked]>
Date: Sun, 13 Jul 2014 08:06:08 +0200

Hi,

it has been suggested several times that future extensions could change
the framing. Reserved flags are marked as "must be sent as zero" and "must
be ignored on receipt". However, if some intermediary is designed to only
inspect contents and relay frames as-is (think an anti-virus or so which
only cares about data and ignores everything else), it might very well
end up desynchronized when future extensions affecting the framing are
negociated between the client and the server and it ignores them. This
is exactly the issue TCP has long been facing with window scaling and
middle firewalls which did not support it and would block supposedly
out-of-window packets.

It would be sad to discover 1 year after the protocol is released that
we cannot update the framing to benefit from the wider experience in
field, just because of some dumb components breaking the net again.

And the problem of extensions is that nobody is required to know them in
advance, so nobody knows in advance if leaving them enabled can cause some
breakage or is desirable.

One radical solution to this I'm seeing is to explicitly require that
any unknown extension must be removed from the settings frames by these
"dumb" intermediaries designed to forward everything. This might not be
the easiest thing for them, and would lead to other useful extensions to
be removed.

Another solution would help dumber intermediaries and would avoid getting
rid of all extensions : we can define *right now* that one of the reserved
bits will indicate "alternate framing settings present" and that all
extensions which affect the framing rely on this bit to be set. That way,
the dumber intermediaries which are not capable of updating the settings
could simply clear this bit and force the communication to fall back to
the legacy specification, and the recipient will simply ignore these
extensions.

The last option I'm seeing is to leave things as they are right now and
to add a specific paragraph in the spec for those dumb intermediaries
which we could call "frame sniffers" or so, and indicate what the absolute
minimal amount of processing can be : you can forward everything as-is
but you MUST at least clear all reserved bits [list here].

What do you think ?
Willy

---

From: Yoav Nir <[email-masked]>
Date: Sun, 13 Jul 2014 15:57:03 +0300

I think regardless of what we do, changing the framing of HTTP/2 will cause breakage by at least a subset of “frame sniffers”, and therefore that is something we shouldn’t do.

If we want a different framing, we should neg^H^H^H advertise it not in HTTP/2’s SETTINGS frame, but in ALPN/Upgrade and call it HTTP/3 or HTTP/2.1. 

Yoav

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 13 Jul 2014 19:37:40 +0200

On Sun, Jul 13, 2014 at 03:57:03PM +0300, Yoav Nir wrote:
> I think regardless of what we do, changing the framing of HTTP/2 will cause
> breakage by at least a subset of ?frame sniffers?, and therefore that is
> something we shouldn?t do.

It just depends whether we consider that they will exist and whether or
not we indicate them an easy fallback solution (eg: clearing a bit must
be possible even for hardware-assisted devices). Anyway, I'm mentionning
this since larger frames have been suggested several times as been
proposed later as an extension and I'm quite sure that at least without
minimal efforts like this it cannot be done at all.

> If we want a different framing, we should neg^H^H^H advertise it not in
> HTTP/2?s SETTINGS frame, but in ALPN/Upgrade and call it HTTP/3 or HTTP/2.1. 

The problem here is that it may quickly require a large number of versions
especially if multiple extensions are defined.

Regards,
Willy

---

From: Martin Thomson <[email-masked]>
Date: Sun, 13 Jul 2014 13:21:05 -0700

> especially if multiple extensions are defined.

I think that this is the path we walk.  Anything major will need a new
ALPN tag.  I would certainly advise people that this is better than
something else.

That said, I'm not particular sympathetic to the use case of having an
intermediary that isn't really a participant in the protocol.  I know
this happens, but I think that the onus is on those entities to deal
with the consequences of the protocol, in all its aspects.  That means
not passing on settings (and frames, etc...) they don't understand.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sun, 13 Jul 2014 20:36:23 +0000

>> HTTP/2?s SETTINGS frame, but in ALPN/Upgrade and call it HTTP/3 or HTTP/2.1. 

I think ALPN/Upgrade should be reserved for such major steps.  We
do not want to use such a big hammer for small steps like an improved
header-compressor or some other optimization.

Also, some protocol improvements (like CPU-heavy or large-state
compressions) might only be opened for once non-hostility has been
shown convincingly some way or other, and it would be best to not
require an entire new connection for that.

The problem with low-fidelity implementations is that they, as the
name says, are not going to pay a lot of attention to what we write
they should do.

We sort of have an unstated (should we?) principle so far of "I
allow but You choose" attitude to variance from the base protocol,
if we stick to that, all extensions/variances MUST be announced
in SETTINGS, but the other end is not obligated to actually use them.

Given that, low-fidelity proxies can simply strip anything from
SETTINGS they don't understand.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Matthew Kerwin <[email-masked]>
Date: Mon, 14 Jul 2014 10:24:38 +1000

> SETTINGS they don't understand.
This issue of announcing extensions in settings, and the chance for lo-fi
implementations to mangle things (e.g. forward settings but drop frames)
was the main reason I decided to alter my encoded-data extension draft [1]
to use a custom frame type to announce support for encodings, rather than a
setting. It has other benefits, but that was the main driver. I begin to
suspect that it will be nigh impossible to build an extension that modifies
existing frames or framing and works on the open internet, for this exact
reason. Thus all extensions under the banner of "h2" will have to be new
things (new settings, or new frame types).

[1] http://phluid61.github.io/internet-drafts/http2-encoded-data/

-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 14 Jul 2014 10:51:45 +1000

Willy,

I think that the only realistic way of expecting h2 extensions to have
worked through intermediaries was to have a framing layer that was
independent of HTTP semantics.   Then perhaps we could have had websocket
and similar type extensions work (although there is still an issue with
common routing meta data).

Because we have conflated HTTP semantics with framing, then the only way to
extend h2 will be to make your extended traffic look exactly like
non-extended http - regardless of semantics.

Unfortunately this is exactly the kind of protocol misuse the charter was
asking us to fix.  But then I think the charter kind of snookered us here
by asking us to fix the misuse, but then saying that we are only chartered
to carry HTTP semantics.

cheers

-- 
Greg Wilkins <[email-masked]>
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 14 Jul 2014 08:05:36 +0200

Hi Greg,

> by asking us to fix the misuse, but then saying that we are only chartered

Here my comments were not related to HTTP, just encoding. We've been used
to see bad intermediaries only sniff packets, other ones work in tunnel mode
and work as TCP proxies and just slightly mangle what passes through (haproxy
used to do that in its early versions), etc... We know that these ones can
cause trouble because they don't modify Connection or so. Here it will be
the same, with h2 we'll have some intermediaries capable of parsing the
framing to keep in sync and which will not have per-request buffer. They
won't interfer with flow control because they'll pass every frame they
receive. And if/when we later decide to slightly improve the framing as
was already suggested, they'll break, so as a result we'll always refrain
from doing any such change even as an extension by fear that these
undetectable intermediaries will fail and possibly hang. Note that this
is only about the framing and not about the transported protocol. That's
why I think we should take some advance on this and tell them how they
can ensure they'll remain compatible with future versions at a low cost
(eg: trim extensions or just clear one or a few bits).

We have an advantage over h1 : the protocol is so much non-obvious that
it requires implementers to read the spec. 15 years ago, everyone would
proudly implement his own server/client/proxy/gateway without reading
the spec, because he (thought he) knew. Here we're sure that the advices
will be (at least) read.

Willy

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 14 Jul 2014 08:18:47 +0200

> ALPN tag.  I would certainly advise people that this is better than

I know but it's possible that we'll have different compatible extensions,
and that will make it harder to use with ALPN since you'll have to advertise
all combinations.

> not passing on settings (and frames, etc...) they don't understand.

I agree, which is why I'd like to add some wording to their intent. In my
experience, you don't design an intermediary which breaks on purpose, you
just try to implement the minimalist set of features that ensures your code
works well for all your use cases. Haproxy started this way. It used to
only rewrite some cookie headers for a benchmark. Quickly it had to pass
"Connection: close" because it didn't handle keep-alive. And that was enough
for a long time. We all know how a convenient hack quickly becomes a widely
deployed product... If we help implementers avoid the biggest errors, it
may not work optimally for them but it will not break.

I think that requiring that tunnels which have to follow the framing remove
unknown extensions from settings frames (but not the frame since it may
require an ACK) and clear all reserved bits would be a minimum explicit
requirement. In order to make that more likely to be correctly implemented,
I was thinking that we could decide that no extension would modify the
framing without using one of the reserved bits, so if we only require
them to clear reserved bits, it should be enough for recipients to ignore
problematic extensions.

Willy

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 14 Jul 2014 08:31:42 +0200

Hi Poul-Henning,

> name says, are not going to pay a lot of attention to what we write

But at least they'll have to read the spec, so a few big warnings
for them to tell them how to remain reliable can be read, and can
be applied if the cost is low.

> SETTINGS they don't understand.

That was my goal, but for having seen lazy firewalls write "XXX"
over header fields or values because they were not able to change
the contents size, I'd rather suggest we offer an option for the
laziest possible implementation. Either we have some fill byte
to clear these extensions (why not, TCP has a NOP for that). And
we can simply insist that they must clear reserved bits even if
they are tunnels. In short : if you understand the framing you
must apply the senders' rules.

Maybe something along these lines :

   HTTP/1 did not allow multiple outstanding requests, allowing some
   products to work as tunnels and inspect contents on the fly. HTTP/2
   is a multiplexed protocol, and content inspectors will have to parse
   the framing. Any product which has to remain synchronized with the
   framing must comply with the rules that apply to any sender :
      - clear all reserved bits from outgoing frames (these bits will
        likely be used to negociate a different frame format with the
        other end).

      - trim extensions they do not know/support from settings frames
        (since these extensions may be used to change the framing).
        [ insert here how to easily trim extensions ]

Willy