Subject: disabling header compression

From: Peter Lepeska <[email-masked]>
Date: Thu, 12 Dec 2013 11:50:48 -0500

Is it possible for a browser, proxy, or web server to disable header
compression in HTTP2? If not, I'm sure there was discussion around this
decision but my search of the archives has not come up with anything.

Thanks,

Peter

---

From: Roberto Peon <[email-masked]>
Date: Thu, 12 Dec 2013 08:59:31 -0800

One can set a zero compression context size, in which case you have to
retain no state after it is applied.
-=R

---

From: James M Snell <[email-masked]>
Date: Thu, 12 Dec 2013 09:04:28 -0800

A receiving peer can set it's SETTINGS_HEADER_TABLE_SIZE to 0,
effectively disallowing the sending peer to store any state. That does
not disable header compression as a whole, however.

---

From: Michael Sweet <[email-masked]>
Date: Thu, 12 Dec 2013 12:20:23 -0500

Peter,

You can't disable it but you can always just send literal values.  The kicker, of course, is that you will still need to support decompression...

On Dec 12, 2013, at 11:50 AM, Peter Lepeska <[email-masked]> wrote:

> Is it possible for a browser, proxy, or web server to disable header compression in HTTP2? If not, I'm sure there was discussion around this decision but my search of the archives has not come up with anything.
> 
> Thanks,
> 
> Peter
> 
> 

_________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair

---

From: Peter Lepeska <[email-masked]>
Date: Thu, 12 Dec 2013 13:09:31 -0500

Thanks for the quick replies.

So just in summary, in the current spec compression is the choice of
the sender but receivers always must support decompression.

It would be nice to have just a nice clean DISABLE_COMPRESSION in the
SETTINGS frame so that receivers do not need to support decompression.
Has that been discussed? I know there is a concern of cluttering up
SETTINGS, but this seems like an important option.

Peter

On Thu, Dec 12, 2013 at 12:20 PM, Michael Sweet <[email-masked]> wrote:
> Peter,
>
> You can't disable it but you can always just send literal values.  The
> kicker, of course, is that you will still need to support decompression...
>
>

---

From: David Morris <[email-masked]>
Date: Thu, 12 Dec 2013 10:19:54 -0800 (PST)

I agree ... being able to disable compression is an important
complexity reduction for low foot print clients and/or servers.

I think it is also a very useful capability in support of general
testing and performance evaluation.

I'd also like so see an explicit mechanism for specification of
an alternate compression algorithm .. I think it is way to early
to lock down a specific approach and it would be very powerful to
allow easy deployement of an alternative.

---

From: Roberto Peon <[email-masked]>
Date: Thu, 12 Dec 2013 10:41:46 -0800

These things are all possible with ALPN.
There has been a proposal w.r.t. setting the default compression context
size to zero, but I don't think there was a lot of support for that.
... which, honestly, is reasonable if a shade disappointing: One can RST
the streams one doesn't like and wait for one RT for the setting of the
compression context size to occur at the remote end.

Alternate compression schemes are also doable via ALPN.

If you don't want to handle compression you:
1) Send a SETTINGS frame requesting that the receive side-context be sized
to zero.
2) RST all streams that would store state in the compression context until
you receive the acknowledgement that this has occurred from the remote side.
3) Proceed normally to handle stuff with zero context size.

While not optimal in the DoS case, it certainly would be difficult to argue
it doesn't work.
In the non-DoS case, you end up wasting an RT and a few bytes. I'd imagine
that much of the time when someone is interacting with such a low-footprint
thing that either the additional RT of delay isn't a big deal.

I don't think it is too much to ask people to handle the static table (it
really is just a lookup into an array at that point) or huffman stuff.
-=R

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 13 Dec 2013 11:11:49 +1300

> These things are all possible with ALPN.

ALPN, ALPN, ALPN!  I'm hearing a *lot* of dependency being placed on 
ALPN capabilities.
Are we designing HTTP/2 as a protocol or as a set of ALPN extensions?

I rather think we need to avoid locking so much HTTP/2 feature 
dependency into an *optional* extension feature of a protocol which only 
covers a sub-set of the HTTP use-cases. ALPN provides a performance 
optimization for starting HTTPS (both 1.x and 2.0), very little else.

If something does not work on HTTP/2-over-TCP without an intermediary 
protocol layer, even inefficiently, then it does not meet requirements 
implicit in section 2 of the HTTP/2 specification.

"
2. HTTP/2.0 Protocol Overview

    HTTP/2.0 provides an optimized transport for HTTP semantics.

    An HTTP/2.0 connection is an application level protocol running on
    top of a TCP connection ([TCP]).  The client is the TCP connection
    initiator.
"
... *TCP* ... no mention of mandatory TLS there, or anywhere else in 
section 2.*.

Please stop assuming non-TCP layers underneath HTTP/2.

Amos

---

From: Roberto Peon <[email-masked]>
Date: Thu, 12 Dec 2013 14:27:29 -0800

Protocol negotiation is extremely useful.
You can happily translate any time I say "ALPN" to mean "protocol
negotiation using the registry used for ALPN"

Don't worry so much :)
-=R

On Thu, Dec 12, 2013 at 2:11 PM, Amos Jeffries <[email-masked]> wrote:

> ALPN, ALPN, ALPN!  I'm hearing a *lot* of dependency being placed on ALPN
> capabilities.
> I rather think we need to avoid locking so much HTTP/2 feature dependency
> into an *optional* extension feature of a protocol which only covers a
> sub-set of the HTTP use-cases. ALPN provides a performance optimization for
> starting HTTPS (both 1.x and 2.0), very little else.
>

---

From: Matthew Kerwin <[email-masked]>
Date: Fri, 13 Dec 2013 08:47:34 +1000

> negotiation using the registry used for ALPN"

Are you proposing a protocol negotiation phase for non-TLS HTTP?  Would
this occur at a lower level (or earlier, like DNS), or inside HTTP itself,
or in a new intermediate level?  How would it integrate with a HTTP1->2
upgrade mechanism?

> Don't worry so much :)

I had the same reaction as Amos, just not the brass to voice it.  If any
negotiation that happens in ALPN affects the semantics of the HTTP session,
then there really should be a pathway to achieve it without requiring
support for ( that particular ALPN protocol token | ALPN in general | TLS
).  It seems a bit ripe saying: you can negotiate leaving out the
complexity of supporting compression, but only if you add in the complexity
of supporting TLS+ALPN.

-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: David Morris <[email-masked]>
Date: Thu, 12 Dec 2013 15:11:56 -0800 (PST)

Removing the implementation of the header compression will simplify the
minimal implementation of HTTP/2. Requiring support of the static table
increase complexity. Worse is the RST rain dance you describe as that
is essentially a hack adding complexity and increasing the probablility
of failure.

It at least partially makes any performance evaluations or other testing
attempts burdened with the overhead of opting out and perhaps introducing
additional failure points in the RST process. Certainly, if I as a
developer or system admin suspect an issue with header integrity, I can't
perform true isolation by turning off compression.

---

From: Roberto Peon <[email-masked]>
Date: Thu, 12 Dec 2013 15:12:51 -0800

> or in a new intermediate level?  How would it integrate with a HTTP1->2

ALPN, DNS, alt-svc, whatever. Upgrade is particularly clunky, and so I'd
prefer other things, but I wouldn't block someone standardizing it. I'd
just not implement it.

> complexity of supporting compression, but only if you add in the complexity

I'm not opposed to any particular mechanism so long as at least one
mechanism exists which is lowest-reasonable-possible-latency and is robust
in the face of the problems the internet throws at us.

Be careful about assertions about complexity in this area: the 'easy to
describe' or 'cheap' solutions are not the same as the 'reliably
deployable' solutions, and the set of each changes depending on the
use-case, clients, etc.
-=R

---

From: Roberto Peon <[email-masked]>
Date: Thu, 12 Dec 2013 15:25:12 -0800

On Thu, Dec 12, 2013 at 3:11 PM, David Morris <[email-masked]> wrote:

> is essentially a hack adding complexity and increasing the probability
> of failure.
>

Optionality/negotiation also increases the likliehood of failure.
It does not reduce complexity, but rather moves it around.

There is always a tradeoff between optional features and reliability.

-=R

---

From: Peter Lepeska <[email-masked]>
Date: Fri, 13 Dec 2013 10:50:02 -0500

+1 to what David said. I especially agree with this "Removing the
implementation of the header compression will simplify the
minimal implementation of HTTP/2", which I think is very important to
increase adoption.

Peter

On Fri, Dec 13, 2013 at 10:49 AM, Peter Lepeska <[email-masked]> wrote:
> +1
>
> On Thu, Dec 12, 2013 at 6:11 PM, David Morris <[email-masked]> wrote:
>>

---

From: Peter Lepeska <[email-masked]>
Date: Fri, 13 Dec 2013 10:49:20 -0500

+1

---

From: James M Snell <[email-masked]>
Date: Fri, 13 Dec 2013 07:54:29 -0800

+1.

On Fri, Dec 13, 2013 at 7:50 AM, Peter Lepeska <[email-masked]> wrote:
>[snip]

---

From: Patrick McManus <[email-masked]>
Date: Fri, 13 Dec 2013 11:17:57 -0500

This becomes exactly like sending Content-Encoded: gzip on requests in
http/1 if compression is optional. In other words - it would be horribly
broken.

game it out.. you increase adoption of servers that don't receive
compression and you don't negotiate that separately with alpn. Now clients
send a burst of 30 parallel requests in the same cwnd on the first rtt (the
real value of the compression) and they all get reset and have to start
over,, that's one wasted rtt.. and now the client has to send those same
requests again, but uncompressed and across 3 rtts because they don't fit
in the cwnd anymore.

That's +3 rtt. blech!

I don't want to increase adoption of whatever that lazy protocol is called
because it sucks compared to http/2. :)

I want http/2 to be able to always count on what makes spdy work well:
multiplexing, compression, prioritization, and [though some will panic at
its mere mention] better crypto. It requires some effort, stipulated.

fwiw - CRIME forced me to effectively disable gzip upstream in firefox-spdy
(its gzip formatted, but all literals) and its easy to see the +~2RTT
penalty in benchmarks. Give spdy a lot of credit here - it demonstrated the
value of header compression and that we don't want to do it with a stream
compressor. Running code ftw.

-P

On Fri, Dec 13, 2013 at 10:50 AM, Peter Lepeska <[email-masked]>wrote:

> >>> > > > On Dec 12, 2013, at 11:50 AM, Peter Lepeska <
> [email-masked]>
> >>> > wrote:
> >>> > > >

---

From: Peter Lepeska <[email-masked]>
Date: Fri, 13 Dec 2013 11:34:14 -0500

If it's on by default and disabled by a flag in SETTINGS, then we'd
still get the benefit you describe except when we've explicitly
decided we didn't want it. I think removing options (making more
things mandatory) is a way to manage complexity. But HPACK actually
ADDs complexity to any implementation so it makes sense to make it
optional. Lastly, your example of 30 parallel requests is only true
for rendering web pages. If I'm using HTTP 2.x to send a large file,
or any one of a large number of other use cases, header compression
gives me no benefit.

Peter

On Fri, Dec 13, 2013 at 11:17 AM, Patrick McManus <[email-masked]> wrote:
> requests again, but uncompressed and across 3 rtts because they don't fit in
> the cwnd anymore.
>
> That's +3 rtt. blech!
>
>> >>> > > > On Dec 12, 2013, at 11:50 AM, Peter Lepeska
>> >>> > > > <[email-masked]>
>> >>> > wrote:
>> >>> > > >

---

From: Patrick McManus <[email-masked]>
Date: Fri, 13 Dec 2013 11:45:33 -0500

this is all well trodden territory. HTTP/1 has taught us the folly of
optional features. (gzip requests, trailers, etc..)

Figure out what is important, define it, and then use it widely. HTTP/2 has
done a pretty decent job of that so far with really just one significant
exception (and that has a decent reason (flow control)).

On Fri, Dec 13, 2013 at 11:34 AM, Peter Lepeska <[email-masked]>wrote:

> >> >>> you receive the acknowledgement that this has occurred from the
> remote
> >> >>> side.
> >> >>> While not optimal in the DoS case, it certainly would be difficult
> to
> >> >>> argue
> >> >>> it doesn't work.
> >> >>> I don't think it is too much to ask people to handle the static
> table
> >> >>> (it
> >> >>> > > So just in summary, in the current spec compression is the
> choice
> >> >>> > > of
> >> >>> > > It would be nice to have just a nice clean DISABLE_COMPRESSION
> in
> >> >>> > > the
> >> >>> > > On Thu, Dec 12, 2013 at 12:20 PM, Michael Sweet <
> [email-masked]>
> >> >>> > wrote:
> >> >>> > > > Peter,
> >> >>> > > >

---

From: David Morris <[email-masked]>
Date: Fri, 13 Dec 2013 09:44:17 -0800 (PST)

I don't agree with the conclusion you reach from the HTTP/1 space.

HTTP/1 has been all over the map because there were multiple ways
to do the same thing (compression). The lesson I get is that we
need clear unambigous specifcation of the options that are defined.

Our design intent needs to remain replacing all possible requirement
for continuing use of HTTP/1.x. Mandatory support for header compression
will reduce adoption for the large space of devices which won't perceive
any significant value because of their particular use cases.

Being able to turn off header compression will make failure analysis
triage easier, and it will allow for more comprehensive performance
comparision.

My second point is that it should be easy to substitute an alternative
algorithm. We need to measure and compare alternatives.

I believe header compression is a very good idea, I'm not convinced that
the current algorithm is the best we can do.

---

From: Chris Burdess <[email-masked]>
Date: Fri, 13 Dec 2013 18:04:54 +0000

> the current algorithm is the best we can do.

Not compressing headers is simply a value of zero in the
compressing-headers space. We cannot know that there will not be a
better header compression algorithm developed tomorrow, and we shouldn't
prevent such an algorithm from becoming a de facto standard. If you
grant that header compression may take more than one form, it seems
logical to allow no compression also since there may be use cases that
support it.

---

From: Patrick McManus <[email-masked]>
Date: Fri, 13 Dec 2013 13:23:22 -0500

there is no negotiation before sending inside http/2. So the receiver must
be able to decode compressed headers in some form in order for the protocol
to work well. (resetting those streams and sending them again slowly does
not qualify as working well). Right now that base format is hpack.

We could add some kind of switch-alogirthms negotiation via settings that
could change things later on the fly.. (ironic given the motivation in this
thread is simplicity). Though I think just revving the protocol with *PN
makes a lot more sense.

---

From: Martin Thomson <[email-masked]>
Date: Fri, 13 Dec 2013 10:28:51 -0800

> exception (and that has a decent reason (flow control)).

I think that there's a simple out for someone who is somehow unwilling
to implement a decompressor, set the context to zero, and send
RST_STREAM on stuff that relies on the header table.  That will work
perfectly well at the cost of a round trip.

I'd rather that those implementations take that hit than every
implementation.  As Patrick says: game it out and you will see that
making it optional creates some perverse incentives.

(We made push optional, but that's because we don't have the same
clear indication that this is valuable.  Even there, the decision
wasn't certain.)

---

From: James M Snell <[email-masked]>
Date: Fri, 13 Dec 2013 10:48:13 -0800

Personally, I feel that if we have header compression in the protocol,
it shouldn't be optional. However, it does need to be recognized that
hpack is a brand new mechanism that *does* add significant new
complexity to the protocol, *does* drive up implementation cost, *is*
still largely unproven (saying "trust us our security guys say it's
ok" doesn't really count as "proof"), and has so far only been
implemented by a handful of people who appear to view the use of HTTP
through only very narrow Web browser centric glasses. You cannot sweep
these issues under the rug by shrugging and saying "trust us". Greater
care ought to be taken when adopting such significant new features
(and requirements). It would be interesting to get a gauge of just how
much consensus there is in the WG for adopting hpack as *the* header
compression mechanism for http/2.

On the question of adoption. Let me pose this question: what benefits,
if any, does adoption of HTTP/2 offer to developers of HTTP-based
RESTful APIs? What significant problems does HTTP/2 address that would
justify the implementation costs? (Or is this another, "well they can
just keep using HTTP/1" answer?)

- James

---

From: Peter Lepeska <[email-masked]>
Date: Fri, 13 Dec 2013 15:44:52 -0500

Patrick said: "Now clients send a burst of 30 parallel requests in the
same cwnd on the first rtt (the real value of the compression)"

I agree. This is actually the most compelling reason to do header
compression -- to send as many GET requests as possible on a single
cwnd. And my job is to make the web faster so I'm all about
performance gains, which this certainly is, but since we're designing
a protocol it does feel a little hackish. It reminds me of domain
sharding, which was a great idea until everyone realized that it
increased the impact of TCP slow start. I wonder if instead we should
be pushing for increasing initcwnd on more than just server operating
systems. Or using multiple TCP connections for our HTTP/2 sessions. Or
making the protocol itself leaner in the case when multiple GETs are
issued to the same web server. Anyway, adding header compression
primarily for what is essentially one narrow (and possibly short term)
case of a bad fit between HTTP and TCP feels like a potential detour,
especially since trading cpu for lower bandwidth usage makes less and
less sense as bandwidth gets cheaper and more abundant.

On an unrelated note I appreciate that adding enable/disable to
SETTINGS is complicated by the fact that the browser sends GETs before
receiving the server's SETTINGS frame and I would not want the browser
to wait. After thinking about it it seems like what Roberto suggested
is basically the best we can do if it is not part of protocol
negotiation and we don't want the browser to wait:

"If you don't want to handle compression you:
1) Send a SETTINGS frame requesting that the receive side-context be
sized to zero.
2) RST all streams that would store state in the compression context
until you receive the acknowledgement that this has occurred from the
remote side.
3) Proceed normally to handle stuff with zero context size."

I'm fine with this approach, though it's a little obscure. It'd be
nice if there was simply a "don't compress" button, but I get why that
is now.

Thanks,

Peter

---

From: Peter Lepeska <[email-masked]>
Date: Fri, 13 Dec 2013 15:46:11 -0500

Correction: Domain sharding specifically increased the impact of DNS
lookups on page load times.