Subject: h2#404 requiring gzip and/or deflate

From: Martin Thomson <[email-masked]>
Date: Thu, 20 Feb 2014 11:14:57 -0800

Mark raises a point on which the spec is a little vague:

https://github.com/http2/http2-spec/issues/404
--
9.3 GZip Content-Encoding says:

Clients MUST support gzip compression for HTTP request bodies.
Regardless of the value of the accept-encoding header field, a server
MAY send responses with gzip or deflate encoding.

... Is it both gzip and deflate (which last I checked, some clients
don't support)? If so, the first sentence and section title should be
changed to reflect this.
--

I think that this is largely inherited from SPDY, but I get the sense
that there is support for the concept in general.

I want to clarify the text above... Do we want to mandate (i.e., use
MUST) 1. gzip or 2. gzip+deflate ?

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Thu, 20 Feb 2014 20:38:36 +0100

>https://github.com/http2/http2-spec/issues/404

The gzip format is a container format and the container includes data
like filename, operating system, checksums, timestamps, comments, and
a DEFLATE stream with the compressed data. The overhead is not really
useful in this context, so mandating only gzip but not raw DEFLATE
support would be bad. Mandating both is okay, gzip is fairly trivial.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Thu, 20 Feb 2014 20:04:50 +0000

In message <[email-masked]>, Bjoer
n Hoehrmann writes:

>useful in this context, [...]

I disagree.

The overhead is minute, for instance the filename is optional and seldomly
used in HTTP context.

The CRC32 it brings is a very good integrity check which suprisingly
often uncovers trouble.

Gzip further has the advantage that there are plenty of command line
tools available for it, whereas deflate is quite hard to produce with
regular tools.

After considerable research, I decided to only support gzip in Varnish,
and I have only ever had one user ask for deflate, and that was based
on somebody raising the same bogus "overhead" argument as you do above.

If anything deflate should die.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 20 Feb 2014 21:08:14 +0100

On Thu, Feb 20, 2014 at 08:38:36PM +0100, Bjoern Hoehrmann wrote:
> a DEFLATE stream with the compressed data.

The other difference is the use of the faster adler32 checksum in deflate
instead of crc32 in gzip, so *if* we want to mandate something, deflate
is cheaper for both ends. That said, I'm still very concerned that we
want to mandate such antique bit-oriented algorithms which are extremely
slow and memory invasive while we have many much better ones such as
snappy, lz4, quicklz and I-don't-know-what which are much more friendly
for both ends and better suited for the 21th century's machines and
networks.

Willy

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 21 Feb 2014 15:13:12 +1100

The other aspect of this is that HTTP already says:

>    A request without an Accept-Encoding header field implies that the
>    user agent has no preferences regarding content-codings.  Although
>    this allows the server to use any content-coding in a response, it
>    does not imply that the user agent will be able to correctly process
>    all encodings.
> 
>    A server tests whether a content-coding for a given representation is
>    acceptable using these rules:
> 
>    1.  If no Accept-Encoding field is in the request, any content-coding
>        is considered acceptable by the user agent.

...

<http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-26#section-5.3.4>

So, what the current text *really* says is that servers can ignore the 'identity' content-encoding when it appears alone in the request.

Cheers,

--
Mark Nottingham   http://www.mnot.net/

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 21 Feb 2014 18:32:26 +1300

> So, what the current text *really* says is that servers can ignore the 'identity' content-encoding when it appears alone in the request.

Huh? identiy coding being explicitly sent is a non-empty field.

I read that as saying only an empty header means "anything goes, YMMV",
and * value meaning "anything goes, honest, I promise"

Amos

---

From: Matthew Kerwin <[email-masked]>
Date: Fri, 21 Feb 2014 15:39:57 +1000

On 21 February 2014 15:32, Amos Jeffries <[email-masked]> wrote:

> > So, what the current text *really* says is that servers can ignore the
> 'identity' content-encoding when it appears alone in the request.
>

The other current text, in HTTP/2, that says:

| Clients MUST support gzip compression for HTTP request bodies.
| Regardless of the value of the accept-encoding header field, a server
| MAY send responses with gzip or deflate encoding.

IOW "Accept-Encoding: identity" can be ignored, because you can still gzip
the response.

-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: Patrick McManus <[email-masked]>
Date: Fri, 21 Feb 2014 08:08:24 -0500

I think what the http/2 text means to say is that an "Accept-Encoding:
gzip, deflate" request header is implicit in every request and the client
needs to be able to process responses with those encodings.

On Thu, Feb 20, 2014 at 11:13 PM, Mark Nottingham <[email-masked]> wrote:

> <
> http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-26#section-5.3.4
> >
>

---

From: Martin Thomson <[email-masked]>
Date: Fri, 21 Feb 2014 09:13:28 -0800

On 21 February 2014 05:08, Patrick McManus <[email-masked]> wrote:
> I think what the http/2 text means to say is that an "Accept-Encoding: gzip,
> deflate" request header is implicit in every request and the client needs to
> be able to process responses with those encodings.

Yes, that is the implication.  Even if you include "Accept-Encoding:
rot13", you effectively get *either* "Accept-Encoding: rot13,
identity, gzip, deflate" or "Accept-Encoding: rot13, identity, gzip".
This is a change from 1.1, which implies "Accept-Encoding: rot13,
identity" only.

I had inferred the gzip+deflate, but Mark points out that this could
also be interpreted as the latter.  Hence this thread requesting
clarification.

I hope that I'm not completely off-base on this.  No one wants to
remove this entirely and revert to "identity" only, do they?

---

From: Roberto Peon <[email-masked]>
Date: Fri, 21 Feb 2014 09:38:03 -0800

No. Reverting to solely identity would be a huge step backwards.
On Feb 21, 2014 9:17 AM, "Martin Thomson" <[email-masked]> wrote:

> gzip,

---

From: Martin Thomson <[email-masked]>
Date: Fri, 21 Feb 2014 09:44:35 -0800

> No. Reverting to solely identity would be a huge step backwards.

That's what I thought.  Which would you prefer gzip + deflate or just gzip?

---

From: Roberto Peon <[email-masked]>
Date: Fri, 21 Feb 2014 09:50:46 -0800

The thinking when we put it into SPDY was that if you do gzip, you've
effectively required a deflate implementation, so one may as well allow it.

In the end, I'm ambivalent so long as there is a reasonable payload
compression requirement.
-=R

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 21 Feb 2014 19:09:52 +0100

>for both ends and better suited for the 21th century's machines and

I expect we will make sure through appropriate specification and testing
that we can deploy new compression schemes much more easily than it is
for HTTP/1.1, so I am not too concerned about that. Anyway, there are 3
formats here, RFC 1950 "zlib" is the format with Adler32 checksums; raw
DEFLATE is RFC 1951 without checksum (and the third is RFC 1952 "gzip").
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Albert Lunde <[email-masked]>
Date: Fri, 21 Feb 2014 13:02:18 -0600

On 2/21/2014 12:09 PM, Bjoern Hoehrmann wrote:
> * Willy Tarreau wrote:
>>[...]That said, I'm still very concerned that we
> for HTTP/1.1, so I am not too concerned about that. [...]

Another question is whether compression schemes introduce side channels
better to attack TLS. This has been mainly a concern with regards to 
authentication information in headers, but the BREACH attach:

http://en.wikipedia.org/wiki/BREACH_%28security_exploit%29

used HTTP body compression.

These are really attacks on web browsers rather than HTTP, as such, but 
in practical terms they are part of the larger problem space.

-- 
     Albert Lunde  [email-masked]
                   [email-masked]  (address for personal mail)

---

From: Martin Thomson <[email-masked]>
Date: Fri, 21 Feb 2014 11:16:21 -0800

On 21 February 2014 11:02, Albert Lunde <[email-masked]> wrote:
> better to attack TLS.

Yes, there is a concern here.  That's why we have padding.  There are
also other tools HTTP/2 provides, like push, that help mitigate these
attacks.

I think that it would be best if we left this problem to our friends
in TLS and websec and other more qualified groups.  If at some point
they come to us and say "we have decided to forbid compression" or,
more likely, "you are going to have to do the following things if you
want to use compression", then we will have to consider our options.

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 21 Feb 2014 20:20:27 +0100

>tools available for it, whereas deflate is quite hard to produce with

I think `gzip` is a good match for HTTP/1.1 where compression got added
later and the headers are so large that you do not feel the 18 bytes of
minimum overhead, but for a mandatory-to-implement compression scheme I
do not think tools make for a good argument, and I note that libraries
in my experience more commonly support the RFC 1950 zlib format or raw
RFC 1951 streams than RFC 1952 gzip streams; if checksums and an estab-
lished format are important, RFC 1950 has only 6 bytes minimum overhead
(2 bytes flags, 4 bytes Adler32 checksum).
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Michael Sweet <[email-masked]>
Date: Fri, 21 Feb 2014 15:17:59 -0500

Willy,

> for both ends and better suited for the 21th century's machines and

Other than lz4, it looks like the others need at least an order of magnitude more memory than gzip/deflate, which is particularly important for servers and embedded clients since they tend to be memory constrained...

Regardless, there is an IANA registry for this, so nothing prevents future clients and servers from supporting better compression algorithms as they become available/accepted.

_________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair

---

From: Zhong Yu <[email-masked]>
Date: Fri, 21 Feb 2014 14:46:46 -0600

> effectively required a deflate implementation, so one may as well allow it.

If everybody can use either of the algorithms with ease, there's no
point to mandate both. I also agree with Bjoern's point that
gzip/deflate is pretty slow; enabling it may actually decrease
throughput in a lot of deployments.

Zhong Yu

---

From: Roberto Peon <[email-masked]>
Date: Fri, 21 Feb 2014 13:03:12 -0800

We know from both experimentation and real; deployment experience that
having compressed resources is usually a large latency win.
Compressing resources certainly can be expensive.
Decompression (which is really the only thing that is required to implement
here) is usually fairly cheap.

It would be silly to require everything to be compressed. That is not what
is happening here.

-=R

---

From: Martin Thomson <[email-masked]>
Date: Fri, 21 Feb 2014 13:03:03 -0800

On 21 February 2014 12:46, Zhong Yu <[email-masked]> wrote:
> I also agree with Bjoern's point that

I was under the impression that it was compression speed that was the
concern.  Here we're talking about decompression only.  A server is
under no obligation to use compression; if the server wants more speed
and thinks that avoiding the compression schemes on offer is the best
way to achieve that, then go for it.

The question is what options we require clients to support so that
servers always have the option to select something better than
"identity".

---

From: Zhong Yu <[email-masked]>
Date: Fri, 21 Feb 2014 16:37:46 -0600

> servers always have the option to select something better than

Yes. And the argument is that if a server can do gzip, it can do
deflate as easily, so there's no point to give it both options.

(Compression of requests is probably out-of-scope here, but since
http/2 server implementation is a heavy task requiring abundant
resources to accomplish, mandating servers to accept compressed
requests may not be too much to ask? Or is it the last straw that
breaks the camel's back?)

Zhong Yu

---

From: Jeff Pinner <[email-masked]>
Date: Fri, 21 Feb 2014 14:44:28 -0800

As mentioned before -- deflate vs gzip support varies by language / library
-- most gzip libraries support deflate-only streams but not all deflate
libraries support gzip.

---

From: Jesse Wilson <[email-masked]>
Date: Tue, 25 Feb 2014 09:32:00 -0500

In Billy Hoffman's blog[1], and elsewhere, there's discussion over the fact
that 'deflate' is used to describe two different algorithms. And several
browsers and webservers got it wrong.

"So, DEFLATE, and Content-Encoding: deflate, *actually means the response
body is composed of the **zlib** format* (zlib header, deflated data, and a
checksum)."

[1]: http://zoompf.com/blog/2012/02/lose-the-wait-http-compression

If the spec is going to require that clients support deflate compression,
it should use strong language to remind implementors that 'deflate' means
zlib.

And if 'deflate' is implicit, then middleboxes that bridge HTTP/2 clients
to HTTP/1.1 servers will need to transcode/decompress deflated data for the
legacy browsers[2] that misinterpret the spec.

[2]: http://www.vervestudios.co/projects/compression-tests/results

Plus, middleboxes that bridge HTTP/1.1 servers to HTTP/2 clients may need
to transcode/decompress deflated data from broken servers that also get the
APEC wrong.

Otherwise all HTTP/2 clients will need to use heuristics to guess which
algorithm the peer is using. From Sam
<http://stackoverflow.com/users/17174/sam-saffron>Saffron<http://stackoverflow.com/users/17174/sam-saffron>on
Stack Overflow[3]:

"So, over the years browsers started implementing a *fuzzy logic* deflate
implementation, they try for zlib header and adler checksum, if that fails
they try for payload."

[3]:
http://stackoverflow.com/questions/388595/why-use-deflate-instead-of-gzip-for-text-files-served-by-apache

My preference is to avoid this rats nest and require gzip only.

---

From: Cory Benfield <[email-masked]>
Date: Tue, 25 Feb 2014 15:51:27 +0000

> My preference is to avoid this rats nest and require gzip only.

Agreed. I'd like to be able to remove my horrible deflate/zlib fuzzy
logic entirely if at all possible, especially as the gains are almost
irrelevant.

---

From: Roberto Peon <[email-masked]>
Date: Tue, 25 Feb 2014 09:02:05 -0800

There is zero guarantee that http2 would reliably remove any of that
logic-- the origin of any particular entity body may be at an HTTP/1 host,
which will use 'deflate'. :/

-=R

---

From: Cory Benfield <[email-masked]>
Date: Tue, 25 Feb 2014 17:09:53 +0000

On 25 February 2014 17:02, Roberto Peon <[email-masked]> wrote:
> There is zero guarantee that http2 would reliably remove any of that logic--
> the origin of any particular entity body may be at an HTTP/1 host, which
> will use 'deflate'. :/

That's not entirely true. The difference between HTTP/1.1 and HTTP/2.0
is that in HTTP/1.1 if I send `Accept-Encoding: identity', the server
must not return me a compressed response. In HTTP/2.0, however, the
server may always return a response compressed with gzip or deflate.
This means all HTTP/2.0 user-agents must allow decompression of gzip
and deflate encoded responses, because we cannot reliably inform
upstream that we don't want them.

Jesse and I want to step that back to say that servers may always
return a response compressed with gzip only: any other compression
must be explicitly requested. In this sitatuion, if I then send
`Accept-Encoding: gzip' I'll only ever get gzip back, even if the
origin of the entity body is a HTTP/1.1 host. This would allow me to
rip my deflate/zlib code out as I can ensure that it won't be sent to
me.

---

From: Roberto Peon <[email-masked]>
Date: Tue, 25 Feb 2014 09:18:05 -0800

True, and the data Jesse links to seems compelling...
-=R

On Tue, Feb 25, 2014 at 9:09 AM, Cory Benfield <[email-masked]> wrote:

> logic--

---

From: Patrick McManus <[email-masked]>
Date: Tue, 25 Feb 2014 12:32:04 -0500

fwiw I'm fine with just having http/2 imply a single encoding.

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 25 Feb 2014 18:33:29 +0100

>rip my deflate/zlib code out as I can ensure that it won't be sent to

The Accept-Encoding header indicates preferences and honouring them is a
SHOULD, not a MUST. It is not unusual to ignore it, e.g. typical Apache
configurations would send .gz files with `Content-Encoding: gzip` and
would not fail content negotiation and would not decompress the file
automatically when the client is not prepared to accept `gzip`, and some
deliberate configure it this way, e.g. to conserve resources.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Daniel Sommermann <[email-masked]>
Date: Tue, 25 Feb 2014 09:44:50 -0800

As another data point, this requirement has exposed some bugs in client 
HTTP stacks. There are some broken HTTP/1.1 implementations cannot 
process chunked responses. For reverse proxies that speak SPDY or 
HTTP/2.0 to the webserver and receive 1.1 requests without 
Accept-Encoding: gzip, the proxy must decompress the gzipped response 
from the server. Since it is difficult to predict the inflated size a 
priori, the response must be chunked, which breaks those clients. To not 
chunk the response, the proxy would have to buffer the entire body which 
is not great from a DOS or latency perspective.

In the end, I don't think the standard should change for those broken 
1.1 implementations, but it is good to be aware of this incompatibility.

---

From: Martin Thomson <[email-masked]>
Date: Tue, 25 Feb 2014 09:53:25 -0800

> MUST) 1. gzip or 2. gzip+deflate ?

This has been a fruitful discussion, but I'm still not getting an
answer to this question.  I think that I have counted a single "don't
care" from Roberto and a "gzip only" from PHK, Willy seems keen on
"deflate".

(I have taken note of the problem with deflate, and am prepared to add
a "MUST" regarding the zlib format, if we decide to include that
option.)

---

From: Jeff Pinner <[email-masked]>
Date: Tue, 25 Feb 2014 10:20:04 -0800

I'm +1 for an implicit "Accept-Encoding: gzip, deflate"

---

From: Michael Sweet <[email-masked]>
Date: Tue, 25 Feb 2014 13:27:25 -0500

Jesse,

That's a good (but unfortunate) point, and one that has also caused interoperability problems for IPP which normatively ties "deflate" to RFC 1951 (raw deflate) and not to RFC 1950 (zlib format) like HTTP/1.1 does... :/

+1 for just requiring gzip and perhaps documenting why deflate isn't being required...

On Feb 25, 2014, at 9:32 AM, Jesse Wilson <[email-masked]> wrote:

> In Billy Hoffman's blog[1], and elsewhere, there's discussion over the fact that 'deflate' is used to describe two different algorithms. And several browsers and webservers got it wrong.
> 
> "So, DEFLATE, and Content-Encoding: deflate, actually means the response body is composed of the zlib format (zlib header, deflated data, and a checksum)."
> 
> If the spec is going to require that clients support deflate compression, it should use strong language to remind implementors that 'deflate' means zlib.
> 
> And if 'deflate' is implicit, then middleboxes that bridge HTTP/2 clients  to HTTP/1.1 servers will need to transcode/decompress deflated data for the legacy browsers[2] that misinterpret the spec.
> 
> Plus, middleboxes that bridge HTTP/1.1 servers to HTTP/2 clients may need to transcode/decompress deflated data from broken servers that also get the APEC wrong.
> 
> Otherwise all HTTP/2 clients will need to use heuristics to guess which algorithm the peer is using. From Sam Saffron on Stack Overflow[3]:
> 
> "So, over the years browsers started implementing a fuzzy logic deflate implementation, they try for zlib header and adler checksum, if that fails they try for payload."
> 
> [3]: http://stackoverflow.com/questions/388595/why-use-deflate-instead-of-gzip-for-text-files-served-by-apache
>

---

From: Martin Thomson <[email-masked]>
Date: Tue, 25 Feb 2014 10:31:20 -0800

On 25 February 2014 10:27, Michael Sweet <[email-masked]> wrote:
> That's a good (but unfortunate) point, and one that has also caused
> interoperability problems for IPP which normatively ties "deflate" to RFC
> 1951 (raw deflate) and not to RFC 1950 (zlib format) like HTTP/1.1 does...
> :/

That's an interesting point.  Are you suggesting that IPP is
intentionally divergent from HTTP/1.1?  If so, can you explain why
that choice was made?

---

From: Michael Sweet <[email-masked]>
Date: Tue, 25 Feb 2014 14:07:29 -0500

Martin,

> intentionally divergent from HTTP/1.1?  If so, can you explain why

I'd have to check the old mailing list logs from 1998, but I think it wasn't intentional but a mistake based on the compression keyword values (none, compress, deflate, gzip) - the same sort of mistake that's been made by HTTP implementors due to the unfortunately confusion between zlib the format and deflate the encoding name being used when zlib is meant.

---

From: Martin Thomson <[email-masked]>
Date: Tue, 25 Feb 2014 11:28:19 -0800

> I'd have to check the old mailing list logs from 1998, but I think it wasn't intentional but a mistake based on the compression keyword values (none, compress, deflate, gzip) - the same sort of mis…

Do you think that this is something that we can realistically fix?
People have put up with bad "deflate" in HTTP/1.1, but do you think
that there is any chance that we can address that problem in HTTP/2 by
saying "deflate MUST be RFC1950, reject otherwise"?

---

From: Michael Sweet <[email-masked]>
Date: Tue, 25 Feb 2014 14:28:11 -0500

Martin,

> I'd have to check the old mailing list logs from 1998, but I think it wasn't intentional but a mistake based on the compression keyword values (none, compress, deflate, gzip) - the same sort of mis…

OK, found out the source of this reference; it appears to have come from IPP/1.1 LC comments made by Ned Freed:

    http://www.pwg.org/archives/ipp/2000/009237.html

In it I found the following comment:

	Section 4.4.32. GZIP references RFC 1952 which is fine, but why doesn't
	deflate reference RFC 1951?

	TH> Fixed.

FWIW, the previous IPP/1.0 RFC (2566) did not have references for 'deflate' or 'compress'.

So it looks like this is a 14-year-old editorial choice/mistake that we are stuck with in IPP...

---

From: Michael Sweet <[email-masked]>
Date: Tue, 25 Feb 2014 14:56:49 -0500

> saying "deflate MUST be RFC1950, reject otherwise"?

Well, at the HTTP level you can certainly make that assertion/requirement.  My personal preference is to just require gzip support and document the known interoperability issues of deflate.

I don't think the handful of extra bytes matter - you are still making the message body smaller.

I also don't think the slightly slower speed of CRC32 vs. Adler32 matters - most of the time is spent actually compressing the data, not computing a sum of the compressed bytes.

I *do* think that interoperability will continue to be a bigger concern than raw performance.

....

General comment: there seems to be a lot of focus on getting the best compression in HTTP/2.0.  While this may be a worthwhile goal, I am concerned that it will both delay completion of HTTP/2.0 and result in a hard-to-implement protocol with serious interoperability issues.  IMHO the focus should be on addressing HTTP/1.1's major shortcomings, requiring or recommending existing features/technologies that already work well for HTTP/1.1, and limiting the number of variations that an implementation needs to work with, e.g., 1 compression algorithm vs. 2.

Today, gzip is probably the most widely implemented Content-Encoding for compressing data and has the fewest interoperability issues.  Requiring it in HTTP/2.0 definitely makes sense and will be a compelling improvement over HTTP/1.1.  Deflate/zlib have a known history of interoperability problems and provides almost identical performance characteristics to gzip, so it does not make sense to promote "deflate" to required for HTTP/2.0.

---

From: Jesse Wilson <[email-masked]>
Date: Tue, 25 Feb 2014 15:15:50 -0500

At a risk of jumping the gun, here's a PR that drops deflate from the
content-encodings that clients must support:
https://github.com/http2/http2-spec/pull/415

---

From: Martin Thomson <[email-masked]>
Date: Tue, 25 Feb 2014 12:17:55 -0800

> Well, at the HTTP level you can certainly make that assertion/requirement.  My personal preference is to just require gzip support and document the known interoperability issues of deflate.

This is a reasonable point.  When you consider the possibility of
adding a better, faster, stronger compression scheme under a new name,
that makes the marginal advantages of deflate over gzip less
attractive.

Does anyone absolutely have to have deflate?  Or will I take Jesse's PR?

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 26 Feb 2014 07:37:34 +1100

We clarified this in HTTPbis - see <http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-26#section-4.2.2>.

The downside of only doing gzip is that if a server *does* use deflate, a HTTP1->2 intermediary will have to re-encode responses. Not saying that's a showstopper, just noting it.

---

From: Martin Thomson <[email-masked]>
Date: Tue, 25 Feb 2014 12:54:21 -0800

> The downside of only doing gzip is that if a server *does* use deflate, a HTTP1->2 intermediary will have to re-encode responses. Not saying that's a showstopper, just noting it.

Well, the server should only do that if the request permits it then.
Thus, the intermediary would have to opt in to deflate to get deflate.

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 26 Feb 2014 07:57:35 +1100

> Thus, the intermediary would have to opt in to deflate to get deflate.

Of course; never mind. 

It sounds like we're leaning gzip-only, but that people don't care passionately about it. Let's chat about it briefly next week.

Cheers,

---

From: Martin Thomson <[email-masked]>
Date: Tue, 25 Feb 2014 12:59:44 -0800

> It sounds like we're leaning gzip-only, but that people don't care passionately about it. Let's chat about it briefly next week.

Sounds like a plan.  Either way it's easy to fix.

---

From: Zhong Yu <[email-masked]>
Date: Tue, 25 Feb 2014 15:02:36 -0600

On Tue, Feb 25, 2014 at 8:32 AM, Jesse Wilson <[email-masked]> wrote:
> "So, DEFLATE, and Content-Encoding: deflate, actually means the response
> body is composed of the zlib format (zlib header, deflated data, and a
> checksum)."
>
> If the spec is going to require that clients support deflate compression, it
> should use strong language to remind implementors that 'deflate' means zlib.

Or create a new name to remove the ambiguity of the old name.

("de-flate" always sounds weird to me:)  it sounds like it is undoing
an operation, while it is actually operating on the original data)

> Plus, middleboxes that bridge HTTP/1.1 servers to HTTP/2 clients may need to
> transcode/decompress deflated data from broken servers that also get the
> APEC wrong.
>
> algorithm the peer is using. From Sam Saffron on Stack Overflow[3]:
>
> "So, over the years browsers started implementing a fuzzy logic deflate

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 26 Feb 2014 10:08:57 +1300

On 2014-02-26 08:28, Martin Thomson wrote:
> On 25 February 2014 11:07, Michael Sweet wrote:
>> I'd have to check the old mailing list logs from 1998, but I think it 
>> wasn't intentional but a mistake based on the compression keyword 
>> values (none, compress, deflate, gzip) - the same sort of mistake 
>> that's been made by HTTP implementors due to the unfortunately 
>> confusion between zlib the format and deflate the encoding name being 
>> used when zlib is meant.
> 

I would think the obvious would be better:

  * Make 2.0 change the name to add "zlib" explicitly alongside 
gzip+deflate
  * Add a note that deflate has confusion *in 1.1* and instructing 
1.1<->2.0 gateways to enforce correct deflate/zlib labeling or 
re-encoding.

Saves a few bytes if deflate/zlib is used at all and 1.1<->2.0 gateways 
can translate either just the label, or the encoding as necessary.

Amos

---

From: Martin Thomson <[email-masked]>
Date: Tue, 25 Feb 2014 13:21:40 -0800

On 25 February 2014 13:08, Amos Jeffries <[email-masked]> wrote:
>  * Make 2.0 change the name to add "zlib" explicitly alongside gzip+deflate
>  * Add a note that deflate has confusion *in 1.1* and instructing 1.1<->2.0
> gateways to enforce correct deflate/zlib labeling or re-encoding.

I think perhaps if we do want to *mandate* gzip+deflate that this
would be the best way to do it.  It would be trivially possible to
translate "zlib" into "deflate" at a 2.0->1.1 intermediary.

The question is whether the advantages of deflate over gzip warrant
it.  I have no expertise, but it doesn't sound like they are
substantial enough to warrant the extra complexity.

---

From: Roberto Peon <[email-masked]>
Date: Tue, 25 Feb 2014 13:44:30 -0800

imho, just doing gzip is the most interoperable thing to do given Jesse's
interesting data.
'transcoding' from gzip to other deflate-containing formats shouldn't be
that expensive, assuming it was needed.
-=R

On Tue, Feb 25, 2014 at 1:21 PM, Martin Thomson <[email-masked]>wrote:

> 1.1<->2.0

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 26 Feb 2014 15:28:53 +1300

On 2014-02-26 10:21, Martin Thomson wrote:
> On 25 February 2014 13:08, Amos Jeffries wrote:

I am thinking we need to have a note about fixing the deflate 
interoperability at the gateway regardless of whether deflate (or zlib) 
is listed as a default. Lets not permit the issue to continue with 
HTTP/2 senders.

Amos