Subject: Partial header indexing in HPACK?

From: Brendan Long <[email-masked]>
Date: Fri, 29 May 2015 21:01:19 +0000

Hi,

I’m comparing header overhead for HTTP/2 and HTTP/1.1, and I’m wondering if there’s a way to reduce the size of a header which is almost exactly the same as another one.

Specifically, I’m looking at YouTube video requests that look like this:

GET /videoplayback?id=o-AMeWenDYa309rl6k2LL_cFV6OvIcLpBd569F3mclE88G&expire=1432943184&mm=31&ms=au&pl=18&mv=m&ipbits=0&ip=184.96.121.63&mt=1432921497&dur=29.988&lmt=1432229686863006&source=youtube&clen=1211379&key=yt5&mime=video%2Fwebm&gir=yes&requiressl=yes&initcwndbps=901250&sver=3&fexp=3300009%2C3300101%2C3300132%2C3300137%2C3300161%2C3310697%2C3312210%2C916634%2C9407746%2C9408142%2C9408215%2C9408710%2C9408737%2C9408788%2C9412774%2C9413058%2C9413503%2C9415011%2C9415304%2C952612&sparams=clen%2Cctier%2Cdur%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cms%2Cmv%2Cpl%2Crequiressl%2Csource%2Cupn%2Cexpire&signature=D14AFCF4AAEBDCB5720575DBA485B91959A042B9.32441FF6369F2CB53409722A132C29E10DBAAE09&itag=243&ctier=L&upn=HW6VvdEQlSk&cpn=bJM7QcFNJQ-4Kf4e&alr=yes&keepalive=yes&ratebypass=yes&c=WEB&cver=html5&range=0-341 HTTP/1.1
Host: r5---sn-qxo7sn7z.googlevideo.com
Connection: keep-alive
Origin: https://www.youtube.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.81 Safari/537.36
Accept: */*
X-Client-Data: CKm1yQEIhbbJAQiktskBCKm2yQEIwbbJAQjpiMoBCNKUygE=
Referer: https://www.youtube.com/watch?v=dQw4w9WgXcQ
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8

When you convert this to HTTP/2 and make multiple requests, all of the headers compress extremely well except :path, since the “id” query parameter changes every time, so we get a literal 800 byte path instead of an indexed path + 20 changes bytes.

I’m wondering if there’s any way to partially index a header, or if this is something the IETF would consider looking at. In the simplest case, it would be nice to have a third header format which is:
Index in table
Length of string to use from table
Literal to append

In my example above, we could sort the path to put “id” at the end, then put the first few hundred bytes from the index and append the new “id” at the end. From a client perspective, it shouldn’t be too difficult to check if any of your strings have a prefix in the index.

For use cases, I expect this would come up all the time with :path. Another example is “Accept-Encoding”, where nearly everyone will use the version in the static table (“gzip, deflate”), but some browsers would want to append to that (including Chrome apparently, which uses “sdch”).

Was this considered? Is is possible and I’m just missing it? Is there any reason we wouldn’t want to do this?

Thanks,
Brendan Long

---

From: Martin Thomson <[email-masked]>
Date: Fri, 29 May 2015 14:20:24 -0700

On 29 May 2015 at 14:01, Brendan Long <[email-masked]> wrote:
> I’m wondering if there’s any way to partially index a header, or if this is
> something the IETF would consider looking at.

This was considered as part of the HTTP/2 (HPACK) design process and
the conclusion was that there were some non-trivial security concerns
resulting from any scheme that could compress parts of header fields.
That combined with a strong desire to minimize the complexity of HPACK
meant that this didn't make the cut.

Without thinking about the security problem too hard, something like
what you describe might be made to work, maybe.  But at this point we
have no way of changing the protocol.