Subject: I ran across this while working on the spec.

From: Roberto Peon <[email-masked]>
Date: Wed, 16 Oct 2013 17:37:19 -0700

https://github.com/http2/http2-spec/issues/281

Setting the max header size to 0 does not clear out all encoder state.

In particular, elements to the reference set that are pointing to static
table elements are not cleared out.

There are a couple of obvious ways of fixing this:
1) Include the space used in the reference set in the overhead (e.g.
2-bytes per reference)
2) explicit SETTING for clearing the compression state

I prefer #1.

---

From: Fred Akalin <[email-masked]>
Date: Wed, 16 Oct 2013 19:09:55 -0700

(copying from issue)

#1 <https://github.com/http2/http2-spec/issues/1> has the problem that it
introduces an ambiguity in what to do when reducing the max size (clear out
the reference set first?). But if we specify exactly what to do when
reducing the max size, then that isn't that bad.

---

From: RUELLAN Herve <[email-masked]>
Date: Thu, 17 Oct 2013 14:34:38 +0000

I'm wondering if we want to be able to clear out all compression state. We currently have 59 entries in the static table, meaning 8 bytes to store whether they are in the reference set or not.

I we want this, I think #1 is too complex.

It means that sending an index (therefore adding a new entry to the reference set) may trigger an eviction.

We have several other solutions for this:
- #2
- 3 Adding the possibility of clearing the reference set inside a header block.
- 4 Having a special rule for a max header table of 0, stating that this clears all the compression state.

My preference is for 2 or 3.

Hervé.

> -----Original Message-----
> From: Fred Akalin [mailto:[email-masked]]
> Sent: jeudi 17 octobre 2013 04:10
> To: Roberto Peon
> Cc: [email-masked] Group
> Subject: Re: I ran across this while working on the spec.
> 
> (copying from issue)
> 
> the reference set first?). But if we specify exactly what to do when reducing
> the max size, then that isn't that bad.
> 
> 
>

---

From: Roberto Peon <[email-masked]>
Date: Thu, 17 Oct 2013 10:44:17 -0700

We most definitely want to be able to clear out all compression state.

An example use-case is that we'll have long-lived connections from just
about any client that wants server-side notifications. Leaving lots of
memory lying about unused (~99% to ~99.9% of the time things are idle) has
strongly negative utility. I'd rather use that memory to have larger
compression contexts for connections which are active, or to allow more
connections, or, in the case of mobile devices, to yield more memory to
actively running processes.

So, it would be a very good thing to be able to completely de-allocate the
compression context until communication is going on.
Terminating the connection is not a desirable mechanism for accomplishing
this, as it requires both significant latency and significant CPU to
reconnect. Reconnecting implies a worse experience for all users, and for
mobile devices also implies decreased battery life. Ick/should be avoided.

#1 strictly bounds the amount of memory used. I do like that a lot.
  I'd imagine that #1 would be specified approximately as:
     Evict entries normally as per eviction rules.
     if size(header table) is zero, size(header context) >
MAX_HEADER_CONTEXT_SIZE, evict the highest indexed reference from the
reference set.
  if MAX_HEADER_CONTEXT_SIZE is zero, then the receiver side need never
allocate a header context/may completely deallocate it.

  MAX_HEADER_CONTEXT_SIZE would replace MAX_HEADER_TABLE_SIZE. We'd still
use only one setting (just rename it).

#2 is fairly clean, and would be a close-second choice. This perhaps
minimizes complexity. I could be made to like this one :)
    I'd assume that the mechanism to accomplish #2 would be a setting which
toggles whether or not the reference set is allowed to have any elements.
    Thus, if HEADER_TABLE_SIZE is 0 and REFERENCE_SET_DISABLED is true, the
receiver is guaranteed to require no compression context.
     The main thing I dislike about this option is that it requires another
setting. The main advantage is that this is essentially all it requires.

#3 doesn't solve the problem better than #2, as we'd still need something
like #2 for the decoder-side to signal to the encoder-side that it would
like this state to be cleared.

#4 is less clean than #2, but I prefer it to #3, as it is also a complete
solution.
    We would also need to specify that this implies that one cannot add
elements to the reference set when the size is zero (essentially, the same
mechanism used for adding entries to the header table which don't fit).

There is another option as well:

#5 When referencing an element from the static set, instead of inserting a
reference to the static-table entry, instead copy the element into the
header table and add a reference to it there.
     References stored in the reference set must point only to elements of
the header table with this option.

   This solves the problem without requiring any additional signaling, but
at the cost of worse compression, and increased CPU consumption (copying).

-=R

On Thu, Oct 17, 2013 at 7:34 AM, RUELLAN Herve
<[email-masked]>wrote:

> I'm wondering if we want to be able to clear out all compression state. We
> currently have 59 entries in the static table, meaning 8 bytes to store
> whether they are in the reference set or not.
>

---

From: Jeff Pinner <[email-masked]>
Date: Thu, 17 Oct 2013 11:15:35 -0700

Another option:

Do not allow indexed representations to use the static set. If you can only
use the set for literal representations (with or without indexing) then you
could never add anything from the static set to the reference set. No need
to track emitted status. Problem sovled.

This would mean a slight overhead when sending the initial ":scheme" or
":method" headers since you would have to encode the values.

It would mean that we could remove values (and thus duplicates) from the
static set since they could never be referenced.

On Thu, Oct 17, 2013 at 10:44 AM, Roberto Peon <[email-masked]> wrote:

>     I'd assume that the mechanism to accomplish #2 would be a setting
> which toggles whether or not the reference set is allowed to have any
> elements.
>     Thus, if HEADER_TABLE_SIZE is 0 and REFERENCE_SET_DISABLED is true,
> the receiver is guaranteed to require no compression context.
>      The main thing I dislike about this option is that it requires
> another setting. The main advantage is that this is essentially all it
> requires.
>
> On Thu, Oct 17, 2013 at 7:34 AM, RUELLAN Herve <[email-masked]
> > wrote:
>
>> I'm wondering if we want to be able to clear out all compression state.
>> We currently have 59 entries in the static table, meaning 8 bytes to store

---

From: Roberto Peon <[email-masked]>
Date: Thu, 17 Oct 2013 11:17:13 -0700

Lets call this #6.
-=R

On Thu, Oct 17, 2013 at 11:15 AM, Jeff Pinner <[email-masked]> wrote:

> Another option:
>
> Do not allow indexed representations to use the static set. If you can
> only use the set for literal representations (with or without indexing)
> then you could never add anything from the static set to the reference set.
> No need to track emitted status. Problem sovled.
>
>> So, it would be a very good thing to be able to completely de-allocate
>> the compression context until communication is going on.
>> #5 When referencing an element from the static set, instead of inserting
>> a reference to the static-table entry, instead copy the element into the
>>      References stored in the reference set must point only to elements
>> of the header table with this option.
>>
>>    This solves the problem without requiring any additional signaling,
>> but at the cost of worse compression, and increased CPU consumption
>> (copying).
>>
>> -=R
>>
>>
>> On Thu, Oct 17, 2013 at 7:34 AM, RUELLAN Herve <
>> [email-masked]> wrote:
>>

---

From: Martin Thomson <[email-masked]>
Date: Thu, 17 Oct 2013 11:43:16 -0700

> Lets call this #6.

For #6, I think that you want to allow indexed representations to use
the static set, otherwise you don't get to take advantage of the
values there, just the keys.  You might say, however, that indexing
these doesn't add them to the reference set.

I really don't have a particular preference here, but I do note that
there is a strict upper bound on the size of the reference set after
evicting all dynamic entries.  That is, it's the size of your
reference, times the size of the static table.  Have you considered
the "don't worry"?

---

From: Roberto Peon <[email-masked]>
Date: Thu, 17 Oct 2013 11:47:25 -0700

I have (that was what Herve proposed :) ), but I do worry: A set typically
requires two pointers per element, plus the size of the element itself plus
pointer to the index itself.
If we call that 8+8+4 bytes of overhead per entry, we're talking about a
kilobyte of potential overhead. That is noticable.

-=R

---

From: Mike Bishop <[email-masked]>
Date: Thu, 17 Oct 2013 18:49:40 +0000

If you maintain the reference set into the static table as a bitmask, we're talking about eight bytes per connection.

From: Roberto Peon [mailto:[email-masked]]
Sent: Thursday, October 17, 2013 11:47 AM
To: Martin Thomson
Cc: Jeff Pinner; RUELLAN Herve; Fred Akalin; Roberto Peon; [email-masked] Group
Subject: Re: I ran across this while working on the spec.

I have (that was what Herve proposed :) ), but I do worry: A set typically
requires two pointers per element, plus the size of the element itself plus
pointer to the index itself.
If we call that 8+8+4 bytes of overhead per entry, we're talking about a kilobyte of potential overhead. That is noticable.

-=R

On Thu, Oct 17, 2013 at 11:43 AM, Martin Thomson <[email-masked]<mailto:[email-masked]>> wrote:

---

From: Roberto Peon <[email-masked]>
Date: Thu, 17 Oct 2013 11:53:42 -0700

That is true. That implementation would require substantial code additions,
though, moreso than the other options.
-=R

On Thu, Oct 17, 2013 at 11:49 AM, Mike Bishop
<[email-masked]>wrote:

>  If you maintain the reference set into the static table as a bitmask,
> we’re talking about eight bytes per connection.****
>
> ** **
>
> *From:* Roberto Peon [mailto:[email-masked]]
> *Sent:* Thursday, October 17, 2013 11:47 AM
> *To:* Martin Thomson
> *Cc:* Jeff Pinner; RUELLAN Herve; Fred Akalin; Roberto Peon;
> [email-masked] Group
>
> *Subject:* Re: I ran across this while working on the spec.****
>
> ** **
>
> ****
> ****
> pointer to the index itself.****
>
> kilobyte of potential overhead. That is noticable.****
>
> ** **
>
> -=R****
>
> ** **
>
> On Thu, Oct 17, 2013 at 11:43 AM, Martin Thomson <[email-masked]>
> wrote:****
>
> the "don't worry"?****
>  ** **

---

From: Jeff Pinner <[email-masked]>
Date: Thu, 17 Oct 2013 12:39:11 -0700

> these doesn't add them to the reference set.
Yep, that's why you'd lose some compression efficiency for the first time
you sent "http", "GET", "/" or "POST". After which these would be indexible
at the front of the table.