Subject: HPACK, Dynamic Table Size Update

From: Pavel Rappo <[email-masked]>
Date: Tue, 9 Jun 2015 14:14:54 +0100

Hi,

1. Can a "Dynamic Table Size Update" be unsolicited or its only purpose is to
acknowledge the fact that encoder has understood the request sent by decoder and
agreed to use a new maximum upper bound for the dynamic table size?

2. From "4.2.  Maximum Table Size":

   ...Multiple updates to the maximum table size can occur between the
   transmission of two header blocks.  In the case that this size is
   changed more than once in this interval, the smallest maximum table
   size that occurs in that interval MUST be signaled in a dynamic table
   size update.  The final maximum size is always signaled, resulting in
   at most two dynamic table size updates.  This ensures that the
   decoder is able to perform eviction based on reductions in dynamic
   table size (see Section 4.3)...

Am I right saying that _semantically_ it would be equivalent to this schematic
behaviour (depicted by pseudo-code):

    Encoder.setMaxSize( size ) {

      if size < minSizeSoFar
        minSizeSoFar = size

      lastSize = size;

    }

    Encoder.encodeAndSendAsHeaderBlock( headers ) {

      sendMaximumSizeUpdate( minSizeSoFar );

      if minSizeSoFar != lastSize
        sendMaximumSizeUpdate( lastSize );

      for each (Header h in headers) {
          ...
      }
      ...
    }

Or this paragraph means something completely different?

Thanks.

-Pavel

---

From: Martin Thomson <[email-masked]>
Date: Tue, 9 Jun 2015 10:28:13 -0700

> agreed to use a new maximum upper bound for the dynamic table size?

This is the encoder's choice.  The value can be set to whatever value
they like, whenever they like, as long as it is <= the maximum set by
the decoder.
> behaviour (depicted by pseudo-code):

Yes, you have it (almost entirely) correct.

>     Encoder.encodeAndSendAsHeaderBlock( headers ) {

+        if minSizeSoFar < currentSize   {

>       sendMaximumSizeUpdate( minSizeSoFar );

       }
         minSizeSoFar = Infinity

---

From: Mike Bishop <[email-masked]>
Date: Tue, 9 Jun 2015 17:27:52 +0000

It can be unsolicited.  The decoder sends (via SETTINGS) the largest context it will allow the encoder to use -- the encoder, however, could choose to go even smaller than it's allowed for whatever reasons seem good to it.  (Think low-memory embedded devices, for example.)  The encoder can also be fancy and vary the size it's using based on current memory pressure.  4.2 is trying to reduce the chattiness of an encoder that's doing dynamic variation like this.

Your pseudocode would work, but you're sending at least one, possibly two, size updates on every encoded header block.  Most of the time, it won't have changed, so you don't need to send anything.  So more like:

      if size < minSizeSinceUpdate
        minSizeSinceUpdate = size

      currentSize = size;

    }

      if minSizeSinceUpdate < lastSent  {
        sendMaximumSizeUpdate( minSizeSoFar );
        lastSent = minSizeSinceUpdate;
      }

      if lastSent != currentSize {
        sendMaximumSizeUpdate( lastSize );
        lastSent = currentSize;
        minSizeSinceUpdate = currentSize;
      }

The key is to let the remote side know the smallest the table got (so it can prune appropriately) and what size it is now, if it has grown again.  The point of 4.2 is that if you're dynamically varying size, you don't have to tell the remote about *every* size change, you can just condense them into at most two.  But most of the time, zero or one will be sufficient.

-----Original Message-----
From: Pavel Rappo [mailto:[email-masked]] 
Sent: Tuesday, June 9, 2015 6:15 AM
To: [email-masked]
Subject: HPACK, Dynamic Table Size Update

Hi,

1. Can a "Dynamic Table Size Update" be unsolicited or its only purpose is to acknowledge the fact that encoder has understood the request sent by decoder and agreed to use a new maximum upper bound for the dynamic table size?

Am I right saying that _semantically_ it would be equivalent to this schematic behaviour (depicted by pseudo-code):

      if size < minSizeSoFar
        minSizeSoFar = size

      lastSize = size;

    }

Thanks.

-Pavel

---

From: Pavel Rappo <[email-masked]>
Date: Tue, 9 Jun 2015 21:57:13 +0100

Michael, Martin, thanks for your answers! I have two more questions if I may.

1. ...This dynamic table size update MUST occur at the beginning of the first
   header block following the change to the dynamic table size. In HTTP/2,
   this follows a settings acknowledgment (see Section 6.5.3 of [HTTP2]).

To me it seems like it _only_ applies to solicited updates. Mostly because this
paragraph mentions "settings acknowledgment" which is (as I understand) absent
when size update is initiated by encoding side. So the question is -- could an
unsolicited update go in the middle of a header block, for example:

    -- header block begin --
    == Literal indexed ==
    == Literal indexed ==
    == Indexed - Add ==
    == Size Update ==
    == Literal indexed ==
    == Literal indexed ==
    ...
    -- header block end --

?

Just curious, what was the rationale behind binding size updates to a specific
location in a header block (for the solicited case)?

2. As I understand a header block cannot consist of only size updates because
"size update" is not a header field representation. So size updates (if any) has
to go with some header fields. Am I right?

-- header block begin --
== Size Update ==
== Size Update ==
== Size Update ==
-- header block end --

Thanks.

-Pavel

---

From: Martin Thomson <[email-masked]>
Date: Tue, 9 Jun 2015 15:12:58 -0700

On 9 June 2015 at 13:57, Pavel Rappo <[email-masked]> wrote:
> To me it seems like it _only_ applies to solicited updates.

That is my understanding too (though it might not be well-tested; you
can check with the test suites here:
https://github.com/http2jp/hpack-test-case)

> location in a header block (for the solicited case)?

The rule here is that the change had to be immediate.  That is, once
you acknowledge the setting at the HTTP/2 framing layer, you can't
delay enacting the change within HPACK.

> to go with some header fields. Am I right?

Well, technically a header block can be empty, but only for trailers.
All the other blocks have mandatory header fields.  It's an odd case,
and (again) might not be well-tested, so I wouldn't do it.  But it
isn't technically disallowed.

---

From: Mike Bishop <[email-masked]>
Date: Tue, 9 Jun 2015 23:53:57 +0000

For #1, the required position is because the encoder MUST respond to the newly-received setting as soon as it's received and acknowledged.  Functionally, it *could* occur sooner -- there's no reason it couldn't immediately signal a lower size upon receive during compression of a header block, then acknowledge the SETTINGS frame after the currently-in-process frame was done, but that's likely to be needlessly complex.  But the latest you can drop to within the required bounds is the start of the next HEADERS frame, because it's the next time the encoder gets to speak after the ack.

So in short, it could legitimately go earlier if you were in the process of sending a HEADERS frame already, but it cannot go later.

For #2, you're correct -- an otherwise-empty HPACK frame will no longer result in a valid header-set.  (This was possible with previous drafts.)

-----Original Message-----
From: Pavel Rappo [mailto:[email-masked]] 
Sent: Tuesday, June 9, 2015 1:57 PM
To: Martin Thomson; Mike Bishop
Cc: [email-masked]
Subject: Re: HPACK, Dynamic Table Size Update

To me it seems like it _only_ applies to solicited updates. Mostly because this paragraph mentions "settings acknowledgment" which is (as I understand) absent when size update is initiated by encoding side. So the question is -- could an unsolicited update go in the middle of a header block, for example:

    -- header block begin --
    == Literal indexed ==
    == Literal indexed ==
    == Indexed - Add ==
    == Size Update ==
    == Literal indexed ==
    == Literal indexed ==
    ...
    -- header block end --

?

Just curious, what was the rationale behind binding size updates to a specific location in a header block (for the solicited case)?

2. As I understand a header block cannot consist of only size updates because "size update" is not a header field representation. So size updates (if any) has to go with some header fields. Am I right?

-- header block begin --
== Size Update ==
== Size Update ==
== Size Update ==
-- header block end --

Thanks.

-Pavel