Subject: HTTP router point-of-view concerns

From: Christian Parpart <[email-masked]>
Date: Thu, 11 Jul 2013 00:52:55 +0200

Hey guys,

I am just new to this list and I've just recently started working through
the HTTP/2 draft and all the blog articles found about.
That being said, I myself have also very concerns others have noticed as
well, and would like to deeply show my intention to help standing aside :)

I am the implementor of one of the many HTTP servers that's being used in
production, and one major feature is HTTP routing / load balancing,
and I would really love to implement an HTTP/1.1 successor that is (to be)
officially labeled HTTP/2[.0].

However, as a varnish [1] and a BSD [2] guy also raised there hands on, is
the lag of easy extraction of envelop information of an HTTP request
message, such as method, path, but most certainly the host.

Please forgive me if this is already on-topic somewhere hidden, however, I
would really highly encourage you to consider
adding some dedicated frame type for this kind of envelope information.

With that in mind, one might say that an HTTP/2 stream is not initiated by
a HEADERS frame but the ENVELOPE frame that contains the actual
uncompressed and unmystified but compact information about this request
message.

One humble proposal might indeed be:

type: (something unassigned)
flags: bit 1 = END_STREAM /* this HTTP message is complete with just these
envelope frame, e.g. a simple GET, and no need for user-agent etc. */
id: unique stream ID, semantics like any other stream ID
body: a key/value table of the envelope data

ENVELOPE FRAME DATA:

The envelope data table is a simple table of key/value pairs where the key
is an 8bit value identifying the entry
and a variable length value that is interpreted depending on the key. The
list of provided envelope fields ends
as the end of the envelope frame has been reached. that means, an envelope
must always fit into a single (first) frame.

ENVELOPE KEY/VALUE FIELDS:

   - :scheme => uint8: 0x01
      - http => uint8: 0x01
      - https => uint8: 0x02
      - custom => same as in method (if this is distinction is really
      demanded)
   - :method => uint8: 0x02
      - GET => uint8: 0x01
      - POST => uint8: 0x02
      - PUT => uint8: 0x03
      - DELETE => uint8:0x04
      - custom => uint8: 0xFF, followed by one uint8 encoding the size of
      the following bytes declaring the plaintext method value, e.g. "PROPFIND"
   - :path => uint8: 0x03, uint16 length in network byte order, followed by
   $length octects declaring the path's value.
   - :host => uint8: 0x04, uint16: length in network byte order, $length
   octets declaring the host's value.
   - :route => uint8: 0x05, uint8: length, $length octets that declare this
   value. (field is only specified if known, thus previousely announced by the
   remote server or this frame is part of a response and we are to announce a
   routing identifier)
   - :status => uint8: 0x06, uint16: code in network byte order  /* if
   HTTP/2 considers starting response streams the same way */

This is the exact information an HTTP client (scheme,method,path,host) or
server (status) MUST currently sent as part of the (first) HEADERS frame.
So the change I propose is, to extract this information from the HEADERS
frame and put it into its own frame that also initiates the stream
implicitly.

Having this in mind, it is a pleasure to implement HTTP routers because
those now don't have to decode the full HEADERS frames but just decode the
ENVELOPE frame and pass any continued frame to the directed next-hop server.

"ROUTE" FIELD:

You might have noticed that I quietly introduced routing key. This idea is
on behalf of what's blamed in  [1] and [2] about the missing session
feature, so you might easily call it "session" instead of "route", but I
called it "route" because in the use-case I am speaking for (HTTP router)
this value would have been used to create backend stickiness for client
sessions.

This whole subject is a problem of its own, and I didn't intend to talk
about this, but at least wanted to mention, that it should be part of  the
ENVELOPE header as well, as the HTTP router must inspect this field as fast
as possible, too.

p.s.: in [2] is also noted, that the message payload (DATA) length should
always be known in advance, a highly disagree on that, as there are types
of messages that actually do not always know the length in advance (.e.g.
live streaming servers, if it's video streaming, audio, or text based) or
other kind of feed messages that are layered atop of HTTP, so I'd propose
against that. :0

Now, I hope you won't kill me for my proposals, but I would like to show my
strong intend for the upcoming HTTP/2 to have as less pains as possible,
otherwise upgrading server (and client) software wouldn't make much sense
if the margin of benefit is just too small.

Best regards,
Christian Parpart

[1] https://www.varnish-cache.org/docs/trunk/phk/http20.html
[2] http://phk.freebsd.dk/words/httpbis.html

---

From: Amos Jeffries <[email-masked]>
Date: Thu, 11 Jul 2013 14:53:38 +1200

On 11/07/2013 10:52 a.m., Christian Parpart wrote:
> Hey guys,
>
> I am just new to this list and I've just recently started working 
> through the HTTP/2 draft and all the blog articles found about.
> That being said, I myself have also very concerns others have noticed 
> as well, and would like to deeply show my intention to help standing 
> aside :)
>
> I am the implementor of one of the many HTTP servers that's being used 
> in production, and one major feature is HTTP routing / load balancing,
> and I would really love to implement an HTTP/1.1 successor that is (to 
> be) officially labeled HTTP/2[.0].
>
> However, as a varnish [1] and a BSD [2] guy also raised there hands 
> on, is the lag of easy extraction of envelop information of an HTTP 
> request message, such as method, path, but most certainly the host.
>
> Please forgive me if this is already on-topic somewhere hidden, 
> however, I would really highly encourage you to consider
> With that in mind, one might say that an HTTP/2 stream is not 
> initiated by a HEADERS frame but the ENVELOPE frame that contains the 
> actual
> uncompressed and unmystified but compact information about this 
> request message.
>
> flags: bit 1 = END_STREAM /* this HTTP message is complete with just 
> these envelope frame, e.g. a simple GET, and no need for user-agent 
> etc. */
> The envelope data table is a simple table of key/value pairs where the 
> key is an 8bit value identifying the entry
> and a variable length value that is interpreted depending on the key. 
> The list of provided envelope fields ends
> as the end of the envelope frame has been reached. that means, an 
> envelope must always fit into a single (first) frame.
>
>   * :scheme => uint8: 0x01
>       o http => uint8: 0x01
>       o https => uint8: 0x02
>       o custom => same as in method (if this is distinction is really
>         demanded)
>   * :method => uint8: 0x02
>       o GET => uint8: 0x01
>       o POST => uint8: 0x02
>       o PUT => uint8: 0x03
>       o DELETE => uint8:0x04
>       o custom => uint8: 0xFF, followed by one uint8 encoding the size
>         of the following bytes declaring the plaintext method value,
>         e.g. "PROPFIND"
>   * :path => uint8: 0x03, uint16 length in network byte order,
>     followed by $length octects declaring the path's value.
>   * :host => uint8: 0x04, uint16: length in network byte order,
>     $length octets declaring the host's value.
>   * :route => uint8: 0x05, uint8: length, $length octets that declare
>     this value. (field is only specified if known, thus previousely
>     announced by the remote server or this frame is part of a response
>     and we are to announce a routing identifier)
>   * :status => uint8: 0x06, uint16: code in network byte order  /* if
> This is the exact information an HTTP client (scheme,method,path,host) 
> or server (status) MUST currently sent as part of the (first) HEADERS 
> frame.
> So the change I propose is, to extract this information from the 
> HEADERS frame and put it into its own frame that also initiates the 
> stream implicitly.
>
> Having this in mind, it is a pleasure to implement HTTP routers 
> because those now don't have to decode the full HEADERS frames but 
> just decode the ENVELOPE frame and pass any continued frame to the 
> directed next-hop server.

Sadly the compression draft as written is completely incompatible with 
this type of load balancing. It operates a *stateful* compressor, such 
that every single HEADERS frame being received has to be decoded in 
order to re-encode using a separate connection-specific stateful 
compressor on the next-hop connections. This is mandatory for the 
compressed frames regardless of whether the ENVELOPE header is used to 
provide uncompressed details.
  The best load balancers can do under the current compression draft is 
to avoid complex re-encoding by emiting only Literal header 
representations and skipping all the traffic optimizations compression 
offers. Which converts them into near perfect DDoS 
bandwidth-amplification sources.

The sad state of affairs is that the *only* type of middleware which 
benefits from the proposed HTTP/2 is those which performs transparent 
interception and passive monitoring/recording of users traffic (ie the 
worst kind). Anything which starts modifying or manipulating (ie doing 
something useful for the ISP or CDN) MUST implement a full 
compressor/decompressor pair in order to keep the HTTP/2 statefulness in 
order.

Amos

---

From: Roberto Peon <[email-masked]>
Date: Wed, 10 Jul 2013 20:13:56 -0700

The intent was that one could negotiate a zero compression state, in which
case it ends up being similar to what is proposed here, I think, so long as
the appropriate key-value (or key) is in the static table.

-=R

On Wed, Jul 10, 2013 at 7:53 PM, Amos Jeffries <[email-masked]> wrote:

>> However, as a varnish [1] and a BSD [2] guy also raised there hands on,
>> is the lag of easy extraction of envelop information of an HTTP request
>> Please forgive me if this is already on-topic somewhere hidden, however,
>> I would really highly encourage you to consider
>> With that in mind, one might say that an HTTP/2 stream is not initiated
>> by a HEADERS frame but the ENVELOPE frame that contains the actual
>> these envelope frame, e.g. a simple GET, and no need for user-agent etc. */
> this type of load balancing. It operates a *stateful* compressor, such that
> every single HEADERS frame being received has to be decoded in order to
> re-encode using a separate connection-specific stateful compressor on the
> next-hop connections. This is mandatory for the compressed frames
> regardless of whether the ENVELOPE header is used to provide uncompressed
> details.
>  The best load balancers can do under the current compression draft is to
> avoid complex re-encoding by emiting only Literal header representations
> and skipping all the traffic optimizations compression offers. Which
> converts them into near perfect DDoS bandwidth-amplification sources.
>

---

From: Amos Jeffries <[email-masked]>
Date: Thu, 11 Jul 2013 16:20:12 +1200

On 11/07/2013 3:13 p.m., Roberto Peon wrote:
> The intent was that one could negotiate a zero compression state, in 
> which case it ends up being similar to what is proposed here, I think, 
> so long as the appropriate key-value (or key) is in the static table.

It seems not to be negotiable from the recipients side. The sender can 
choose to implement a non-Substituting HEADERS frame, but the recipient 
cannot exactly require it and thus is required to implement Substitution 
state for decoding regardless of whether it is ever used. If I have 
missed something please point it out to me.

PS. not that I am arguing for or against the current aproach. It is just 
sad that the worse form of middleware gains the most benefits and 
everything else is put to a bit of trouble.

Amos

>
> -=R
>
>
> On Wed, Jul 10, 2013 at 7:53 PM, Amos Jeffries <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>         I am just new to this list and I've just recently started
>         working through the HTTP/2 draft and all the blog articles
>         found about.
>         That being said, I myself have also very concerns others have
>         noticed as well, and would like to deeply show my intention to
>         help standing aside :)
>
>         I am the implementor of one of the many HTTP servers that's
>         being used in production, and one major feature is HTTP
>         routing / load balancing,
>         and I would really love to implement an HTTP/1.1 successor
>         that is (to be) officially labeled HTTP/2[.0].
>
>         However, as a varnish [1] and a BSD [2] guy also raised there
>         hands on, is the lag of easy extraction of envelop information
>         of an HTTP request message, such as method, path, but most
>         certainly the host.
>
>         Please forgive me if this is already on-topic somewhere
>         hidden, however, I would really highly encourage you to consider
>         initiated by a HEADERS frame but the ENVELOPE frame that
>         contains the actual
>         uncompressed and unmystified but compact information about
>         this request message.
>
>         flags: bit 1 = END_STREAM /* this HTTP message is complete
>         with just these envelope frame, e.g. a simple GET, and no need
>         for user-agent etc. */
>         The envelope data table is a simple table of key/value pairs
>         where the key is an 8bit value identifying the entry
>         and a variable length value that is interpreted depending on
>         the key. The list of provided envelope fields ends
>         as the end of the envelope frame has been reached. that means,
>         an envelope must always fit into a single (first) frame.
>
>         This is the exact information an HTTP client
>         (scheme,method,path,host) or server (status) MUST currently
>         sent as part of the (first) HEADERS frame.
>         So the change I propose is, to extract this information from
>         the HEADERS frame and put it into its own frame that also
>         initiates the stream implicitly.
>
>         Having this in mind, it is a pleasure to implement HTTP
>         routers because those now don't have to decode the full
>         HEADERS frames but just decode the ENVELOPE frame and pass any
>         continued frame to the directed next-hop server.
>
>
>     Sadly the compression draft as written is completely incompatible
>     with this type of load balancing. It operates a *stateful*
>     compressor, such that every single HEADERS frame being received
>     has to be decoded in order to re-encode using a separate
>     connection-specific stateful compressor on the next-hop
>     connections. This is mandatory for the compressed frames
>     regardless of whether the ENVELOPE header is used to provide
>     uncompressed details.
>      The best load balancers can do under the current compression
>     draft is to avoid complex re-encoding by emiting only Literal
>     header representations and skipping all the traffic optimizations
>     compression offers. Which converts them into near perfect DDoS
>     The sad state of affairs is that the *only* type of middleware
>     which benefits from the proposed HTTP/2 is those which performs
>     transparent interception and passive monitoring/recording of users
>     traffic (ie the worst kind). Anything which starts modifying or
>     manipulating (ie doing something useful for the ISP or CDN) MUST
>     implement a full compressor/decompressor pair in order to keep the
>     HTTP/2 statefulness in order.
>
>     Amos
>
>

---

From: Christian Parpart <[email-masked]>
Date: Thu, 11 Jul 2013 11:49:30 +0200

> regardless of whether the ENVELOPE header is used to provide uncompressed

I hope we did not misunderstand each other, with "ENVELOPE" I meant a frame
on its own that 1.) initiates a stream and 2.) contains the most important
envelope information needed to route a message.

> converts them into near perfect DDoS bandwidth-amplification sources.

here I don't think that will be much of the case, since HTTP/2 header
encoding should still be a way smaller in bytes than HTTP/1.1 and prior.

> compressor/decompressor pair in order to keep the HTTP/2 statefulness in

If HTTP/2 now would just put the routing information (method, path, host,
scheme [, routing-key]) cleartext upfront (still binary) an active HTTP
router could just parse this and pass the remaining as-is (e.g. unmodified)
to the next-hop. Although, if it then still whishes to add special headers
(such as Via), it could do so via the trailing HEADERS frame) - if I am not
too wrong.

Regards,
Christian Parpart.

---

From: Amos Jeffries <[email-masked]>
Date: Thu, 11 Jul 2013 22:08:53 +1200

On 11/07/2013 9:49 p.m., Christian Parpart wrote:
> On Thu, Jul 11, 2013 at 4:53 AM, Amos Jeffries <[email-masked] 
> I hope we did not misunderstand each other, with "ENVELOPE" I meant a 
> frame on its own that 1.) initiates a stream and 2.) contains the most 
> important envelope information needed to route a message.

I think I understand you. We considered something very similar early in 
the arguments against original SPDY design.

Consider that the client may send the load banacer a set of HEADERS 
after your ENVELOPE block. One of these HEADERS (doesn't matter which 
one) contains a header which is added to the decompressor state table 
and then used by teh following HEADERS blocks.
Your load balancer which does not do compression may send that HEADERS 
lock to server #1 and the others which depend on it to server #2, #3... etc.
The net result is corrupted decompression state at the servers #2 etc, 
and over time unpredictable HTTP syntax being received and acted on by 
all servers.

> encoding should still be a way smaller in bytes than HTTP/1.1 and prior.

Only in compressed form. The Literal form is equivalent to 
non-compressed HTTP/1.1 syntax. So the bandwidth is being expanded from 
HTTP/2 size up to HTTP/1 size. Low input causing the relay to generate 
large output is the very definition of amplification.

> If HTTP/2 now would just put the routing information (method, path, 
> host, scheme [, routing-key]) cleartext upfront (still binary) an 
> active HTTP router could just parse this and pass the remaining as-is 
> (e.g. unmodified) to the next-hop. Although, if it then still whishes 
> to add special headers (such as Via), it could do so via the trailing 
> HEADERS frame) - if I am not too wrong.

See my example about the HEADERS block state inter-dependency.

Amos

---

From: Michael Sweet <[email-masked]>
Date: Thu, 11 Jul 2013 08:16:29 -0400

Amos,

On 2013-07-11, at 6:08 AM, Amos Jeffries <[email-masked]> wrote:
> ...
>> here I don't think that will be much of the case, since HTTP/2 header encoding should still be a way smaller in bytes than HTTP/1.1 and prior.
> 
> Only in compressed form. The Literal form is equivalent to non-compressed HTTP/1.1 syntax. So the bandwidth is being expanded from HTTP/2 size up to HTTP/1 size. Low input causing the relay to generate large output is the very definition of amplification.

Well, a typed encoding would still offer some modest savings, and there are alternate compression schemes that could be used that are not so stateful.

I /do/ like the idea of a start frame (request, response, push promise) before the headers...

---

From: Martin Thomson <[email-masked]>
Date: Thu, 11 Jul 2013 09:36:02 -0700

On 10 July 2013 21:20, Amos Jeffries <[email-masked]> wrote:
> It seems not to be negotiable from the recipients side.

Compression context size = 0 is entirely negotiable from the recipient
end, with a small wrinkle, that I know some folks are working on.
Which is, a client can start using a default compression context size
prior to learning that a server has no space (substitute intermediary
as appropriate there).

---

From: James M Snell <[email-masked]>
Date: Thu, 11 Jul 2013 10:14:42 -0700

Yes, the ability to set compression context size to 0 is very useful.
My fears around this area are:

1. In order to achieve maximum throughput, Intermediaries may opt to
*always* set compression context to 0, forcing the headers to always
be passed as Literals, killing the utility of having the header
compression mechanism there in the first place.

2. The assumption of a non-zero default compression context size when
the connection is established opens a race condition that a malicious
sender could exploit in a denial of service attack. Yes, the receiver
could opt to terminate the connection once it detects bad behavior,
but there is still a potential window of time there where the receiver
could be forced to do significant additional work.

  (This is particularly bad given that header continuations are unbounded.)

3. Setting the compression context size to 0 does not stop the sender
from sending the Indexed Literal instructions anyway. The receiving
endpoint would still be required to process those instructions even if
the data is not actually being indexed, causing CPU cycles to be
consumed. For any individual block of headers it may not be a
significant load, but it's something that needs to be addressed.

  (This can be fixed in the spec by stating that any attempt to Index
any individual (name,value) whose size is greater than the available
header table size results in a Compression Error. Making this change
would mean that when Compression Context size is 0, the only operation
that would not result in an error is Literal without Indexing. This
was discussed on the list but as far as I can tell it's not yet
captured in the spec).

4. The fact that header continuations can be unbounded is deeply
troubling, especially given that the endpoint is required to buffer
and process the complete header block (well.. that's only half true,
the encoding does allow for incremental processing of the HEADERS
frame payloads but the spec requires that the complete header block is
always processed). Sure, the recipient is free to terminate the
connection as soon as it detects bad behavior, but the sender could
end up forcing the recipient to do a significant amount of extra
processing with a never ending sequence of HEADERS frames. Smart
implementations will know how to deal with this, yes, but overall it
adds to the already growing list of "New Complex Things" that an
HTTP/2 implementer needs to know about.

  (In the implementation I've done, I provide a configuration
parameter that allows a developer to cap the number of the
continuations and the total size of the header block)

I know that we're in "implementation" phase right now and that
everyone is busy getting their code ready for testing in August, but
after updating my implementation to the latest version of the draft,
my concerns with regards to stateful header compression definitely
remain.

---

From: Sam Pullara <[email-masked]>
Date: Thu, 11 Jul 2013 10:21:00 -0700

How sure are we that the entire idea of header compression isn't a bad idea? I implemented something similar in the WebLogic T3 protocol (BubblingAbbrevTable, probably still in there) and it was mostly just a pain. If I were to go back I would just use gzip with some agreed upon seed dictionary. Thought I would bring this up since it seems like it is a very controversial feature to begin with.

Sam

---

From: Roberto Peon <[email-masked]>
Date: Thu, 11 Jul 2013 12:35:34 -0700

I think it is perfectly reasonable for an intermediary to set the
compression size to zero if it wishes.

Market forces will (in the long-term) pick the correct strategy for this--
assuming the compression is effective at reducing latency, and that people
care about latency reductions, then eventually intermediaries might evolve
to use it.
If it is ineffective at reducing latency, or if reduced latency is not
actually desirable, then intermediaries would not use it.

The DoS vector you're talking about is not a DoS vector if the intermediary
resets all streams before the change-of-state-size comes into effect.
When the state size is 0, one should be able to use some kinds of 'indexed'
representations, so long as those representations refer only to items in
the static tables. Why do you believe that this would use more or less CPU?
(It should use less CPU and less memory...)

Headers are unbounded in HTTP. If any header is too large, there are only
two options:
1) terminate the connection
2) send a RST for that request and shove the bytes through the compressor.
This can be done in a streaming fashion so long as the recipient is smart
enough to tease out the instruction boundaries (not too hard, and certainly
no more difficult than interpreting HTTP/1.0 header fields). Assuming that
the state size is 0, these bytes can instead by discarded and do not need
to be interpreted at all.

---

From: Roberto Peon <[email-masked]>
Date: Thu, 11 Jul 2013 12:37:15 -0700

If one doesn't care about number of bytes on the wire, or if one doesn't
care about user-perceived latency, then obviously compression is a waste.
If one does care, then, especially on slower links, header compression does
a great deal to reduce latency as the HTTP metadata eats up a significant
fraction of available bandwidth on those links.

-=R

On Thu, Jul 11, 2013 at 10:21 AM, Sam Pullara <[email-masked]> wrote:

> How sure are we that the entire idea of header compression isn't a bad
> idea? I implemented something similar in the WebLogic T3 protocol
> (BubblingAbbrevTable, probably still in there) and it was mostly just a
> pain. If I were to go back I would just use gzip with some agreed upon seed
> dictionary. Thought I would bring this up since it seems like it is a very
> controversial feature to begin with.
>
> Sam
>

---

From: Sam Pullara <[email-masked]>
Date: Thu, 11 Jul 2013 12:51:10 -0700

It would be great to have a quantitative analysis of the benefit we can expect to get on various types of links and header sets so we could compare various implementations. I'm unconvinced these solutions are much better for real requests than gzip with an initial dictionary. Also, isn't bytes on the wire the wrong metric? Aren't these slow links much more sensitive to the number of packets / round trips?

Sam

On Jul 11, 2013, at 12:37 PM, Roberto Peon <[email-masked]> wrote:

> If one doesn't care about number of bytes on the wire, or if one doesn't care about user-perceived latency, then obviously compression is a waste.
> If one does care, then, especially on slower links, header compression does a great deal to reduce latency as the HTTP metadata eats up a significant fraction of available bandwidth on those links.
> 
> -=R
> 
>

---

From: Patrick McManus <[email-masked]>
Date: Thu, 11 Jul 2013 15:54:11 -0400

Header compression on the upstream path is more or less required to enable
effective  prioritized mux of multiple transactions due to interactions
with TCP congestion control. If you don't have it you cannot effectively
achieve the parallelism needed to leverage the most important HTTP/2
feature with out an RTT penalty. and RTT's are the enemy.

Without compression you can "pipeline" between 1 and 10 requests depending
on your cookie size. Probably closer to 3 or 4. With compression, the sky
is more or less the limit.

---

From: Roberto Peon <[email-masked]>
Date: Thu, 11 Jul 2013 12:56:55 -0700

A fair bit of this quantitative analysis was published with the SPDY
whitepapers.

Yes, packets matter.
Yes, RTT matters most.
number of packets is highly correlated with bytes on the wire

The encoders/compressors here were developed because:
1) any stream compressor is subject to the CRIME attack
2) gzip uses more memory/cpu than these more specific schemes
3) the encoders/compressors here were developed with an eye towards
allowing intermediaries to have much more control over the size and cost of
the compression stuff

The upstream path is often very limited.
If we want to have server push or similar be competitive with inlining, we
need the cost of that metadata to be low.
-=R

On Thu, Jul 11, 2013 at 12:51 PM, Sam Pullara <[email-masked]> wrote:

> It would be great to have a quantitative analysis of the benefit we can
> expect to get on various types of links and header sets so we could compare
> various implementations. I'm unconvinced these solutions are much better
> for real requests than gzip with an initial dictionary. Also, isn't bytes
> on the wire the wrong metric? Aren't these slow links much more sensitive
> to the number of packets / round trips?
>
> Sam
>
> If one does care, then, especially on slower links, header compression
> does a great deal to reduce latency as the HTTP metadata eats up a
> significant fraction of available bandwidth on those links.
>
> -=R
>
>

---

From: James M Snell <[email-masked]>
Date: Thu, 11 Jul 2013 13:02:30 -0700

On Thu, Jul 11, 2013 at 12:35 PM, Roberto Peon <[email-masked]> wrote:
>[snip]
> representations, so long as those representations refer only to items in the
> static tables. Why do you believe that this would use more or less CPU? (It
> should use less CPU and less memory...)
> [snip]

Well, as far as I can tell, according to the current header
compression draft, there is no "static table". The header table is
pre-populated, yes, but those items would fall out of the header table
via eviction as the table fills. There is nothing in the current
header compression draft that says those items are permanent.... Given
that, and given that we've already established that reducing the
header table size forces eviction, setting the size to zero would
cause all of the pre-populated items to be evicted.

- James

---

From: Roberto Peon <[email-masked]>
Date: Thu, 11 Jul 2013 13:07:29 -0700

I'm calling that the static table. Those are the elements you get for
"free" (as in the memory is allocated once in the process, as opposed to
for every connection).

-=R

On Thu, Jul 11, 2013 at 1:02 PM, James M Snell <[email-masked]> wrote:

> the

---

From: James M Snell <[email-masked]>
Date: Thu, 11 Jul 2013 13:29:33 -0700

Yes, and I'm saying that, the way the mechanism is currently defined,
as soon as you set the size to zero, those "free" items are removed
from the table and you no longer get them for "free"... so there is a
certain amount of state that *is* allocated per connection (even if
it's relatively minor).

Ultimately, the question of whether header compression can be used as
an attack vector will rest entirely on implementation and
experimentation.

On Thu, Jul 11, 2013 at 1:07 PM, Roberto Peon <[email-masked]> wrote:
> I'm calling that the static table. Those are the elements you get for "free"
> (as in the memory is allocated once in the process, as opposed to for every
> connection).
>
> -=R
>
>

---

From: Roberto Peon <[email-masked]>
Date: Thu, 11 Jul 2013 13:34:46 -0700

yup!
-=R

On Thu, Jul 11, 2013 at 1:29 PM, James M Snell <[email-masked]> wrote:

> "free"
> >> > static tables. Why do you believe that this would use more or less
> CPU?
> >> > (It

---

From: Sam Pullara <[email-masked]>
Date: Thu, 11 Jul 2013 15:43:36 -0700

The CRIME attack seems the biggest reason to switch something else. Brutal. I assume that we are positive that the proposed encoders don't have similar issues since they don't expose substrings of the cookies.

Sam

On Jul 11, 2013, at 12:56 PM, Roberto Peon <[email-masked]> wrote:

> A fair bit of this quantitative analysis was published with the SPDY whitepapers.
> 
> Yes, packets matter.
> Yes, RTT matters most.
> 3) the encoders/compressors here were developed with an eye towards allowing intermediaries to have much more control over the size and cost of the compression stuff
> 
> If we want to have server push or similar be competitive with inlining, we need the cost of that metadata to be low.
> -=R
> 
>

---

From: Roberto Peon <[email-masked]>
Date: Thu, 11 Jul 2013 15:56:44 -0700

Thusfar, the feedback we're received from security experts indicates that
it is comparable to an attack without the compression (i.e. requires
exponential time w.r.t. the size of the plaintext, or comparable to forcing
the use of a brute-force attack).

-=R

On Thu, Jul 11, 2013 at 3:43 PM, Sam Pullara <[email-masked]> wrote:

> The CRIME attack seems the biggest reason to switch something else.
> Brutal. I assume that we are positive that the proposed encoders don't have
> similar issues since they don't expose substrings of the cookies.
>
> Sam
>

---

From: Martin Thomson <[email-masked]>
Date: Thu, 11 Jul 2013 16:32:57 -0700

On 11 July 2013 15:56, Roberto Peon <[email-masked]> wrote:
> Thusfar, the feedback we're received from security experts indicates that it
> is comparable to an attack without the compression (i.e. requires

There are, of course, limitations on this.  If a particular header is
small, it becomes easier to guess.  If you were to say, spread a
bearer token into small pieces across multiple headers, then you would
open yourself up to a CRIME-like attack.

That said, the value that an attacker can gain is fairly marginal, and
there are ways to mitigate this.

The security considerations will, ultimately, expand on these sorts of
caveats as our knowledge improves.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 12 Jul 2013 06:58:06 +0000

In message <[email-masked]>, Sam Pullara writes
:

>idea?

I'm entirely convinced it is a bad idea.

The main gain that can be had from it, is compressing cookies and
that issue should be solved by moving the cookies onto the server,
indexed by a client provided session-id.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 12 Jul 2013 17:20:06 +1000

Hi PHK,

This has been brought up a number of times. I think what we need is a concrete proposal *with* a detailed plan for a workable transition to the new mechanism -- which seems to be the (or at least one) sticking point whenever this comes up.

Otherwise, it's amounts to "it'd be nice if the world worked this way..."

Cheers,

--
Mark Nottingham   http://www.mnot.net/

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 12 Jul 2013 20:26:19 +1200

On 12/07/2013 7:35 a.m., Roberto Peon wrote:
> Market forces will (in the long-term) pick the correct strategy for 
> this-- assuming the compression is effective at reducing latency, and 
> that people care about latency reductions, then eventually 
> intermediaries might evolve to use it.
> The DoS vector you're talking about is not a DoS vector if the 
> intermediary resets all streams before the change-of-state-size comes 
> into effect.

If you means RST_STREAM on all the initial streams which use a larger 
compression size then what you are doing is adding an RTT penalty to all 
those requests over and beyond what HTTP/1 suffers from already on a 
normal transaction. This is not a useful way forward (wastes packets, 
RTT and stream IDs) and resolving it is to make decompression with the 
default state size mandatory for all recipients. Which brings us full 
circle on the problem of having a default >0 in the dynamic part of the 
state tables.

> When the state size is 0, one should be able to use some kinds of 
> 'indexed' representations, so long as those representations refer only 
> to items in the static tables. Why do you believe that this would use 
> more or less CPU? (It should use less CPU and less memory...)

I did not mention CPU. Only the bandwidth amplification effects that 
agents disabling compression would incur and need to consider carefully.

Personally I would like to see a 127 entry mandatory static table in the 
spec itself and tied to the "2.0" version with a 127 entry optional 
dynamic table indicated by the high-end bit of the byte code. With a 
capacity byte size for dynamic table sent each way and senders forbidden 
to add new entries to the dynamic table until they hold the value from 
both ends of the connection. Agreed value being the minimum of both ends 
capacities.

Amos

---

From: Mike Belshe <[email-masked]>
Date: Fri, 12 Jul 2013 02:11:30 -0700

I'm also in favor of removing the compressor completely.

The reason is because we've seen the "negotiable compression protocol"
movie before.  At this point, its negotiable, and therefore just a big time
sink.  In the end, it will not be deployed because some player will have a
bug forcing everyone else to turn it off to avoid the hassle.   I apologize
if this sounds pessimistic - but history shows that this is a likely result.

If we can't agree on mandatory compression (which was long ago thrown out)
why not just add session state.  Cookies & User Agents can be set as state
which applies across all streams and be done.  It's mandatory, its super
simple, it fixes the single biggest redundant-bandwidth problem elegantly,
and in the end, I think this is basically what pkh and christian are asking
for under different names.

We'll probably save a lot of time debating and end up with a protocol which
is almost as good as a true compressor.
Mike

On Fri, Jul 12, 2013 at 1:26 AM, Amos Jeffries <[email-masked]> wrote:

>> this-- assuming the compression is effective at reducing latency, and that
>> people care about latency reductions, then eventually intermediaries might
>> evolve to use it.
>> intermediary resets all streams before the change-of-state-size comes into
>> effect.
>>
>
> those requests over and beyond what HTTP/1 suffers from already on a normal
> transaction. This is not a useful way forward (wastes packets, RTT and
> stream IDs) and resolving it is to make decompression with the default
> state size mandatory for all recipients. Which brings us full circle on the
> problem of having a default >0 in the dynamic part of the state tables.
>
>
>
>> 'indexed' representations, so long as those representations refer only to
>> items in the static tables. Why do you believe that this would use more or
>> less CPU? (It should use less CPU and less memory...)
>>
>
> spec itself and tied to the "2.0" version with a 127 entry optional dynamic
> table indicated by the high-end bit of the byte code. With a capacity byte
> size for dynamic table sent each way and senders forbidden to add new
> entries to the dynamic table until they hold the value from both ends of
> the connection. Agreed value being the minimum of both ends capacities.
>
> Amos
>
>

---

From: Gábor Molnár <[email-masked]>
Date: Fri, 12 Jul 2013 11:46:13 +0200

I've been thinking of the benefits using a streaming decompressor and
compressor here to quickly extract the routing information. But in the end,
I think it wouldn't solve the problem, at least when using a stateful
compression algorithm.

Let's assume we have a streaming decompressor that emits headers as soon as
it decompresses them, and always emits the headers needed to make routing
decision first (headers starting with colon). It would make routing very
quick. Let's suppose that the header block format would also be optimized
to support this behaviour. The proxy would start streaming the headers to a
compressor, which can forwards them immediately on the upsteam connection,
without waiting for the whole header set to be decoded.

The problem is that there's a DoS vector here. While sending the HEADERS
frames on the upstream connection, there must not be anything else sent
there. Now suppose a client is very slow, for example, it waits one second
between sending the first and the second (last) HEADERS frame to the proxy.
During this time (which can be arbitrary large), the proxy cannot send
anything on its upstream connection (and it cannot create a new connection
as it's forbidden in the current spec), so it's basically is blocked.

2013/7/12 Mike Belshe <[email-masked]>

> We'll probably save a lot of time debating and end up with a protocol
> which is almost as good as a true compressor.
> Mike
>
>
>
> On Fri, Jul 12, 2013 at 1:26 AM, Amos Jeffries <[email-masked]>wrote:
>

---

From: Gábor Molnár <[email-masked]>
Date: Fri, 12 Jul 2013 12:21:23 +0200

However, if we would make it possible to have multiple compression contexts
on a single HTTP/2 connection, that would make it work... Only those
requests would be blocked, which belong to the same compression context.
There would be a compression context for each user of the proxy, and the
upstream connection would multiplex HEADERS frames belonging to these
compression context. The proxy would have to maintain all those compression
contexts anyway... except, of course, if it negotiates a stateless
compression strategy with the clients.

This would make things more complicated (having a new, compression-context
id for every headers frame, limiting the number of compression contexts,
etc...), so I don't think it's a very good solution, but it could maybe
generate more fresh ideas regarding this problem.

2013/7/12 Gábor Molnár <[email-masked]>

> Let's assume we have a streaming decompressor that emits headers as soon
> as it decompresses them, and always emits the headers needed to make
> routing decision first (headers starting with colon). It would make routing
> very quick. Let's suppose that the header block format would also be
> optimized to support this behaviour. The proxy would start streaming the
> headers to a compressor, which can forwards them immediately on the upsteam
> connection, without waiting for the whole header set to be decoded.
>
>> If we can't agree on mandatory compression (which was long ago thrown
>> out) why not just add session state.  Cookies & User Agents can be set as
>> state which applies across all streams and be done.  It's mandatory, its
>> super simple, it fixes the single biggest redundant-bandwidth problem
>> elegantly, and in the end, I think this is basically what pkh and christian
>> are asking for under different names.
>>

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 12 Jul 2013 11:44:55 +0000

In message <[email-masked]>, Mark Nottingham wri
tes:

>This has been brought up a number of times. I think what we need is a =
>concrete proposal *with* a detailed plan for a workable transition to =
>the new mechanism -- which seems to be the (or at least one) sticking =
>point whenever this comes up.

I have given a concrete example multiple times, it's very simple:

	The client always sends along a session-identifier of N (128?)
	bits.

	If the first bit is zero, this is an anonymous, transient
	session, not (to be) associated with any other session.

	If the first bit is one, this is a persistent session
	identifier, which the server can use to look up any relevant
	state or information from previous instances of this
	session, in its local database.

	This replaces the Cookie: and Set-Cookie: headers, which
	SHALL NOT be sent in the HTTP/2.0 protocol.

Advantages:

	We get a fixed size session-identifier for HTTP routers to
	use for flow-routing.

	We get an actual (client controlled) session-concept, rather
	than all sorts of ad-hoc simulations with cookies.

	Data with privacy-concerns are stored on the server not on
	random clients the user happens to borrow or use.

	The overhead of encrypting and signing the data in cookies
	is avoided, since they are stored on the server side where
	nobody can fudge them.

Backwards compatibility:

	It should be obvious that simulating the Cookie concept for
	framework compatibility on the server side is a trivial
	matter of programming:  Rather than send set-cookies, write
	them to a database, indexed by the session-id.  Rather than
	receive Cookie: headers, look them up in the database.

There, solved.

Again.

---

From: Michael Sweet <[email-masked]>
Date: Fri, 12 Jul 2013 08:40:48 -0400

Then just add the User-Agent and Server values to the settings frames the client and server exchange?

Sent from my iPad

---

From: Christian Parpart <[email-masked]>
Date: Fri, 12 Jul 2013 14:47:09 +0200

> client and server exchange?

if it would be *that* simple, then dont' forget the cookie and the
user-agent.

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 12 Jul 2013 14:56:28 +0200

Hi Poul-Henning,

> 	receive Cookie: headers, look them up in the database.

Not really in fact. While I tend to generally agree with the points
you make for scalability, this one does not scale. One of the big
benefits of cookies is that client is responsible for synchronizing
information between multiple servers *if needed*. When you're building
an architecture using anycast DNS + ECMP + L4 load balancers to reach
your servers, you can't predict if a client will come back to the same
place to retrieve its context, and having the ability to make it hold
*some* data is really useful.

If you store everything on server-side, you're forced to synchronize
everything between all servers of all datacenters because you don't
even know where your client will go with next hit, let alone parallel
requests. And this is clearly not possible on many of the platforms
where both of our products are deployed to achieve connection rates
in the 6 digits.

So there *is* some use to store data on the client, it's just that it
has been long abused to store session identifiers because it was the
only mechanism available.

While I'd like to see a simple session management system (like the one
you propose maybe, even if it does not cover the case of low entropy
client devices), I don't want to see the cookies disappear. I just want
to be able to rely on session ID without having to parse cookies when
that's not needed.

Regards,
Willy

---

From: Patrick McManus <[email-masked]>
Date: Fri, 12 Jul 2013 08:59:30 -0400

cookies and user agents, while very redundant, are not 100% consistent
between transactions. Especially in proxying scenarios, but other places
too.

And even if we removed those, a request header would still be ~300 bytes -
which is just too big to support the many dozens of subresources that
commonly make up a page today in one rtt.

I understand some of the frustration, but this is something we've seen work
and imo is necessary to enable the fundamental muxxing changes http/2
presents.

---

From: Jeff Pinner <[email-masked]>
Date: Fri, 12 Jul 2013 08:22:17 -0700

> I'm also in favor of removing the compressor completely.

So the compressor buys us the ability to share headers between streams and
possibly to reduce the size of the headers via some sort of encoding
(whether it's typed encodings, or huffman compressed strings, or varint
lengths, etc). So a dumb proposal:

A HEADERS frame consists of encoded name values pairs, let's say varint
length followed by UTF-8 bytes of the string (we can argue over compressed
strings, types, etc. later, but basically no indexing into shared state).

Sending a HEADERS frame on Stream-ID 0 creates a set of headers that gets
saved and added to the HEADERS frame that opens any streams after it is
sent. Sending a new HEADERS frame on Stream-ID 0 overwrites the previous
frame.

This allows us to share Cookies, User-Agent, Host, etc. between requests,
but wouldn't allow for any response header sharing. It would allow us to
share headers for pushed responses since those are streams opened by the
server.

---

From: Martin Thomson <[email-masked]>
Date: Fri, 12 Jul 2013 08:51:19 -0700

> as it's forbidden in the current spec), so it's basically is blocked.

So don't do that then.

If you have to have two headers frames, then we simply don't create a
restriction like the current one whereby both have to appear in
immediate succession.

As has been suggested, if routing information is important, we can
conceive of ways to ensure that it appears first.  That doesn't
necessarily constrain the rest of the design.  And existing
constraints are a product of the existing design.  A changed design
necessarily creates new constraints.

---

From: Roberto Peon <[email-masked]>
Date: Fri, 12 Jul 2013 09:31:46 -0700

Correct, I mean to say that, if you can't deal with 4k of state in the
first RT then you RST those requests, causing them to suffer one RT of
latency.

Personally, I think one should be able to deal with state for the first RT,
especially you're going to have more than that in general in the IO
buffers, kernel buffers, etc.
But, anyway, assuming you're under DoS attack, there are multiple options:

1) send a new settings frame with the size you want, and RST everything
'till that becomes effective.
2) we implement James' proposal of a goaway-and-come-back after sending the
settings, where the settings are effective on the next connection
3) If we kept the persistent settings on the client, the first time the
client spoke to the intermediary, it would learn and have appropriate
settings in the future.
4) reject HTTP/2 (which uses more state in exchange for lower latency) in
preference for HTTP/1.0, which will put less data as persistent state for
the first RT.

5) assuming we did the DNS thing, the client would already have the correct
setting, and there'd be no additional latency.

I'm confused the complaining about extra latency of any of the solutions
above, however.
Do we care about latency or not?
Arguments that complain that we have to hold state for 1 RT, and that we
want to eliminate all state make me think that latency is viewed as a
distant-second consideration.
Is latency a prime consideration, as indicated in the charter, or not?

(And you can call it session caching or whatever, it is still just state on
the other side and is all the same idea).
-=R

---

From: "Ludin, Stephen" <[email-masked]>
Date: Fri, 12 Jul 2013 11:32:02 -0500

>to be able to rely on session ID without having to parse cookies when

+1.  

The distributed data store that the sea of user-agents compromises dwarfs
anything we could engineer easily on the server (or proxy) side.  Though I
agree completely that there are numerous advantages to the idea of the
user-agent holding only the session 'key' and the bulk of the data being
on the server, the complexities of dealing with that in a distributed and
massive origin/proxy environment are daunting and fear would erase much of
the benefit that the distribution hopes to provide.

I just implemented the header compression spec and it felt incredibly
complex so I am definitely inclined to figure out a scheme for
simplification.  My concern is similar to Mike's: we will get buggy
implementations out there that will cause us to to avoid its use in the
long run.  The beauty of the rest of the HTTP/2.0 spec is its simplicity
of implementation.  I like to think we can get there with header
compression as well.

---

From: Roberto Peon <[email-masked]>
Date: Fri, 12 Jul 2013 09:36:29 -0700

This was the first thing I experimented. :)
It either requires two different state size settings, or it makes state
size management .... interesting....
Having a single table made much more sense and was less complicated,
especially for proxies.

-=R

---

From: James M Snell <[email-masked]>
Date: Fri, 12 Jul 2013 09:53:06 -0700

Most of the feedback that I've seen around header compression can be
summarized as:

  Sending less data on the wire... Good.
  Less latency... Good.
  More efficiency... Good.
  Less visibility in the protocol... Bad.
  Complicated implementation... Bad.
  Introducing statefulness in the protocol... Bad.

Where I come down on this is simple:

Yes, we need a way of encoding headers more efficiently in HTTP/2
relative to HTTP/1... however, as the mantra states, Perfect is the
Enemy of Good. The header compression algorithm as currently stands is
*technically* a very good approach, but that doesn't mean it's the
right approach for right now.

The problems we see in the verbosity and repetitiveness of HTTP
request and response headers has more to do with how those headers are
defined and used that it does with how those headers are encoded on
the wire. There are reasonable application-level approaches we could
be exploring to reduce redundancy and increase efficiency that would
not require complex new mechanisms and requirements to be added to the
base framing protocol. For instance, I have demonstrated many times
that by simply changing the way we define header values, we can make
significant reductions in header transmission size.

>From everything that I can see so far, Header Compression is
completely orthogonal to the basic operation of the new HTTP/2 framing
layer. It is not critical for the new framing layer to be successful.
Yes, it has it's benefits. Yes, it makes things more efficient. Yes,
it has security advantages relative to other compression options. But
no, it is not the Simplest Thing That Could Possibly Work and no, it's
not technically required in order to make HTTP/2 successful... and so
far, the majority public reaction seems to be "Whoa! WTF is with all
the new complexity! Dial it back a bit please!"

It's good that people are starting to get implementation experience
with what's been written up so far. It's good to start getting interop
testing going. For me, going through my own implementation of current
header compression mechanism has simply reinforced what I had already
suspected and feared: it's a lot of very well designed and functional
new complexity that has some benefit but is not strictly required to
get the job done.. and, what's worse, is the fact that it makes us
jump through some additional design hoops (e.g. the routing question)
that would otherwise be fairly simple to address.

That all said, I'm definitely willing to be convinced otherwise. Let's
see what happens in Germany next month, but let's definitely keep an
open mind towards alternative, less complicated (albeit, possibly less
efficient) approaches that do not incur such a large New Complexity
Tax.

- James

On Fri, Jul 12, 2013 at 9:36 AM, Roberto Peon <[email-masked]> wrote:
> It either requires two different state size settings, or it makes state size
> management .... interesting....

---

From: Amos Jeffries <[email-masked]>
Date: Sat, 13 Jul 2013 05:29:59 +1200

> where both of our products are deployed to achieve connection rates

In these cases you build your session identifier to contain a shard 
reference as well as an UID.
For example I'm pretty sure that Facebook does not replicate all users 
entire account details onto each and every server, or in a Cookie being 
sent back and forward. For good reason. No, they have an ID which allows 
extremely fast backend referencing for whatever assembly server is 
running to access whatever data server the users account it stored on. 
Not exactly complex even when running at large scale.

> only mechanism available.

Only the Session ID and/or a shard reference (combined in one opaque 
blob is even better) need be stored on the client.

Amos

---

From: Amos Jeffries <[email-masked]>
Date: Sat, 13 Jul 2013 06:03:35 +1200

On 13/07/2013 4:31 a.m., Roberto Peon wrote:
> Personally, I think one should be able to deal with state for the 
> first RT, especially you're going to have more than that in general in 
> the IO buffers, kernel buffers, etc.
> 1) send a new settings frame with the size you want, and RST 
> everything 'till that becomes effective.

-0. RST wastes Stream IDs. We only have a limited 31-bit resource there. 
It will be exhausted easily enough on long running or high throughput 
connections. Every RST is one step clsoer to exhaustion. They are not so 
much the enemy as RTT pehapse, but still an enemy.

> 2) we implement James' proposal of a goaway-and-come-back after 
> sending the settings, where the settings are effective on the next 
> connection

-1. We have absolutely zero confidence that the followup will be to the 
same half of the planet as the first connection, let alone the same server.

> 3) If we kept the persistent settings on the client, the first time 
> the client spoke to the intermediary, it would learn and have 
> appropriate settings in the future.

-1. same reasons as for (2) above.

> 4) reject HTTP/2 (which uses more state in exchange for lower latency) 
> in preference for HTTP/1.0, which will put less data as persistent 
> state for the first RT.

-1. Counter to WG primary goals of rolling out HTTP/2.

> 5) assuming we did the DNS thing, the client would already have the 
> correct setting, and there'd be no additional latency.

-1. For all the reasons discussed earlier about DNS failures. There is 
zero confidence that the next-hop server is the one DNS was mentioning. 
In fact from the router/middleware viewpoint this thread is about there 
is nearly 100% confidence that the DNS is *not* about the next-hop.

> I'm confused the complaining about extra latency of any of the 
> solutions above, however.

Latency? who mentioned latency? This is all about implicit security 
vulnerabilities/considerations and potential frame routing problems in 
the compression design. Latency is miles away from all that.

> Arguments that complain that we have to hold state for 1 RT, and that 
> we want to eliminate all state make me think that latency is viewed as 
> a distant-second consideration.

Lets put this the other way. Home router with 512KB baked into the 
drivers for the HTTP stack responsible for routing traffic between ~9 
devices (2 family members with phone, tablet, and laptop each, and old 
house PC system, a games console, and digital TV - maybe more but that 
is a fairly accurate description of my non-tech friends household).
Someone is watching TV, with ~256KB of streaming state in the stack, 
both have their phones on 24/7 logged into their favourite Social media 
site with combined 184KB of state in the stack. Then someone opens a 
connection to website X a log sin with 12x 7KB of Cookie data on the 
first request headers.
  --- this is a realworld HTTP/1 situation I only just got finished 
debugging the device crash for yesterday. What happens in HTTP/2 if the 
device is unable to specify a max-72KB dynamic table size to the client? 
or even to advertise new lower dynamic table sizes to the existing 
clients fast enough not to block the new client?

Naturally I am worried. About the setup and whether it is negotiation or 
mandatory initial state, etc.

Amos

>
> (And you can call it session caching or whatever, it is still just 
> state on the other side and is all the same idea).
> -=R
>
>
> On Fri, Jul 12, 2013 at 1:26 AM, Amos Jeffries <[email-masked] 
>         I think it is perfectly reasonable for an intermediary to set
>         the compression size to zero if it wishes.
>
>         Market forces will (in the long-term) pick the correct
>         strategy for this-- assuming the compression is effective at
>         reducing latency, and that people care about latency
>         reductions, then eventually intermediaries might evolve to use it.
>         If it is ineffective at reducing latency, or if reduced
>         latency is not actually desirable, then intermediaries would
>         not use it.
>
>
>         intermediary resets all streams before the
>         change-of-state-size comes into effect.
>
>
>     If you means RST_STREAM on all the initial streams which use a
>     larger compression size then what you are doing is adding an RTT
>     penalty to all those requests over and beyond what HTTP/1 suffers
>     from already on a normal transaction. This is not a useful way
>     forward (wastes packets, RTT and stream IDs) and resolving it is
>     to make decompression with the default state size mandatory for
>     all recipients. Which brings us full circle on the problem of
>     having a default >0 in the dynamic part of the state tables.
>
>
>
>         When the state size is 0, one should be able to use some kinds
>         of 'indexed' representations, so long as those representations
>         refer only to items in the static tables. Why do you believe
>         that this would use more or less CPU? (It should use less CPU
>         and less memory...)
>
>
>     I did not mention CPU. Only the bandwidth amplification effects
>     that agents disabling compression would incur and need to consider
>     carefully.
>
>     Personally I would like to see a 127 entry mandatory static table
>     in the spec itself and tied to the "2.0" version with a 127 entry
>     optional dynamic table indicated by the high-end bit of the byte
>     code. With a capacity byte size for dynamic table sent each way
>     and senders forbidden to add new entries to the dynamic table
>     until they hold the value from both ends of the connection. Agreed
>     value being the minimum of both ends capacities.
>
>     Amos
>
>

---

From: Roberto Peon <[email-masked]>
Date: Fri, 12 Jul 2013 12:42:58 -0700

On stream ID exhaustion:
Ok, so if you do 10k requests/second, you'll have to make a new connection
every ~30 hours. Is this really problematic?

On persistent settings/goaway-and-come-back:
Correct. If your loadbalancing arrangement is flapping and almost never
puts connections back in the location where they were from ~100ms before,
then these solutions will not work reliably.
If you're flapping, then I think you'd have bigger problems then, however :)

On rejecting HTTP/2:
HTTP/2 is thusfar trading a constant sized in-userspace buffer for latency
reduction and kernel-space reduction in memory.
If that isn't acceptable, then the only options are to hobble HTTP/2 by
trading latency for memory, or to allow HTTP/2 to trade constant amounts of
memory for latency and do an HTTP/1 to HTTP/2 gateway.

On DNS reliability:
If DNS is unreliable for some small portion of time.. .who cares? If it
screws up much of the time, then I agree it is not an adequate mitigation.

Thusfar, we have two kinds of deployments in mind for HTTP/2: on the
internet, and within a private network.
For the former, most people seem to be saying that they'll deploy clients
that use ALPN to perform the negotiation.
In such a case, any proxy must be explicit, and thus there is a chance to
do the DNS resolution.
Within a private network people seem to be saying that they'd negotiate
unencrypted over port 80, in which case the server performs the upgrade and
can have already stated that it wants the compression state to be zero
sized before the client sends any information.

Thus, the only place where there is any potential for problems are for
explicitly configured proxies, or for actual endpoints where a DNS lookup
for that entity is likely.

In any case, we're talking about having a dynamic table size of 4k by
default, nothing near the large amount you had to debug in the router (I
feel your pain!)
In the intercepting-home-router case, btw, the RTTs involved are so small
that doing a RST for one RTT costs very little. Unlike HTTP/1, we do have a
mechanism that causes the browser to try again!
In such cases, thus, the tradeoff of latency for memory makes sense because
the costs in latency is so tiny.

-=R

On Fri, Jul 12, 2013 at 11:03 AM, Amos Jeffries <[email-masked]>wrote:

>> Personally, I think one should be able to deal with state for the first
>> RT, especially you're going to have more than that in general in the IO

>
>  2) we implement James' proposal of a goaway-and-come-back after sending
>> the settings, where the settings are effective on the next connection
>>
>
> zero confidence that the next-hop server is the one DNS was mentioning. In
> fact from the router/middleware viewpoint this thread is about there is
> nearly 100% confidence that the DNS is *not* about the next-hop.
>
>
> vulnerabilities/considerations and potential frame routing problems in the
> compression design. Latency is miles away from all that.
>
>
>
> Lets put this the other way. Home router with 512KB baked into the drivers
> for the HTTP stack responsible for routing traffic between ~9 devices (2
> family members with phone, tablet, and laptop each, and old house PC
> system, a games console, and digital TV - maybe more but that is a fairly
> accurate description of my non-tech friends household).
> Someone is watching TV, with ~256KB of streaming state in the stack, both
> have their phones on 24/7 logged into their favourite Social media site
> with combined 184KB of state in the stack. Then someone opens a connection
> to website X a log sin with 12x 7KB of Cookie data on the first request
> headers.
> device is unable to specify a max-72KB dynamic table size to the client? or
> even to advertise new lower dynamic table sizes to the existing clients
> fast enough not to block the new client?
>
>> (And you can call it session caching or whatever, it is still just state
>> on the other side and is all the same idea).
>> -=R
>>
>>
>> On Fri, Jul 12, 2013 at 1:26 AM, Amos Jeffries <[email-masked]<mailto:
>> [email-masked]>> wrote:
>>

---

From: Yoav Nir <[email-masked]>
Date: Fri, 12 Jul 2013 19:46:46 +0000

Hi

This thread has forked to discussing session management. I'd like to call your attention to the fact that in the past, this working group was considered too busy with HTTP/2.0 to spend time on things like session management or HTTP authentication schemes. For this reason it was suggested that the WebSec working group work on session management. 

Right now, on the WebSec mailing list ([1]) there is a thread ([2]) about taking on this work. Being engineers, the discussion there is veering towards solutions before there's even an agreed-upon problem statement, but that's besides the point. If you are interested in defining an alternative to session management based on session cookies, please subscribe ([3]) to the WebSec mailing list and say so. 

Thanks

Yoav

[1] http://www.ietf.org/mail-archive/web/websec/current/threads.html
[2] http://www.ietf.org/mail-archive/web/websec/current/msg01680.html
[3] https://www.ietf.org/mailman/listinfo/websec

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 12 Jul 2013 22:00:49 +0200

> blob is even better) need be stored on the client.

A shard reference or session ID are two completely different things. A
session ID comes from one referential. A shard reference indicates what
referential to use. That's why a user-provided session ID cannot be used
as a shard reference and prevents from scaling. I *do* use cookies exactly
as you describe because it's the only way to scale, and have been doing so
for almost a decade now. In a cookie, you can indicate the DC, the server,
a location in a database, etc... Whatever you need to perform very fast
access. You simply can't do that by learning massive amounts of randoms
generated by clients.

Regards,
Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 13 Jul 2013 07:11:29 +0000

In message <[email-masked]>, Yoav Nir writ
es:

>This thread has forked to discussing session management. I'd like to call 
>your attention to the fact that in the past, this working group was consider
>ed too busy with HTTP/2.0 to spend time on things like session management
>or HTTP authentication schemes. For this reason it was suggested that the 
>WebSec working group work on session management. 

Then just call it a "flow-routing-label" here, and let WebSec figure
out for themselves that it can also be used as a session identifier.

There's no need to let procedural overhead get in the way of good ideas.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 13 Jul 2013 07:08:03 +0000

In message <[email-masked]>, Willy Tarreau writes:

>information between multiple servers *if needed*. 

Since when has a minor scalability issue for Big Companies
become more important than end-users privacy ?

Sorry, that argument simply does not fly with me.

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 13 Jul 2013 11:31:34 +0200

> become more important than end-users privacy ?

Huh ? What end-user privacy issue do you see in having the DC and
server id in a cookie ? This is totally irrelevant.

Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 13 Jul 2013 09:44:39 +0000

>server id in a cookie ? This is totally irrelevant.

You should read up on why EU decided to go in and regulate cookie
use, not everybody use them the way you seem to do:

http://www.ico.org.uk/~/media/documents/library/Privacy_and_electronic/Practical_application/cookies_guidance_v3.pdf

We can agree or disagree about the wisdom of the actual regulation
enacted, but I think their rationale for why regulation was needed
is pretty spot on:  Cookies were being used to strip users of
any resemblance of privacy.

IETF and HTTPbis should grasp a clue from this, and make sure that
in HTTP/2.0 Cookies are stored where they belong:  In the server
end, and that a session-concept, 100% under the users control
is replacing it.

---

From: Sam Pullara <[email-masked]>
Date: Sat, 13 Jul 2013 09:49:42 -0700

This can be (and in many cases is already) solved at any web company big enough to need to solve it. I'm 100% in favor of using a client generated session identifier. This would dramatically simplify HTTP/2 in a real way. Cookies are from another era when building a server-side scalable session data store was difficult and expensive. I would argue that isn't the case anymore.

Sam

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 13 Jul 2013 19:32:22 +0200

On Sat, Jul 13, 2013 at 09:49:42AM -0700, Sam Pullara wrote:
> This can be (and in many cases is already) solved at any web company big
> enough to need to solve it. I'm 100% in favor of using a client generated
> session identifier. This would dramatically simplify HTTP/2 in a real way.
> Cookies are from another era when building a server-side scalable session
> data store was difficult and expensive. I would argue that isn't the case
> anymore.

Until you are able to shrink the time it takes to synchronize two servers
at opposite sides of the world, you'll end up causing delays that are higher
than the average RTTs we're trying to get rid of. Not to mention the amount
of inter-DC traffic.

I'm sorry, but cookies are *not* evil. Some uses of cookies are evil. You
don't need to break the web just because of some improper usages. Otherwise
you can as well advocate against computers because computers are also used
to track people and retrieve a lot of information about them that is not
possible to collect by hand. That's simply non-sense.

We could possibly support very short cookies (eg: 16 bit). That should be
enough for most large deployments, and clearly not enough to track users.

But I want to insist that scalable state management is an important piece
of the net that we must not break just because it makes us feel better.

Willy

---

From: "Mark Delany" <[email-masked]>
Date: 13 Jul 2013 18:29:02 +0000

(Gack wrong sender address - you might see this twice).

> This can be (and in many cases is already) solved at any web company big enough to need to solve it. I'm 100% in favor of using a client generated session identifier. This would dramatically simpli…

Just so I understand, does "solved" in this context means:

a) Clients stop exchange cookies

b) Clients start exchanging sessions ids

   (As we know, both of these can be achieved thru the cookie
   mechanism today by the server generating a session id cookie and
   dropping/expiring all other cookies).

c) Assuming origins and other internal systems will still need to see
   a cookie flow for many years to come, participating edge systems
   will need to act as a persistent, consistent cookie cache/proxy on
   behalf of the clients.

   Client -> sessionid -> Edge --sessionid+Cookies--> origin
                           |
                           ^
                           |
                    | Cookie Jar |
                           |
                           ^
                           |
   Client <- sessionid <- Edge <--New Cookies-- origin

d) In a decade or two, when all internal systems grok session id, the
   edges can discard their cookie proxy implementations.

The benefit being that you achieve perfect cookie compression on
your external links without inventing a fancy compression system.

The cost being that large/complicated sites will need to implement a
persistent, consistent, distributed cookie store at their edges until
they upgrade all internal systems.

The argument being that the big guys get the most benefit, ergo they
should bear the cost?

Mark.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 13 Jul 2013 18:43:20 +0000

>I'm sorry, but cookies are *not* evil. 

Cookies are not evil, but they cause problems which HTTP/2.0 does not
need to cause.

"Automatic EU Cookie directive compliance" would be a really great
selling point.

>enough for most large deployments, and clearly not enough to track users.

I think it is smarter to both solve the cookie and session problems with
a single field.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 13 Jul 2013 18:48:31 +0000

In message <[email-masked]>, "Mark Delany"

>   Client <- sessionid <- Edge <--New Cookies-- origin

Yes, that's the obvious backwards compatibility design, except
that for "Edge" you might often just read "apache" or "ngnix".

>your external links without inventing a fancy compression system.

Good way to express it.

>The argument being that the big guys get the most benefit, ergo they

It could actually be interesting if somebody would go through their
HTTP/1.1 traffic and estimate how much bandwidth would be saved.

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 13 Jul 2013 21:12:02 +0200

> I think it is smarter to both solve the cookie and session problems with

No problem but we really need the server side to be able to adjust
part of this field. If we have a 128-bit session ID whose 16 first
bits are preset to zero by the client and may be changed by the
server, we can most likely replace the existing cookie system (it
will also permit servers to handle some of the duplicates that
clients would inevitably cause).

Willy

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 13 Jul 2013 21:14:42 +0200

> HTTP/1.1 traffic and estimate how much bandwidth would be saved.

On most static servers (ie most of the round trips for loading a page),
it will actually *inflate* the traffic since we normally don't send
cookies there, but with the proposal, I suppose the session ID will be
sent anyway (or we need at least one bit to indicate its presence).

Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 13 Jul 2013 19:21:32 +0000

In message <[email-masked]>, Willy Tarreau writes:

>part of this field.

Could you oblige me with an example, since I seem unable to come
up with one myself ?

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 13 Jul 2013 21:30:52 +0200

> Could you oblige me with an example, since I seem unable to come

Yes. Client picks a random session ID with the highest 16 bits = 0,
sends the request to the first server. The load balancer overthere
puts the DC ID and the local server ID in these bits and sends this
ID back to the client along with the response. When upon a subsequent
request the client is directed to a different DC, the information
about the location of the client's context is found and the context
can be retrieved.

Willy

---

From: Yoav Nir <[email-masked]>
Date: Sat, 13 Jul 2013 20:24:24 +0000

> There's no need to let procedural overhead get in the way of good ideas.

It's more than procedural. Whatever is wrong with the use of cookies to manage sessions is wrong in HTTP/1.x just as much as it is in /2.0.

There will come a day when we can say that HTTP/1.1 is only for legacy applications, and that it's fine to make sweeping changes only in HTTP/2.0 (or perhaps introduce them in HTTP/3.0). That day is not today, and if we're going to fix it, we might as well fix it in 1.0 as well.

Your proposal (http://lists.w3.org/Archives/Public/ietf-http-wg/2013JulSep/0284.html) includes prohibiting cookies on HTTP/2.0. This directly conflicts with this working group's goal of creating a protocol that is a drop-in replacement for HTTP/1.1. If I have a web application running on an Apache server that makes use of cookies to handle session management, I can't just install modhttp2, add a line in the config file and have everything continue to work. I would have to do significant work to make my website work with HTTP/2.0. If we accept your proposal, this would be an impediment to deploying HTTP/2.0.

So I think that a new session management scheme is something that should be orthogonal to HTTP version, and should be a separate work item. This is regardless of whether that work item belongs in httpbis or in websec. 

It should also be noted that you're missing two requirements from session management:
 1. That the session be bound to an authenticated identity.
 2. That users won't have to authenticate too often.

This is easy to accomplish with cookies: you set the cookie when the user authenticates, and you allow it to persist for as long as you want. I'm not sure how this can be accomplished by a session identifier chosen by the UA.

Yoav

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 13 Jul 2013 20:28:13 +0000

>about the location of the client's context is found and the context

Thanks!  Now I understand the goal.

It's unclear to me if it is a good idea to make part of the
session identifier, or if it should be a separate field, but
I really like the basic idea.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 13 Jul 2013 20:41:11 +0000

In message <[email-masked]>, Yoav Nir writ
es:

>It's more than procedural. Whatever is wrong with the use of cookies to 
>manage sessions is wrong in HTTP/1.x just as much as it is in /2.0.

Yes, that's why we should solve the problem in HTTP/2.0 and make the
protocol so good that people will desire to migrate because it is
simply better.

>Your proposal (http://lists.w3.org/Archives/Public/ietf-http-wg/2013JulSep/0284.html) includes prohibiting cookies on HTTP/2.0. This directly conflicts
> with this working group's goal [...]

You say that like those goals were god-given and imutable.

Maybe the reason why the current draft is so mediocre is that the goal
that was aimed at was just not the right one ?

---

From: Yoav Nir <[email-masked]>
Date: Sat, 13 Jul 2013 21:14:08 +0000

> protocol so good that people will desire to migrate because it is

Better, but requires a lot of work to deploy. This runs the risk of repeating the IPv6 experience. We make it too different, and people will not want to deploy it.

IPv6 was rotting on the vine for 15 years before deployment started in earnest. And that happened only because IPv4 addresses were finally running out and all the NAT bandaids were not enough. 

I don't see any scarce resource in HTTP/1.1 that will force people to switch to HTTP/2.0.  Just because something's better does not mean that migration is worth the effort. 

Yoav

---

From: Willy Tarreau <[email-masked]>
Date: Sat, 13 Jul 2013 23:17:29 +0200

> I really like the basic idea.

I think it makes sense to have it part of the session ID because
anyway it's very likely that the server side will use the complete
version to try to improve unicity (which is still not guaranteed).

Willy

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 13 Jul 2013 22:24:43 +0000

In message <[email-masked]>, Yoav Nir writ
es:

>Better, but requires a lot of work to deploy. This runs the risk of repeating
>ng the IPv6 experience. We make it too different, and people will not want 
>to deploy it.

Uhm no.

The "IPv6 experience" is defining a new protocol which offers no
tangible benefits for anybody, and comes with a lot of transistion
headaches.

That's where we are with the current HTTP/2.0 draft.

If IPv6 had addressed one of the major needs, for instance multihoming
without BGP (ie: Anycast), people would have jumped on it, but all
the things people wanted threatned the big ISP monopolies, so those
"unnecessary features" were killed due to "lack of consensus".

If HTTP/2.0 offers something people actually want, they will swich,
if all it offers is "more of the same, only slightly different" they
will not.

Solving the EU/Cookie privacy issue and saving bandwidth at the
same time, would be a desirable feature.

---

From: Stephen Farrell <[email-masked]>
Date: Sat, 13 Jul 2013 23:40:19 +0100

Its a bit of a moot point I guess but...

On 07/13/2013 09:24 PM, Yoav Nir wrote:
> allow it to persist for as long as you want

I've always been amused that HTTP needs to be able to
manage state for decades. It'd be truly impressive if
a browser really managed state that lasts far longer
than the h/w on either side and probably also longer
than any piece of n/w kit in between.

If HTTP/2.0 were to impose an upper bound on cookie
lifetime of say, a session, that'd be good IMO. But I
guess that probably would be out of charter, even if
it'd be a good thing, as it'd break stuff. OTOH, it'd
arguably be a good thing to leave such stuff behind when
moving to HTTP/2.0.

Anyway, unless there's a groundswell of wg opinion that
cookies that expire in years are plain stupid and are
just not doing HTTP state management, I'll probably
shut up about it.

S.

PS: Yes, I know that forcing all cookies to be session
cookies would not be a panacea. Doesn't mean it wouldn't
be a good thing though.

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 14 Jul 2013 07:48:40 +0200

> arguably be a good thing to leave such stuff behind when

I think that *session* cookies are needed, but *permanent* cookies
are mostly used by ads and user tracking, even if a few sites use
them to store user preferences.

One elegant solution would probably be to systematically have UAs
ask end users whenever a cookie is sent to be stored for more than
(say) 24h. Site designers will take care of this because they don't
want to make their site emit warnings that upset end users.

And if the cookie is used to store preferences, it is normal that
the user gives the permission to do that.

Also, we should probably recommend that UAs automatically expire
session cookies after some time (maybe the same delay limit that
is used before emitting a warning). I've dealt with issues where
some smartphone browsers never close and never delete session
cookies, causing the same browser to always go to the same server
past the load balancer. In the end, I had the LB detect those
cookies to fix them!

Regards,
Willy

---

From: Sam Pullara <[email-masked]>
Date: Sat, 13 Jul 2013 23:22:12 -0700

As long as there is someway for users to keep themselves logged into a site, sounds good to me.  Sam   

---Sent from Boxer | http://getboxer.com

cookies to fix them!

Regards,

Willy

---

From: Nicolas Mailhot <[email-masked]>
Date: Sun, 14 Jul 2013 12:34:47 +0000 (UTC)

> access. You simply can't do that by learning massive amounts of randoms

Then have the ID set by the server in answer to the first client request
(and limit its size in the protocol so no one tries to push a cookie there)

-- 
Nicolas Mailhot

---

From: Nicolas Mailhot <[email-masked]>
Date: Sun, 14 Jul 2013 12:41:42 +0000 (UTC)

Yoav Nir <ynir@...> writes:

> There will come a day when we can say that HTTP/1.1 is only for legacy
applications, and that it's fine to make
> sweeping changes only in HTTP/2.0 (or perhaps introduce them in HTTP/3.0).
That day is not today, and if
> we're going to fix it, we might as well fix it in 1.0 as well.
> 
> Your proposal
(http://lists.w3.org/Archives/Public/ietf-http-wg/2013JulSep/0284.html)
> includes prohibiting cookies on HTTP/2.0. This directly conflicts with
this working group's goal of
> creating a protocol that is a drop-in replacement for HTTP/1.1. 

There is zero need to strictly forbid cookies in HTTP/2.0. As long as
HTTP/2.0 provides a better replacement, cookies can still exist in the
protocol as deprecated feature, and social pressures (like the EU decision)
will effectively kill them in time. Lots of bad ideas on the web have been
killed this way without needing spec removal.

If the replacement is good, and if no one tries to mitigate cookie pain in
http/2.0 (via compression for example), cookies will disappear by
themselves. Starting in cookie blocker browser extensions (that will only
allow the new method).

-- 
Nicolas Mailhot

---

From: Nicolas Mailhot <[email-masked]>
Date: Sun, 14 Jul 2013 13:15:38 +0000 (UTC)

> sent anyway (or we need at least one bit to indicate its presence).

How about this:

1. servers mark the parts of a web site that need a routing label (could be
some form of url extension, for example @ without login or @@ in presence of
login)

2. they propose a new routing label to clients that connect to such a part
without one

3. the client then answers either with this label or the one it used
previously (if it remembers it)

4. the client can refuse to reuse a label already allocated to another site
(not same origin or different certificate, perhaps, policy is client-side)

5. there are strong protections against routing label abuse :
– lifetime limited to two hours since last exchange (enough to handle most
interruptions and lunch breaks)
– lifetime limited to 12 hours since set up (a big work day)
— size limited so you can not stuff other things in there

-- 
Nicolas Mailhot

---

From: "Martin Nilsson" <[email-masked]>
Date: Mon, 15 Jul 2013 00:32:50 +0200

On Fri, 12 Jul 2013 14:59:30 +0200, Patrick McManus <[email-masked]>  
wrote:

> between transactions. Especially in >proxying scenarios, but other  
> places too.
>
> And even if we removed those, a request header would still be ~300 bytes  
> - which is just too big to support the >many dozens of subresources that  

Taking a snapshot of 250k requests from mobile devices gives an average of  
567 bytes per request. This is for short URLs and no cookies (except those  
injected by operators) are included.

    0 : 18.9%
  100 :  1.6%
  200 :  7.7%
  300 :  9.8%
  400 :  8.4%
  500 : 11.1%
  600 : 11.4%
  700 :  9.5%
  800 :  7.7%
  900 :  4.7%
1000 :  2.4%

Almost all of these are however request invariant, so just setting these  
on a connection level solves most of the problem without compression or  
compact encoding.

/Martin Nilsson

---

From: Nico Williams <[email-masked]>
Date: Wed, 17 Jul 2013 19:05:18 -0500

>>How sure are we that the entire idea of header compression isn't a bad

Me three.

> indexed by a client provided session-id.

As for [encrypted] session state cookies, there is a trade-off: state
on the server vs. not.

Some state kept by the application on the server side can be cached,
using the client to hold the [encrypted] sub-state cookies and re-send
as necessary (e.g., if the server pushes the client's state out of its
cache).  Obviously this doesn't apply to long-term state (e.g., files
stored on a cloud), just session metadata, nor does it apply to
frequently-changing session state: that just consumes bandwidth --
much more than it would consume memory to just keep around.  But much
session state needs to be able to be pushed out of the server's cache
(e.g., not shopping carts in some apps, but yes in others).

The right balance, IMO, is for the server to assign as small a session
ID as it can (it can be smaller if a "connection" can be used as part
of a lookup index, else it has to be bigger).  Traditionally HTTP/1.x
services haven't really been able to associate state with
"connections", and I find it odd that we'd try to do it in HTTP/2.0.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Fri, 19 Jul 2013 11:54:56 -0500

> I have given a concrete example multiple times, it's very simple:

So you think all session state should always be stored on the server, period?

It's hard to disagree, but I was under the impression that many
services need to be stateless (storing session state in encrypted
cookies) for various reasons.

The assumption being made by the WG that per-connection compression
state is no biggie bothers me.  I think it's a huge deal, but it does
go hand in hand with an assumption that all session state is to be
stored by the server, which makes sense to me.

Is there consensus for such assumptions?

Nico
--

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 19 Jul 2013 17:22:14 +0000

In message <[email-masked]>
>cookies) for various reasons.

In the post-EU-regulation, post-PRISM-world, "various reasons" need
to be "Very Good Reasons" for this practice to continue.

---

From: Nico Williams <[email-masked]>
Date: Fri, 19 Jul 2013 13:06:32 -0500

> to be "Very Good Reasons" for this practice to continue.

I'm not sure how any session identifier would survive silly
anti-cookie regulations from the EU.  A session ID is still a cookie.

I don't see how PRISM affects this either.  If anything, keeping
session state on the server... only helps PRISM: more data to chomp
on.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Fri, 19 Jul 2013 18:19:27 +0000

In message <[email-masked]>

>anti-cookie regulations from the EU.  A session ID is still a cookie.

Read the actual regulation ?

>session state on the server... only helps PRISM: more data to chomp on.

It means that any random computer I use to access a given service is
not polluted with bit-droppings saying I did so.

---

From: Nico Williams <[email-masked]>
Date: Fri, 19 Jul 2013 13:32:59 -0500

> Read the actual regulation ?

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:32002L0058:en:NOT

?

My point stands.  How is a *any* session identifier different from a
cookie?  The only differences might relate to how they might leak to
third parties.

> not polluted with bit-droppings saying I did so.

How do you know when you're done using it that it's not still holding
on to your at-one-time open sessions?  Here there is a somewhat useful
answer: you could ask the service [from another device that you do
trust] to close those sessions.

Session logout is an important feature to have (though in all cases we
have to trust the server).