Subject: #466 segment compression

From: Matthew Kerwin <[email-masked]>
Date: Tue, 29 Apr 2014 16:26:12 +1000

I just saw issue #466 and couldn't find a discussion on the list, so I'm
starting a new one.  The issue says:

> 2. Compression effectively creates a delineation similar to what
> segments allow; aligning the two would avoid issues with reframing,
> padding and other related concerns.

I don't understand this point, but that's probably because I don't
understand segments. AFAICT they were originally a way of ensuring that
certain headers are interleaved with data at specific boundaries, which can
be used as a sort of covert sub-resource scheme if the sender and receiver
both understand the significance of the segmentation.

I presume there's something in there that says that only the resource can
set END_SEGMENT bits -- but I can't find it on a quick skim. If not, that's
potentially a whole other issue.

Per-frame compression, as I envisioned it, doesn't create any delineation
at all; at least, no more than existing frame boundaries already do. A
reluctant intermediary might be more likely to retransmit compressed frames
than to unzip+reframe, I suppose...

In the simplest case, compression lets you reduce the size of your frames;
more optimally it can be used to reduce the number of frames (by reframing
after compression); taking it a step further, the optimally-packed
compressed frames can be padded out to mask the nature of their content.
All three operations (determining frame length, compressing frames, and
padding) are hop-by-hop operations, and any intermediary can second-guess
or obstinately undo the previous hop's choices if it wants.

If the original resource really wants to ensure that two chunks of data
don't ever share a compression context, it puts them in separate frames
with an END_SEGMENT in between. This delineates the compression contexts,
completely different from a delineation resulting from the compression.

Why add complexity and overhead (rules that govern how frames are marked,
machinery to ensure those rules are followed, long-lived compression
contexts, bookkeeping for those contexts, etc.) just to conflate two
concepts that were hitherto unrelated?

-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: Matthew Kerwin <[email-masked]>
Date: Tue, 29 Apr 2014 16:52:43 +1000

Sorry for replying to myself.

> or obstinately undo the previous hop's choices if it wants.
​I just noticed the ​bit in 10.6 that says that intermediaries can't add
compression. Is that the delineation to which you referred in #466 Martin?
 That text introduces complexity I hadn't imagined when proposing per-frame
compression. I'll think about it some more.

---

From: <[email-masked]>
Date: Tue, 29 Apr 2014 08:41:59 +0000

I think the simplest idea is to require the origin to separate attacker-controlled data and private/confidential data by segments.  All of the rest of the intermediary problems go away since an intermediary already can’t re-frame across segment boundaries, plus it allows an intermediary to do whatever it wants within a segment (compress/uncompress/concatenate/de-compress+re-compress).

The thing I like about Martin’s idea to switch to one compression context per segment is that it provides greater compatibility between HTTP/1.x T-E and HTTP/2 transfer codings.  Per-frame compression as we have it now is compatible with T-E when going from 2 -> 1.x, but it’s not compatible from 1.x -> 2.  The added complexity and the “vast increase in DoS surface area” [1] is a big deal though.

[1] https://github.com/http2/http2-spec/issues/466#issuecomment-41361711

--
Keith S. Morgan
Remote Monitoring Unit
Safeguards, International Atomic Energy Agency (IAEA)
Vienna, Austria
Office: +43 1 2600 26672
Handy: +43 699 165 26672

From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Tuesday,29 April 2014 08:53
To: [email-masked]
Cc: Martin Thomson
Subject: Re: #466 segment compression

On 29 April 2014 16:26, Matthew Kerwin <[email-masked]<mailto:[email-masked]>> wrote:
All three operations (determining frame length, compressing frames, and padding) are hop-by-hop operations, and any intermediary can second-guess or obstinately undo the previous hop's choices if it wants.

​I just noticed the ​bit in 10.6 that says that intermediaries can't add compression. Is that the delineation to which you referred in #466 Martin?  That text introduces complexity I hadn't imagined when proposing per-frame compression. I'll think about it some more.

This email message is intended only for the use of the named recipient. Information contained in this email message and its attachments may be privileged, confidential and protected from disclosure. If you are not the intended recipient, please do not read, copy, use or disclose this communication to others. Also please notify the sender by replying to this message and then delete it from your system.

---

From: Roberto Peon <[email-masked]>
Date: Tue, 29 Apr 2014 12:57:54 -0700

If compression is per-segment as opposed to per-frame
   AND we wish to restrict the amount of state the server must keep around
   AND the sender wishes to have all the data in the segment compressed
  THEN then the segment being compressed must be sent in its entirety in a
contiguous series of frames with no muxing of other streams in the middle.

If one doesn't do this, then the server is forced to retain this state
indefinitely, though the client is allowed to make progress on other
streams. This likely masks the consumption of server-state from the user,
and also denies the user(and client) the ability to do compression for any
other stream.

If one does this, then one effectively loses the multiplexing feature of
the protocol.

Arguably, this is worse than simply doing per-frame compression, especially
when on considers that with relatively decent frame sizes, the marginal
benefit of continuing the compression context across frames is likely small.

Now, lets assume the part above is wrong because I've misinterpreted
something. :)

I'm still worried: There is an assumption here that the originator of the
stream actually knows the provenance of the various data within. This is
often a hard problem especially given that servers and services are not
currently architected to keep track of provenance. IMHO, if a segment/frame
wasn't compressed it should never be compressed. That leads to safety.

If the proposal is simply that if any frame in a segment is compressed all
of something in a segment can be compressed, sure. I'm on board with that.

-=R

On Tue, Apr 29, 2014 at 1:41 AM, <[email-masked]> wrote:

>  I think the simplest idea is to require the origin to separate
> attacker-controlled data and private/confidential data by segments.  All of
> the rest of the intermediary problems go away since an intermediary already
> can’t re-frame across segment boundaries, plus it allows an intermediary to
> do whatever it wants within a segment
> (compress/uncompress/concatenate/de-compress+re-compress).
>
>
>
> The thing I like about Martin’s idea to switch to one compression context
> per segment is that it provides greater compatibility between HTTP/1.x T-E
> and HTTP/2 transfer codings.  Per-frame compression as we have it now is
> compatible with T-E when going from 2 -> 1.x, but it’s not compatible from
> 1.x -> 2.  The added complexity and the “vast increase in DoS surface area”
> [1] is a big deal though.
>
>
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of *
> [email-masked]
> *Sent:* Tuesday,29 April 2014 08:53
> *To:* [email-masked]
> *Cc:* Martin Thomson
> *Subject:* Re: #466 segment compression
>
>
>
> This email message is intended only for the use of the named recipient.
> Information contained in this email message and its attachments may be
> privileged, confidential and protected from disclosure. If you are not the
> intended recipient, please do not read, copy, use or disclose this
> communication to others. Also please notify the sender by replying to this
> message and then delete it from your system.
>