Subject: Re: chunk-extensions

From: "Roy T. Fielding" <[email-masked]>
Date: Fri, 13 Sep 2013 15:01:31 -0700

On Oct 21, 2010, at 7:29 PM, Mark Nottingham wrote:

> What do people think about deprecating the use of chunk-extensions -- i.e., requiring that they be consumed, but not produced?
> 
>     chunk          = chunk-size *WSP [ chunk-ext ] CRLF
>                      chunk-data CRLF
>     chunk-size     = 1*HEXDIG
>     last-chunk     = 1*("0") *WSP [ chunk-ext ] CRLF
> 
>     chunk-ext      = *( ";" *WSP chunk-ext-name
>                         [ "=" chunk-ext-val ] *WSP )
>     chunk-ext-name = token
>     chunk-ext-val  = token / quoted-str-nf
> 
> I haven't seen any practical uses for them, and haven't seen any implementations that do anything with them other than throw them away. 

http://trac.tools.ietf.org/wg/httpbis/trac/ticket/343

I know this was closed a long time ago, but I have never been
comfortable with the idea of deprecating chunk-ext.  They were
originally intended to allow per-chunk signatures or hashes,
similar to those used in the Wave protocol.  More recently, they
have been considered as a potential solution to BREACH-style
attacks, specifically when a compression transfer encoding is
used within a TLS-secured connection. [There are other solutions,
of course, like changing the compression algorithms or changing
TLS, but chunk-ext is easier to deploy on a per-hop basis.] 

I am concerned that we might be deprecating something that will
be needed if we get widespread deployment of transfer-encoding
compression (instead of content-encoding compression).  Moreover,
recipients don't actually gain anything from this deprecation,
since they have to continue parsing for chunk-ext to remain
compatible.

Does anyone else share that concern?  It would be different if
we knew of systems that actually break on receipt of chunk-ext,
but I am not aware of any.

Cheers,

Roy T. Fielding                     <http://roy.gbiv.com/>
Senior Principal Scientist, Adobe   <https://www.adobe.com/>

---

From: Daniel Stenberg <[email-masked]>
Date: Sat, 14 Sep 2013 00:08:25 +0200 (CEST)

On Fri, 13 Sep 2013, Roy T. Fielding wrote:

> I know this was closed a long time ago, but I have never been comfortable 
> with the idea of deprecating chunk-ext.  They were originally intended to 
> allow per-chunk signatures or hashes, similar to those used in the Wave 
> protocol.

We've had contributors discussing adding support for chunk extensions to 
libcurl just a few months ago[1]. It didn't happen (yet) but at least it shows 
there's some weirdos out there considering using this!

[1] = http://curl.haxx.se/mail/lib-2013-08/0232.html

-- 

  / daniel.haxx.se

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 14 Sep 2013 09:54:25 +1000

I've considered using them for a few things over the years. However, two things always stopped me; they aren't accommodated by Apis, and they aren't guaranteed to transit a hop. 

For the breach attacks, I don't think deprecating them harms things, since you can still sen them; in these mitigations, the payload / semantics don't matter, as long as something is there. 

Personally, I'm ok either way; the important thing is to document their behavior / limitations. Deprecation I one way to do that, but we could do it in prose too. 

From a chair perspective - we need to wrap this up. If we can get clean consensus to change quickly, great. Otherwise, lets leave it be. 

Sent from my iPhone

---

From: Julian Reschke <[email-masked]>
Date: Sat, 14 Sep 2013 17:56:42 +0200

> For the breach attacks, I don't think deprecating them harms things, since you can still sen them; in these mitigations, the payload / semantics don't matter, as long as something is there.

You can send them, but you'd violate a "SHOULD NOT".

If we believe using them is ok (and you made it sound like that), we 
consequently should back out the change to deprecate it 
(<http://trac.tools.ietf.org/wg/httpbis/trac/changeset/1535#file1>)

> ...

What exactly does not to be documented?

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Sun, 15 Sep 2013 16:01:01 +1000

On 15/09/2013, at 1:56 AM, Julian Reschke <[email-masked]> wrote:

> If we believe using them is ok (and you made it sound like that), we consequently should back out the change to deprecate it (<http://trac.tools.ietf.org/wg/httpbis/trac/changeset/1535#file1>)
> 

Assuming s/not/need/

Document that they may not be persisted beyond a, because chunking (and therefore extensions) don't have any semantic in the message itself. Furthermore, that they're not available in most implementations.

--
Mark Nottingham   http://www.mnot.net/

---

From: Mark Nottingham <[email-masked]>
Date: Sun, 15 Sep 2013 16:21:40 +1000

Sounds like a plan.

On 15/09/2013, at 4:21 PM, Willy Tarreau <[email-masked]> wrote:

> On Sun, Sep 15, 2013 at 04:01:01PM +1000, Mark Nottingham wrote:
> Probably it would be easier to remind that just like chunks themselves,
> they're connection-specific, since any intermediary is allowed to
> rechunk differently. It is also obvious that a compressing gateway
> will rechunk for example.
> 
> But I agree with Roy that it would be too bad to get rid of something
> that most implementations can already parse (even if they don't use
> them) and will continue to parse whatever we write in the spec,
> especially if there is some potential for using them in the future.
> 
> Willy
>

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 15 Sep 2013 08:21:08 +0200

Willy

---

From: Julian Reschke <[email-masked]>
Date: Sun, 15 Sep 2013 10:44:32 +0200

> will rechunk for example.

Can you suggest concrete text?

> especially if there is some potential for using them in the future.

Agreed.

Best regards, Julian

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 15 Sep 2013 11:23:24 +0200

On Sun, Sep 15, 2013 at 10:44:32AM +0200, Julian Reschke wrote:
> >>Document that they may not be persisted beyond a, because chunking (and 
> >>therefore extensions) don't have any semantic in the message itself. 
> >>Furthermore, that they're not available in most implementations.
> >

Just doing a quick attempt :

@p1-23#4.1
-   Chunk extensions within the chunked transfer coding are deprecated.
-   Senders SHOULD NOT send chunk-ext.  Definition of new chunk
-   extensions is discouraged.
+   The current version of the HTTP specification defines no specific
+   use for the chunk extensions. When use of chunk extensions is
+   considered to transport anything (for example, chunks signatures),
+   care must be taken about the nature of the information present in
+   these extensions which by definition are hop-by-hop and not
+   end-to-end, as nothing prevents an intermediary from forwarding a
+   message with different size chunks, or even dropping or replacing
+   these extensions as well.

Willy

---

From: Julian Reschke <[email-masked]>
Date: Sun, 15 Sep 2013 14:41:01 +0200

> +   these extensions as well.

OK, change proposal:

1673,1675c1673,1677
<  Chunk extensions within the chunked transfer coding are deprecated.
<  A sender SHOULD NOT send chunk-ext.  Definition of new chunk
<  extensions is discouraged.
---
 >  Chunk extensions need to be used carefully because use of chunking
 >  does not have any semantics related to the message; it is by
 >  definition hop-by-hop and thus can be removed/reformatted by any
 >  intermediate.  Furthermore, few implementations expose information
 >  about their contents.

3524,3525c3526,3527
<  header and trailer.  Use of chunk extensions is deprecated, and line
<  folding in them is disallowed.  (Section 4.1)
---
 >  header and trailer.  Line folding in chunk extensions is disallowed.
 >  (Section 4.1)

(<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/343/343.2.diff>)

Best regards, Julian

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 15 Sep 2013 15:01:06 +0200

> (<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/343/343.2.diff>)

OK I'm fine with this proposal.

Thanks Julian,
Willy

---

From: David Morris <[email-masked]>
Date: Sun, 15 Sep 2013 09:42:13 -0700 (PDT)

I think the correct word is "intermediary" so the new section would
be:
..
   intermediary.  Furthermore, few implementations expose information
   about their contents.

- Dave Morris

---

From: Willy Tarreau <[email-masked]>
Date: Sun, 15 Sep 2013 20:14:14 +0200

>    intermediary.  Furthermore, few implementations expose information

That's true, I didn't notice :-)

Willy

---

From: Julian Reschke <[email-masked]>
Date: Sun, 15 Sep 2013 20:21:39 +0200

> That's true, I didn't notice :-)

That's the nice thing about review-then-commit :-)

---

From: Julian Reschke <[email-masked]>
Date: Tue, 17 Sep 2013 08:44:23 +0200

>     intermediary.  Furthermore, few implementations expose information

...applied with <http://trac.tools.ietf.org/wg/httpbis/trac/changeset/2405>.

Best regards, Julian