Subject: [hybi] I-D Action: draft-ietf-hybi-permessage-compression-01.txt

From: [email-masked]
Date: Mon, 01 Oct 2012 04:25:13 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories.
 This draft is a work item of the BiDirectional or Server-Initiated HTTP Working Group of the IETF.

	Title           : WebSocket Per-message Compression
	Author(s)       : Takeshi Yoshino
	Filename        : draft-ietf-hybi-permessage-compression-01.txt
	Pages           : 18
	Date            : 2012-10-01

Abstract:
   This specification defines a WebSocket extension that adds
   compression functionality to the WebSocket Protocol.  It compresses
   the payload of non-control WebSocket messages using specified
   compression algorithm.  One reserved bit RSV1 in the WebSocket frame
   header is allocated to control application of compression for each
   message.  This specification provides one compression method
   available for the extension using DEFLATE.

   Please send feedback to the [email-masked] mailing list.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-hybi-permessage-compression

There's also a htmlized version available at:
http://tools.ietf.org/html/draft-ietf-hybi-permessage-compression-01

A diff from the previous version is available at:
http://www.ietf.org/rfcdiff?url2=draft-ietf-hybi-permessage-compression-01

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: Takeshi Yoshino <[email-masked]>
Date: Mon, 1 Oct 2012 20:54:20 +0900

No semantic change.

- Clarified how the next block should be appended to a block with BFINAL=1
- Added texts to examples to explain what data each of the bytes contains

---

From: Joakim Erdfelt <[email-masked]>
Date: Fri, 12 Oct 2012 10:50:04 -0700

About the Examples section ...

Can there be some wording about the current set of examples that they
represent just a single frame's payload data? (or something similar)

Also since we are now dealing with permessage, would it be useful to see
Examples showing compression across multiple frames?

And with that in mind, is there any relationship between frames and blocks
of compressed data?
For example, what would the spec say with regards to a single block of
deflate compressed data arriving via 2+ frames? (hope that's allowed)

--
Joakim Erdfelt <[email-masked]>
www.webtide.com
Developer advice, services and support
from the Jetty & CometD experts.

On Mon, Oct 1, 2012 at 4:54 AM, Takeshi Yoshino <[email-masked]> wrote:

> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi
>
>

---

From: Takeshi Yoshino <[email-masked]>
Date: Sat, 13 Oct 2012 03:28:06 +0900

> represent just a single frame's payload data? (or something similar)

Yes. Only octets to be set in payload data portion is printed. Each of them
(each line starting with * symbol) represents one of a single frame.

> Examples showing compression across multiple frames?
OK. I'll add some examples with fragmentation.

> deflate compressed data arriving via 2+ frames? (hope that's allowed)

Right. Now fragmentation is transparent to compression. Compressed message
payload can be cut at any positions into 2 or more frames. It's fine to cut
in the middle of a deflate block.

> On Mon, Oct 1, 2012 at 4:54 AM, Takeshi Yoshino <[email-masked]>wrote:
>
>> No semantic change.
>>

---

From: Joakim Erdfelt <[email-masked]>
Date: Fri, 12 Oct 2012 12:39:59 -0700

(I'm working on the support for this extension in the Eclipse Jetty Web
Server codebase)

Just a sanity check ...

Are the existing Examples sane / valid?

Using the built-in Deflate algorithms in Java, in the first example that
specifies the use of the LZ77 sliding window, java reports the second block
as "invalid stored block lengths".
( this behavior is indicative that we are getting a Z_DATA_ERROR somewhere
in the native code.  Our initial investigation shows line #159 here as a
code path we are in ...
http://cr.openjdk.java.net/~sherman/7188852/webrev/src/share/native/java/util/zip/Inflater.c-.html
 )

Also, I've been having much heartburn over some of the Deflate specifics
laid out in spec wrt java's native Deflate implementation.
Namely:

* Java always produces a BFINAL 1 block (looking at the openjdk source, I
can see no way for BFINAL 0 to ever be produced).
   Already filed a bug with chromium folks about this (as chromium fails to
decompress if server generates a block using BFINAL 1) ...
     https://code.google.com/p/chromium/issues/detail?id=155060
* The various method parameters .. "s2c_max_window_bits",
"c2s_max_window_bits", "s2c_no_context_takeover", "c2s_no_context_takeover"
are all seemingly impossible to control with the native Java Deflate
implementations.

Also, I'd like to discuss the last 2 paragraphs on section 5.1 indicating
that the client "MUST fail" the websocket connection if these method
configuration parameters are not supported.  Can we get this relaxed to
"MAY fail" instead?  Allowing the endpoints to negotiate capabilities
instead of failing?

As the spec stands now, in order for someone on java to support the
compression, they'll have to use a 100% java (non-native) Deflate/ZLib
implementation to support these parts of the spec.   For performance
reasons, this is troublesome to me, as the native implementations use zlib
currently, are blazingly fast, work on multiple devices (even mobile
phones), and are mature and well debugged.

The native Deflate/Zlib support in java works fine for HTTP w/deflate (and
even gzip) Content-Encoding, its a shame that we'll have to either
implement Deflate ourselves, or pull in a 3rd party Deflate algorithm (as
we have to be careful about Eclipse IP due diligence and cleanliness
requirements).

Thanks for listening.

On Fri, Oct 12, 2012 at 11:28 AM, Takeshi Yoshino <[email-masked]>wrote:

> On Sat, Oct 13, 2012 at 2:50 AM, Joakim Erdfelt <[email-masked]>wrote:
>
> Yes. Only octets to be set in payload data portion is printed. Each of
> them (each line starting with * symbol) represents one of a single frame.
>
>
>>
>> And with that in mind, is there any relationship between frames and
>> blocks of compressed data?

---

From: Joakim Erdfelt <[email-masked]>
Date: Fri, 12 Oct 2012 13:07:37 -0700

Update.

The 2 blocks in Example 1 can be handled by Java's Deflate but the
delimiters for the block of data have to be known ahead of time.
If that example arrives as a single frame of data seen like ...

  0xf2 0x48 0xcd 0xc9 0xc9 0x07 0x00 0xf2 0x00 0x11 0x00 0x00

Then this is invalid to Java's Deflate implementation.
However, if the first block is decoded, followed the octets [0x00 0x00 0xFF
0xFF] (empty block with no compression?) are provided to the inflate
algorithm, then the second block is valid to Java's native implementation
of Deflate, at which point the 2 "Hello" texts can be decoded correctly.

So this leads me to believe that the spec should have some language that
the start of a block can never appear in the middle of a single compressed
message (regardless of number of frames) to support this.  Or is there some
language for an implied delimiter of blocks that isn't clear in the spec
right now?

On Fri, Oct 12, 2012 at 12:39 PM, Joakim Erdfelt <[email-masked]> wrote:

>    Already filed a bug with chromium folks about this (as chromium fails
> to decompress if server generates a block using BFINAL 1) ...
>> Right. Now fragmentation is transparent to compression. Compressed
>> message payload can be cut at any positions into 2 or more frames. It's
>> fine to cut in the middle of a deflate block.
>>
>>
>>>
>>> --

---

From: Takeshi Yoshino <[email-masked]>
Date: Sat, 13 Oct 2012 13:26:01 +0900

> as "invalid stored block lengths".

Have you added 0x00 0x00 0xff 0xff to examples before pouring them into
Inflater?

>  ( this behavior is indicative that we are getting a Z_DATA_ERROR
> somewhere in the native code.  Our initial investigation shows line #159
> here as a code path we are in ...

(with Chromium hat) We'll fix it.

That's the reason why I've added BFINAL=1 options. I skimmed HyBi log and
saw that Greg was talking about that limitation of Java.

> are all seemingly impossible to control with the native Java Deflate

Please just reset your Deflate instance for each message if you received
s2c_no_context_takeover.

About s2c_max_window_bits, yes, you're right. I forgot to investigate if
this is available on JDK.

> "MAY fail" instead?  Allowing the endpoints to negotiate capabilities
It causes problem I think. The server has already declared that it can
accept only data compressed using that window size. Rather than changing
the MUST to MAY, I think we should let the client to announce it's
capability by including c2s_max_window_bits in it's method description.
I.e. if there's not c2s_max_window_bits token in request, the server MUST
NOT request c2s_max_window_bits. JDK based client may exclude
c2s_max_window_bits.

We also need to relax is the last sentence of the paragraph defining
s2c_max_window_bits.

If the
"accepted request" has this method parameter, the server MUST
attach this method parameter with the same value as one of the
"accepted request".

Changing this "MUST" to "SHOULD" allows servers to reject window size
reduction request.

Thanks

---

From: Takeshi Yoshino <[email-masked]>
Date: Sat, 13 Oct 2012 13:38:32 +0900

> If that example arrives as a single frame of data seen like ...

The first example in -01 is not two frame example. It's an example of two
messages case (two consequent messages compressed using the same context).

I'll add two frame (fragment) example in the next version.

---

From: Joakim Erdfelt <[email-masked]>
Date: Fri, 12 Oct 2012 21:51:31 -0700

Ah. subtle.

(heh. funny that 4 of us on our side missed that nuance.)

I didn't have access to Greg for this (our seasoned spec whisperer), and
could have used his interpretations of that bullet point. *sigh*

Let me use this as suggestion to tweak / reword that first bullet point to
be clear about it being 2 separate messages, 2 separate payloads, etc.

Thanks again,

On Fri, Oct 12, 2012 at 9:38 PM, Takeshi Yoshino <[email-masked]>wrote:

> On Sat, Oct 13, 2012 at 5:07 AM, Joakim Erdfelt <[email-masked]>wrote:
>
>> Update.
>>