Subject: [hybi] Permessage-deflate final frame

From: "Arman Djusupov" <[email-masked]>
Date: Fri, 10 Jan 2014 10:40:12 +0200

Oops, apologies for inadvertently hijacking the thread, I am resending under a new topic. Arman

Hello Takeshi,

In the current draft, the description of the compression process states :

   1.  Compress all the octets of the payload of the message using
       DEFLATE.

This doesn’t take into account outbound messages of arbitrary size that NEED to be fragmented, for which it is more favorable to use fragmentation and to compress & send fragments until the end of the message is reached.

In such cases the implementation buffers the input from its source and then flushes it out into a compressed fragment, repeating this process until the end of the source data is reached. At that point it flushes any remaining bytes buffered into a final frame, or flushes a final frame of 0 length.

When the implementation produces compressed fragments, it periodically produces frames with 0x00 0x00 0xFF 0xFF at the end of the frame due to flushing DEFLATE. But because of the following requirements:

   2. If the resulting data does not end with an empty DEFLATE block
       with no compression (the "BTYPE" bits is set to 00), append an
       empty DEFLATE block with no compression to the tail end.

   3.  Remove 4 octets (that are 0x00 0x00 0xff 0xff) from the tail end.
       After this step, the last octet of the compressed data contains
       (possibly part of) the DEFLATE header bits with the "BTYPE" bits
       set to 00.

the implementation must ensure that the message ends empty blocks without a trailing 0x0000FFFF, so the implementation must keep track on what type of block was at the end of the frame that was sent last. If the last frame sent ends with 0x0000FFFF then the implementation cannot remove those 4 bytes from the wire, but it must artificially produce an empty DEFLATE block to send as the final frame. This is only due to the requirement to remove 0x0000FFFF.

Wouldn’t it be easier to make the removal 0x00 0x00 0xFF 0xFF optional, while at the same time requiring that the receiving side appends a 0x00 0x00 0xFF 0xFF to the final frame in case when its missing?

It is much simpler to just check the final bytes of the message instead of having to remember the state of the previous frame.

With best regards,
Arman

---

From: Takeshi Yoshino <[email-masked]>
Date: Tue, 21 Jan 2014 16:48:04 +0900

I posted a reply to your question here.
http://www.ietf.org/mail-archive/web/hybi/current/msg10286.html
Just in case. Thanks.