Subject: RE: [rohc] SigComp user guide

From: "Price, Richard" <[email-masked]>
Date: Fri, 9 Aug 2002 14:14:35 +0100

> Yes, it works that way, but I think it's rather confusing 
> that you use a different bit order for the distance code and 
> the extrabits.

Hi,

Using a different bit order for the distance code and the extra
bits is certainly confusing - but this is how standard DEFLATE
(RFC 1951) works. If we want to decompress messages from an
ordinary DEFLATE compressor then we don't have any choice
but to use this bit ordering.

When we're customising a version of DEFLATE for SigComp
however, it's well worth modifying the algorithm to using a simpler
bit ordering because the resulting bytecode is smaller and faster.

> I tried FHP=001, and it works fine. May the FHP=101 be
> also a remain of the other code? If not, what is the point using it?

FHP=101 is required to decompress messages generated by a
standard DEFLATE compressor.  Setting FHP=001 shouldn't work -
we tried this on the example compressed message in the User
Guide and it didn't decompress properly.

> Can you share us the exact way of your modified DEFLATE, so 
> that I and maybe others as well can prepare the DEFLATE 
> compressor to do it? Thank you!

I've made quite a few modifications to the standard DEFLATE
compressor which increases its efficiency in the SigComp environment,
both by improving the compression ratio and by reducing the size of
the DEFLATE bytecode. Some of the improvements that can be made
are as follows:

*   The bit order of the "extra bits" can be changed to be the same as
    the bit order of the Huffman codes (for both the distance codes and
    the length/literals). This optimisation allows the entire code
    including the extra bits to be decompressed using a single
    instance of the INPUT-HUFFMAN instruction, which reduces the
    overall size of the DEFLATE bytecode.

*   Since the code tables found in DEFLATE are designed for
    generic data (and hence not particularly efficient), they can be
    replaced with code tables optimised for better compression
    of SIP messages.

*   It's possible to upload more than one code table to the
    decompressor and then switch between them on the fly, depending
    on the type of data that needs to be compressed. This can provide
    a substantial performance benefit over "standard" DEFLATE.

If you'd like to see how we've modified the DEFLATE compressor in
more detail then I can send you a copy of our code - please let me
know if this would be useful.

Regards,

Richard

> Regards,
> 
> Zacco
> 
> -----Original Message-----
> From: Price, Richard [mailto:[email-masked]]
> Sent: Friday, August 02, 2002 4:54 PM
> To: 'Lajos Zaccomer (ETH)'; [email-masked]
> Subject: RE: [rohc] SigComp usr guide - DEFLATE data mistake
> 
> 
> Hi,
> 
> Many thanks for pointing out this mistake in the SigComp User Guide.
> However the problem doesn't lie in the compressed message but rather
> the bytecode for DEFLATE.
> 
> When writing the draft I noticed that the size of the DEFLATE bytecode
> could be reduced by using a different bit ordering from the one specified
> in RFC 1951.  This might be a useful enhancement to make when
> deploying SigComp as the bit ordering is chosen by the compressor, so it
> doesn't necessarily have to be the one used by standard DEFLATE
> implementations.
> 
> I changed most of the bytecode back to the bitwise-compatible
> version before publication (so that the bytecode could be tested in
> conjunction with a standard DEFLATE compressor), but I forgot about
> the following lines:
> 
>    INPUT-BITS (5, index, fail)
>    ADD ($index, distance_table_start)
> 
> The less efficient but bitwise-compatible version is given below:
> 
>    INPUT-HUFFMAN (index, fail, 1, 5, 0, 31, distance_table_start)
> 
> Making this change should allow the example message to be successfully
> decompressed.
> 
> Regards,
> 
> Richard
> 
> > -----Original Message-----
> > From: Lajos Zaccomer (ETH) [mailto:[email-masked]]
> > Sent: Friday, August 02, 2002 10:04 AM
> > To: [email-masked]
> > Subject: [rohc] SigComp usr guide - DEFLATE data mistake
> > 
> > 
> > Hi,
> > 
> > I found a mistake in the SigComp user guide DEFLATE example 
> > bit string. The "E" (in "Life, the Universe and Everithing") 
> > is followed by a <length, distance> pair. As FHP=101, the 
> > bits are provided in LSB->MSB order, and the first received 
> > bit is interpreted as LSB. Have a look at the 26th octet 
> > (0x32 = 00110010b) in that bits 6-2 (01100b = 12) represents 
> > the distance code. The belonging range is 65-96 instead of 11 
> > (the pointer should reference the "ver" substring).
> > If you reverse the bit order of these 5 bits, the new result 
> > is 6, and the belonging range is 9-12. So, correct the 26th 
> > octet from 0x32 to 0x1a to have the proper result!
> > 
> > Regards,
> > 
> > Zacco
> > 
> > _______________________________________________
> > Rohc mailing list
> > [email-masked]
> > https://www1.ietf.org/mailman/listinfo/rohc
> > 
> 
> _______________________________________________
> Rohc mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/rohc
>

---

From: "Lajos Zaccomer (ETH)" <[email-masked]>
Date: Sat, 10 Aug 2002 10:29:31 +0200

Hi,

>Guide and it didn't decompress properly.

I didn't say I had used the same compression algorithm. I changed the bit order. So it is also a customized one, but if DEFLATE requires the other order and as I noted there is no compression or bytecode length gain, it is out of question.

Thank you for the details, and of course I'm interested in the code. In return, I can probably provide you with LZJH compressed Douglas Adams quotes by this evening if you require.

Best regards,

Zacco

-----Original Message-----
From: Price, Richard [mailto:[email-masked]]
Sent: Friday, August 09, 2002 3:15 PM
To: 'Lajos Zaccomer (ETH)'; [email-masked]'
Subject: RE: [rohc] SigComp user guide

Hi,

Regards,

Richard