Subject: [lemonade] COMPRESS Thoughts

From: "Burger, Eric" <[email-masked]>
Date: Wed, 31 May 2006 15:29:26 -0400

(1) Going back to COMPRESS in its own document (not CONVERT)

(2) Consensus that compression is directional, i.e., downstream
compressed, upstream uncompressed.  This is because IMAP upstream
commands may be real short (meaning compression actually adds CPU *and*
bits used), while content may be real long.

(3) Basis for COMPRESS document is Arnt's imap-deflate-02
[compress=deflate].  We will call the new document
draft-ietf-lemonade-compress-04 (delete existing compress text)

(4) Protocol allows for client to turn it on or off at will (not per
body part, but point in time in session)

(5) review SASL mechanism for handling glare: server is busy streaming
down compressed content, and the client says, "stop compressing".  Right
now looks like server will continue until finished with IMAP
transaction.

_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade

---

From: Dave Cridland <[email-masked]>
Date: Wed, 31 May 2006 23:36:21 +0100

> bits used), while content may be real long.
This is not quite true.

The issue is that the upstream data flow compresses worse, as far as 
I can tell, than downstream, and compression is, in general, more 
expensive than decompression in terms of memory and resource usage. I 
may have results skewed on how well the upstream data flow 
compresses, though, so be warned.

So there is a chance that a client may decide to avoid compression 
upstream - note that for most mobile handsets this is probably not 
the case, because there is compression (my results indicate about 50% 
- as I say, these may be skewed), and as I understand things, every 
bit on the radio transmitter is costly.

In the case were the client wants to avoid upstream compression, it 
could do so by using uncompressed data blocks in deflate, but these 
do not stream - they're prefixed with an octet count (or two octet 
counts and a block type code, to be precise), so every "send" - ie, 
command or pipeline of them - will incur a 5 octet overhead, and is 
worth avoiding.

As I say, though, actually compressing does reduce the octet count, 
and compressed blocks don't have the overhead - I'm not clear if a 
new block needs to be started for a Z_SYNC_FLUSH or not, in fact. 
I've never seen compression actually expand data. This is solely an 
issue with deliberately not wanting to compress.

Also, please note, if you're intending using TLS, you always want to 
compress, because the CPU cost of compression and encryption is lower 
than the CPU cost of encryption alone, if the compression is at all 
effective.

> draft-ietf-lemonade-compress-04 (delete existing compress text)
Why? It's obviously not a revision of that document. Is there a 
procedural reason for this?

> now looks like server will continue until finished with IMAP
So you apply the same rules to turning compression off as already 
exist for turning it on - the compression stops (or starts) after the 
CR LF of the tagged OK response to the COMPRESS command.

The interesting thing is where you apply compression before SASL 
(SASL exchanges compress, they're base64), and SASL then negotiates a 
security layer. This requires SASL to "slip a new layer" under 
compression. Perfectly possible to do, but I'd be interested in how 
difficult people actually find this in practise.

Also, I think there's a (6) - a direct analogue of Arnt's draft can 
also be used in Submission. Although the "protocol chatter" is less, 
it makes sense to keep to the same form of mechanism if possible. I 
use it both in Submission and in ACAP, currently. (I'll do a draft 
for it if Arnt wants to keep them distinct, unless anyone else wants 
to).

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade