Subject: [BEEPwg] caching and compression profiles?

From: Ryan Ripken <[email-masked]>
Date: Mon, 24 Jun 2002 15:59:42 -0700

Has anyone thought about writing a zlib profile to do gzip compression? 
 What about a profile to do caching?  Is there any reason profiles like 
this couldn't be created?

It seems like a compression profile would be straightforward and useful 
enough that someone may have already done this.

In my case I would like to send a large amount of xml messages (IDMEF) 
from a client to a server.  Using gzip I can usually get around 30x 
compression.  I'd imagine that XMill or some other approach could do 
better.  For security reasons I'd like to use the tls profile (actually 
the idxp profile with tls).  Besides network bandwidth there are 
performance and security reasons why compression is attractive.

  I could turn on compression in openssl in both the client and the 
server machines but that is implementation specific and not general 
 purpose.  I could also compress/decompress the messages in the 
applications - but this ties clients to servers and kinda defeats the 
purpose of using an open format like IDMEF.  I think a compression 
profile would be the best solution and allow peers to negotiate a 
compression profile when it is available on the client and the server.

I did a quick search and found a refernce to a suggestion for a 
compression profile in the mailing list archive, whatever happened to 
that idea?
If a compression profile could be written it wouldn't work very well 
with small messages, that is where a caching profile might be useful - 
cache messages and compress a bunch of them at once.

What do you people think?

Ryan

_______________________________________________
BEEPwg mailing list
[email-masked]
http://lists.beepcore.org/mailman/listinfo/beepwg

---

From: Marshall Rose <[email-masked]>
Date: Tue, 25 Jun 2002 23:12:57 -0700

[ eric rescolra added to thread... ]

eric - what's the story on TLS use of compression? is it automatic, if available? or is there some other "magic" involved.

ryan - i think that caching and compression are probably two different things.

caching is probably profile-specific in the sense that whatever does the caching has to understand the semantics of the profile in use on the channel.

compression is more general. i suppose you could do it either as a tuning profile (for the whole session) or on a per-channel basis. i can see advantages to each, but i'd prefer that the result look like this:

1. if you tune for privacy, then session-wide compression should just happen automatically, as a part of the security layer.

2. if you don't tune for privacy, then you may decide to tune for compression. it should be fairly easy to define a family of profiles to do this, e.g., http://.../zlib, http://.../my-whacky-compression, etc.

if someone feels strongly that compression ought to be done on a per-channel basis, then we'll finally get a chance to use BEEP's feature negotiation mechanism, because there'll have to be a way of letting the two sides negotiate the use of compression before they start adding a "Content-..." header to the messages they exchange over BEEP...

/mtr

---

From: Marshall Rose <[email-masked]>
Date: Wed, 26 Jun 2002 08:07:58 -0700

> > eric - what's the story on TLS use of compression? is it automatic,
> > if available? or is there some other "magic" involved.
> Not good.
> 
> TLS has "support" for compression and automatic algorithm
> negotiation. However, no magic numbers are defined so it doesn't
> actually exist. Essentially what happened was that there were IPR
> concerns about all the algorithms so there was a reluctance to
> standardize.
> 
> A couple of people have implemented compression on an experimental
> scale but there's nothing standard. IIRC, OpenSSL has (disconnected)
> support for RLE and zlib.

eric - thanks, that's very good data. what that indicates to me is that if there's interest in a tuning profile for beep that does compression, then it should be used regardless of whether or not the session has already been tuned for privacy... (but, obviously, tuning for compression should occur after tuning for privacy!)

thanks!

/mtr