Subject: Re: [Shutup] [ietf-smtp] Compressing SMTP streams

From: "John Levine" <[email-masked]>
Date: 29 Jan 2016 18:07:13 -0000

>Compression has been removed completely from TLS v1.3, the outcome of 
>the room consensus at IETF-89.

Bummer.

Well, in that case, here's a straw man proposal.

The extension name is COMPRESS, the EHLO keyword is COMPRESS and is
followed by a space-separated list of compression schemes, currently
consisting only of DEFLATE (RFC 1951.)

There's one new command, COMPRESS which takes as an argument the type
of compression to be used.  If you want to do both STARTTLS and
COMPRESS, the results of doing COMPRESS before STARTTLS are
aggessively undefined.

The responses to COMPRESS are:

500 compress not supported
501 compression scheme unknown
220 go ahead

After a 220 response, subsequent traffic is compressed.  It's up to
each end to ensure that there's a compression block boundary every
time the transmission direction changes, i.e., after each command or
response, or if pipelining after each pipelined group.  If the
compress command fails, the client can at its option continue an
uncompressed session or give up.

Example

S: 220 mx.example smail2 ESMTP

C: EHLO client.example

S: 250-mx.example
S: 250-STARTTLS
S: 250-8BITMIME
S: 250 COMPRESS DEFLATE

C: STARTTLS

S: 220 go ahead

<TLS negotiation>

C: EHLO client.example

S: 250-mx.example
S: 250-8BITMIME
S: 250 COMPRESS DEFLATE

C: COMPRESS DEFLATE
S: 220 go ahead

<subsequent traffic is compressed>
 ... blah blah ...

C: QUIT

S: 221 sayonara

R's,
John

---

From: Kim Alvefur <[email-masked]>
Date: Fri, 29 Jan 2016 20:31:24 +0100

Hi,

> consisting only of DEFLATE (RFC 1951.)

The XMPP community having an application layer compression extension
protocol already, <http://xmpp.org/extensions/xep-0138.html>, it is
pretty much like your proposal.

While CRIME may be less applicable to non-HTTP-protocols, such attacks
are not impossible, as demonstrated by Thijs Alkemade a few years back:

https://blog.thijsalkema.de/blog/2014/08/07/https-attacks-and-xmpp-2-crime-and-breach/

http://mail.jabber.org/pipermail/standards/2014-October/029215.html

The takeaway here is to 1) not allow compression until after any
authentication has been done and 2) flush the compression state between
messages (if the sever supports sending multiple messages over the same
SMTP session).

-- 
Kim "Zash" Alvefur

---

From: Brandon Long <[email-masked]>
Date: Fri, 29 Jan 2016 12:11:23 -0800

That's about exactly what I was thinking.

Brandon

On Fri, Jan 29, 2016 at 10:07 AM, John Levine <[email-masked]> wrote:

> _______________________________________________
> Shutup mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/shutup
>

---

From: Brandon Long <[email-masked]>
Date: Fri, 29 Jan 2016 13:14:06 -0800

On Fri, Jan 29, 2016 at 1:04 PM, Carl S. Gutekunst <[email-masked]> wrote:

>> After a 220 response, subsequent traffic is compressed.
>>
>
> Yes, order must be STARTTLS, then AUTH, then COMPRESS.
>
> We assume that binary MIME (ala RFC 3030) is dead.
>
> Alternative: Is there any benefit to compressing the envelope? With
> pipelining, probably; otherwise no. If we don't compress the envelope, then
> we could make compression an option on the DATA command. Now we have
> something that starts to look like RFC 3030, but without the assumption
> that the sender should not use Base64 (with its attendant and DKIM-breaking
> content transfer encoding conversion at each hop).

1000 recipients at ~30 bytes each with each domain likely the same... maybe
sometimes.

If you're only compressing the data, what's the interaction with RFC 1870?
Do you use the compressed size or the original, and if you use the
compressed size, does that mean the server will accept larger uncompressed
messages?

And I'd think it's an alternative to BDAT, same thing, but compressed?
Unless we extend the BDAT syntax for compression...

Brandon

---

From: "Carl S. Gutekunst" <[email-masked]>
Date: Fri, 29 Jan 2016 13:04:26 -0800

Alternative: Is there any benefit to compressing the envelope? With 
pipelining, probably; otherwise no. If we don't compress the envelope, 
then we could make compression an option on the DATA command. Now we 
have something that starts to look like RFC 3030, but without the 
assumption that the sender should not use Base64 (with its attendant and 
DKIM-breaking content transfer encoding conversion at each hop).

<csg>

---

From: Russ Allbery <[email-masked]>
Date: Fri, 29 Jan 2016 13:19:38 -0800

+ietf-nntp, who may be interested

> Well, in that case, here's a straw man proposal.

[snip proposal for COMPRESS extension to smtp]

BTW, in the NNTP world, we're currently testing:

    http://tools.ietf.org/id/draft-murchison-nntp-compress-01.html

and will probably have a new version of that draft based on early interop
results.

-- 
Russ Allbery ([email-masked])              <http://www.eyrie.org/~eagle/>

---

From: Russ Allbery <[email-masked]>
Date: Fri, 29 Jan 2016 14:04:39 -0800

> and will probably have a new version of that draft based on early interop

Apologies, there's a current version out already:

    http://tools.ietf.org/id/draft-murchison-nntp-compress-02.html