Subject: [Uta] Compression along with TLS in NNTP

From: Julien ÉLIE <[email-masked]>
Date: Fri, 10 Jun 2016 22:43:06 +0200

Hi all,

Last year, in September 2015, we spoke about the removal of TLS-level 
compression in TLS 1.2.
NNTP was relying on this feature provided by previous TLS versions to 
compress data.  We agreed that the right move was to standardize a new 
NNTP command.  It is what we finally did with the COMPRESS extension.
Interoperability is proven:  two news servers (INN, Cyrus NNTP) and a 
news client (flnews) have already implemented it.  It also works fine 
with Python nntplib+zlib libraries.

Here is the latest version of the draft:
     https://tools.ietf.org/html/draft-murchison-nntp-compress-03

If you have any comments, please don't hesitate to tell.

Especially, I have a question:

    o  How are TLS (and SASL) specific exchanges supposed to be handled?
       Should it be mentioned in the RFC that they are outside the scope
       of NNTP compression?  (I speak of stuff like TLS handshakes,
       renegotiations, etc. that can occur after the use of COMPRESS.
       OpenSSL may use SSL_read/SSL_write on its own, mayn't it?  And it
       does not know that it should compress data...)

Here is an extract of the paragraphs dealing with TLS in the draft, so 
that you can easily see what to comment (wording improvement, missing 
stuff...).

    The point of COMPRESS as an NNTP extension is to behave as a
    transport layer, similar to STARTTLS [RFC4642].  Compression can
    therefore benefit to all NNTP commands sent or received after the use
    of COMPRESS.

[...]

1.1.  About TLS-level Compression

    Though data compression is made possible via the use of TLS with NNTP
    [RFC4642], the best current practice is to disable TLS-level
    compression as explained in Section 3.3 of [RFC7525].  The COMPRESS
    command will permit to keep the compression facility in NNTP and
    control when it is available during a connection.

    Compared to TLS-level compression [RFC3749], NNTP COMPRESS has the
    following advantages:

    o  COMPRESS can be implemented easily both by NNTP servers and
       clients.

    o  COMPRESS benefits from an intimate knowledge of the NNTP
       protocol's state machine, allowing for dynamic and aggressive
       optimization of the underlying compression algorithm's parameters.

    o  COMPRESS can be activated after authentication has completed thus
       reducing the chances that authentication credentials can be leaked
       via for instance a CRIME attack ([RFC7457] Section 2.6).

[...]

2.2.2.  Description

[...]

    Both the client and the server MUST know if there is a compression
    layer active (for instance via the previous use of the COMPRESS
    command or the negotiation of a TLS-level compression [RFC3749]).  A
    client MUST NOT attempt to activate compression or negotiate a TLS
    layer (for instance via the use of the COMPRESS or STARTTLS [RFC4642]
    commands) if a compression layer is already active.  A server MUST
    NOT return the COMPRESS or STARTTLS capability labels in response to
    a CAPABILITIES command received after a compression layer is active,
    and a server MUST reply with a 502 response code if a syntactically
    valid COMPRESS or STARTTLS command is received while a compression
    layer is already active.

[...]

6.  Security Considerations

    Security issues are discussed throughout this document.

    In general, the security considerations of the NNTP core
    specification ([RFC3977] Section 12) and the DEFLATE compressed data
    format specification ([RFC1951] Section 6) are applicable here.

    Implementers should be aware that combining compression with
    encryption like TLS can sometimes reveal information that would not
    have been revealed without compression, as explained in Section 6 of
    [RFC3749].  As a matter of fact, adversaries that observe the length
    of the compressed data might be able to derive information about the
    corresponding uncompressed data.  The CRIME and the BREACH attacks
    ([RFC7457] Section 2.6) are examples of such case.

    In order to help mitigate leaking authentication credentials, this
    document states in Section 2.2.2 that authentication SHOULD NOT be
    attempted when a compression layer is active.  Therefore, when a
    client wants to authenticate, compress data, and negotiate a secure
    TLS layer (without TLS-level compression) in the same NNTP
    connection, it SHOULD use the STARTTLS, AUTHINFO, and COMPRESS
    commands in that order.  Of course instead of using the STARTTLS
    command, a client can also use implicit TLS, that is to say it begins
    the TLS negotiation immediately upon connection on a separate port
    dedicated to NNTP over TLS.

    NNTP commands other than AUTHINFO are not believed to divulgate
    confidential information as far as public Netnews newsgroups and
    articles are concerned.  That is why this specification only adds a
    restriction to the use of AUTHINFO when a compression layer is
    active.  In case private and confidential newsgroups or articles are
    accessed, a server SHOULD NOT allow compression, and a client SHOULD
    NOT attempt to activate compression, for the same reasons as
    mentioned above in this Section.

    Implementations MUST support a configuration where compression can be
    easily disabled.

    Future extensions to NNTP that define commands conveying confidential
    data SHOULD ensure to state that they SHOULD NOT be used along with
    compression.

Thanks again for your useful comments!

-- 
Julien ÉLIE

« Pourvu que ça dure ! » (Letizia Bonaparte)

---

From: Julien ÉLIE <[email-masked]>
Date: Fri, 10 Jun 2016 23:33:16 +0200

> Last year, in September 2015, we spoke about the removal of TLS-level

Of course one should read "TLS 1.3".

-- 
Julien ÉLIE

« I don't worry about terrorism.  I was married for two years. »
   (Sam Kinison)

---

From: Ángel González <[email-masked]>
Date: Sun, 12 Jun 2016 04:11:22 +0200

On 2016-06-10 at 22:43 +0200, Julien ÉLIE wrote:
> restriction to the use of AUTHINFO when a compression layer 

The main problem is not as much that the contents are compressed or
not, but that known plain text and confidential data are compressed
together.

As such, I don't think it's usually necessary to disable it for
confidential articles/newsgroups.
What you need to avoid is that in the same compress session where you
read the article with the steps to enter into the nuclear facility, you
also read a different public article that the spies know you will read,
as the compressed size will vary depending on the contents of the
secret.

Of course, if instead of a general text of relatively unknown length,
the attacker knows that you will get a list of 100 nuclear launch
codes, seeing how well did it compress might allow them to guess some
of their properties.

Regarding your draft, I would 
a) add a None algorithm
b) allow compress to be called inside a compress session, replacing the
previous one.

Thus, a client would be able to COMPRESS DEFLATE, <read public groups>,
COMPRESS NONE, <read secret article>, COMPRESS DEFLATE, <more public
activity>

Or even call COMPRESS DEFLATE, <read article A>, COMPRESS DEFLATE,
<read article B> to ensure their contents are not compressed together.

Best regards

---

From: Julien ÉLIE <[email-masked]>
Date: Sun, 12 Jun 2016 15:53:18 +0200

Hi Ángel,

> confidential articles/newsgroups.

First of all, thanks for your valuable comment.  I will take it into 
account in the wording of the draft.

> <read article B> to ensure their contents are not compressed together.

I understand the scenario.
It implies here that the client will have to know somehow what kinds of 
newsgroups or articles need extra-security.  The client will have to 
send the right COMPRESS commands at the right time.

Wouldn't it be better that the server also knows somehow what kinds of 
newsgroups or articles need extra-security, and clears the compression 
dictionary even if not explicitly asked by the client?
We could otherwise have a situation where not all the clients accessing 
secret articles are correctly configured.  Therefore, one of the clients 
may leak information.  If both the client and the server ensure not to 
compress public and confidential data together, it would be better, 
wouldn't it?

Hmm, on second thoughts, this suggestion of a None algorithm and 
allowing to switch to another algorithm at any time adds complexity that 
could be avoided by another suggestion:  if a client wants to access 
both public groups and secret articles, why not open two separate NNTP 
connections (either in parallel or one after the other)?  These two 
sessions can be compressed, and there will be no leakage.

So as to answer your proposal of ensuring that the contents of articles 
are not compressed together, we could call "COMPRESS DEFLATE FLUSH" (or 
another better optional argument) that asks the server to clear the 
compression dictionary after every response.

-- 
Julien ÉLIE

« Ta remise sur pied lui a fait perdre la tête ! » (Astérix)

---

From: Jeffrey Walton <[email-masked]>
Date: Sun, 12 Jun 2016 11:53:43 -0400

> Here is an extract of the paragraphs dealing with TLS in the draft, so that
> you can easily see what to comment (wording improvement, missing stuff...).
>

It kind of sounds doomed under threat models with active attackers.
The active attacker can swallow the STARTTLS message and no one would
be the wiser.

How does the protocol thwart downgrades under active attackers?

Jeff

---

From: Julien ÉLIE <[email-masked]>
Date: Sun, 12 Jun 2016 21:17:05 +0200

Hi Jeffrey,

> How does the protocol thwart downgrades under active attackers?

First of all, one should note that STARTTLS is not widely used in 
practice for NNTP.  Implicit TLS on a dedicated port (TCP 563) is the 
common way for NNTP clients to negotiate a TLS layer.

To answer your question, the Security Considerations Section of RFC 4642 
(Using TLS with NNTP) mentions:

    Before the TLS handshake has begun, any protocol interactions are
    performed in the clear and may be modified by an active attacker.
    For this reason, clients and servers MUST discard any sensitive
    knowledge obtained prior to the start of the TLS handshake upon the
    establishment of a security layer.  Furthermore, the CAPABILITIES
    command SHOULD be re-issued upon the establishment of a security
    layer, and other protocol state SHOULD be re-negotiated as well.

[...]

    During the TLS negotiation, the client MUST check its understanding
    of the server hostname against the server's identity as presented in
    the server Certificate message, in order to prevent man-in-the-middle
    attacks.  Matching is performed according to these rules:

    -  The client MUST use the server hostname it used to open the
       connection (or the hostname specified in TLS "server_name"
       extension [TLS-EXT]) as the value to compare against the server
       name as expressed in the server certificate.  The client MUST NOT
       use any form of the server hostname derived from an insecure
       remote source (e.g., insecure DNS lookup).  CNAME canonicalization
       is not done.

    -  If a subjectAltName extension of type dNSName is present in the
       certificate, it SHOULD be used as the source of the server's
       identity.

    -  Matching is case-insensitive.

    -  A "*" wildcard character MAY be used as the left-most name
       component in the certificate.  For example, *.example.com would
       match a.example.com, foo.example.com, etc., but would not match
       example.com.

    -  If the certificate contains multiple names (e.g., more than one
       dNSName field), then a match with any one of the fields is
       considered acceptable.

    If the match fails, the client SHOULD either ask for explicit user
    confirmation or terminate the connection with a QUIT command and
    indicate the server's identity is suspect.

    Additionally, clients MUST verify the binding between the identity of
    the servers to which they connect and the public keys presented by
    those servers.  Clients SHOULD implement the algorithm in Section 6
    of [PKI-CERT] for general certificate validation, but MAY supplement
    that algorithm with other validation methods that achieve equivalent
    levels of verification (such as comparing the server certificate
    against a local store of already-verified certificates and identity
    bindings).

    A man-in-the-middle attack can be launched by deleting the STARTTLS
    capability label in the CAPABILITIES response from the server.  This
    would cause the client not to try to start a TLS session.  Another
    man-in-the-middle attack would allow the server to announce its
    STARTTLS capability, but alter the client's request to start TLS and
    the server's response.  An NNTP client can partially protect against
    these attacks by recording the fact that a particular NNTP server
    offers TLS during one session and generating an alarm if it does not
    appear in the CAPABILITIES response for a later session.  (Of course,
    the STARTTLS capability would not be listed after a security layer is
    in place.)

    If the client receives a 483 or 580 response, the client has to
    decide what to do next.  The client has to choose among three main
    options: to go ahead with the rest of the NNTP session, to (re)try
    TLS later in the session, or to give up and postpone
    newsreading/transport activity.  If an error occurs, the client can
    assume that the server may be able to negotiate TLS in the future and
    should try to negotiate TLS in a later session.  However, if the
    client and server were only using TLS for authentication and no
    previous 480 response was received, the client may want to proceed
    with the NNTP session, in case some of the operations the client
    wanted to perform are accepted by the server even if the client is
    unauthenticated.

Does it answer your question?

-- 
Julien ÉLIE

« Contra factum non datur argumentum. »

---

From: Ángel González <[email-masked]>
Date: Mon, 13 Jun 2016 02:06:53 +0200

> account in the wording of the draft.

Hello Julien

You are welcome

> It implies here that the client will have to know somehow what kinds
> of newsgroups or articles need extra-security.  The client will have
> to send the right COMPRESS commands at the right time.

Yes. Take into account that often only the client will be able to know
if the newsgroup contents are deemed confidential by the user or not.

> Wouldn't it be better that the server also knows somehow what kinds
> of newsgroups or articles need extra-security, and clears the
> compression dictionary even if not explicitly asked by the client?

Of course, the server MAY flush the compression dictionary more often
than required to satisfy some security properties (assuming that's
supported by the algorithm).

> We could otherwise have a situation where not all the clients
> accessing secret articles are correctly configured.  Therefore, one
> of the clients may leak information.  If both the client and the
> server ensure not to compress public and confidential data together,
> it would be better, wouldn't it?

No objection to that.

> allowing to switch to another algorithm at any time adds complexity 

I expect it isn't too much. It's basically the same operation of
switching into compressed or encrypted. Moreover, it's easier to verify
that the compressor is properly flushed, as the state is completely
reinitialised.

> that could be avoided by another suggestion:  if a client wants to
> access both public groups and secret articles, why not open two
> separate NNTP connections (either in parallel or one after the
> other)?  These two sessions can be compressed, and there will be no
> leakage.

You are sending a stronger signal to someone that was watching the
connection, as he will be able to see the two connections. Plus, that
requires more server resources (supporting several connections per
client) and there is an added overhead of STARTTLS + AUTH on the new
connection. For the client wishing to implement this, it may encounter
itself being throttled or limited to 1 connection to the server per IP.

> So as to answer your proposal of ensuring that the contents of
> articles are not compressed together, we could call "COMPRESS DEFLATE
> FLUSH" (or another better optional argument) that asks the server to
> clear the compression dictionary after every response.

I expect it to have a similar complexity as allowing a complete
algorithm change, while being more limited. What if eg. the user wanted
to authenticate again with a different set of credentials?

Best regards

---

From: Julien ÉLIE <[email-masked]>
Date: Wed, 15 Jun 2016 23:06:06 +0200

Hi Ángel,

Sorry for the delay in the response.  I was asking some input from 
developers of news clients.

> if the newsgroup contents are deemed confidential by the user or not.

In the news.software.nntp, Michael Bäuerle gave the following answer:

"In general I see the problem at layer 8:  the user needs to understand 
what is going on and how to control things with the user interface of 
the programs.
The same thing as with encrypted E-Mail:  it's far too easy with most 
programs to send confidential information as clear text by accident.

For security purposes it is likely already too much to expect that the 
"compression on/off" switch is recognised as security relevant by the 
users. Therefore I think the best solution for security is to use a 
separate newsreader (explicitly compiled without compression support) 
for security relevant tasks."

> supported by the algorithm).

OK.

> itself being throttled or limited to 1 connection to the server per IP.

I understand that point, and that it is a bad idea in general (stronger 
signal + more resources with the issue of limitation to 1 connection / IP).

> to authenticate again with a different set of credentials?

A user cannot authenticate twice in NNTP (RFC 4643).
The AUTHINFO command is no longer available after a successful use.  The 
user needs to open another NNTP connection.  Using an NNTP command 
several times in a row to change a state is not usual.

Suggestion from news.software.nntp not adding more complexity:

"The simple "compression on/off" switch is very easy to implement and 
should be sufficient for nearly all non academic scenarios.

But maybe the default setting for the switch should be defined as "off",
because this is the more secure option.  Possible words instead of the
current:

| Implementations MUST support a configuration where compression can be
| easily disabled.

---
Implementations SHOULD use a default configuration with disabled
compression and MUST support an option to easily enable compression
[either with global scope or server/connection based].
---

This will ensure that e.g. a new version of a newsreader with support
for compression will not enable it automatically (possibly without
knowledge of a user who have not read the changelog)."

I suggest to also say, like your proposal, that news servers MAY have a 
global or connection based configuration parameter permitting to enforce 
more confidentiality on specific newsgroups (notably by ensuring for 
these newsgroups that the contents of two secret articles are not 
compressed together).  The server will then clear the compression 
dictionary either after selecting these newsgroups or before sending any 
article posted to these newsgroups.

Wouldn't these additions:
- default configuration of disabled compression,
- ideas to ensure protection is maximal for those wishing to implement a 
newsgroups based compression,
- clarification that the main problem is that public data and 
confidential data are compressed together (and not that encryption and 
compression are used together),
satisfy the needs of security?

We believe it would work better than complexifying the simple COMPRESS 
command, and that it no longer introduces potential security issues in 
the default configuration (whereas the previous version did).

-- 
Julien ÉLIE

« A man inserted an 'ad' in the classifieds:  "Wife wanted".
   Next day he received a hundred letters.
   They all said the same thing:  "You can have mine." »