Subject: Re: [xmpp] a dissertation on message delivery

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 09 Aug 2010 15:28:01 -0600

The updated versions of 3920bis and 3921bis will have some adjusted and
harmonized text along the lines discussed below. I will submit them in a
few minutes.

On 7/31/10 10:25 AM, Peter Saint-Andre wrote:
> I started to write a post about server handling of a headline message that
> is sent to a full JID where no such resource is online, but while doing so
> I recognized that there are broader issues here, thus the dissertation
> that follows. My apologies for the length, but I think it's important to
> get this right.
> 
> There are three primary considerations at play in server handling of
> inbound messages, which sometimes are at odds but need to be managed in a
> consistent way:
> 
> 1. It's good to deliver a message if possible.
> 
> 2. If a message can't be delivered, it's helpful to inform the sender.
> 
> 3. It's bad to facilitate directory harvesting attacks and presence leaks.
> 
> Take groupchat messages as a fairly non-controversial example. Clearly we
> want to deliver such messages to the user who has joined a groupchat room.
> However, if the user who was in a room then went offline, it's helpful to
> return an error to the chat service so that it can clean up any "ghosts"
> in the room. On the other hand, it's dangerous to return an error if the
> user doesn't exist when the same kind of message sent to an existing
> account would not return an error (because that would enable a directory
> harvesting attack).
> 
> One simple message handling rule that would meet all of the foregoing
> criteria is "the server always returns an error in response to messages of
> type groupchat sent to bare JIDs".
> 
> However, handling of groupchat messages sent to full JIDs is a bit more
> complex. If the server delivers the message when there is an active
> resource that matches the full JID but returns an error when there is no
> matching full JID, then an attacker could send lots of messages to the
> same account with various random resources and perhaps discover whether
> the user is online. Whether the attack is practical is another matter --
> the server could start to return a different error condition if it detects
> such an attack from a particular JID, although the attacker could be smart
> and send its messages from multiple JIDs. We need to weigh costs and
> benefits, and recognize that there are legitimate tradeoffs between
> functionality and security. IMHO it is more difficult to figure out what
> the right tradeoffs are with respect to presence leaks than directory
> harvesting attacks, which is one reason why I think we're having trouble
> agreeing on the right behavior in the case of a headline message that is
> sent to a full JID where no such resource is online.
> 
> I think there is one more principle here: from the perspective of an
> attacker, there is little if any effective difference between the server's
> (i) delivering the message or storing it offline and (ii) silently
> ignoring it (because an error is not returned immediately in any of those
> cases). Therefore, in scenarios where a server delivers a message or
> places a message into offline storage for an existing account, it needs to
> silently ignore the message if that account does not exist. This implies
> that if a server deploys offline storage and thus would store offline a
> message of type chat sent to the bare JID of an account that has no active
> resources, it would need to silently ignore such a message if sent to a
> bare JID for which there is no account.
> 
> I see two subsidiary conclusions:
> 
> 1. Server handling of messages sent to bare JIDs has implications for
> directory harvesting.
> 
> 2. Server handling of messages send to full JIDs has implications for
> presence leaks. However, the attack is less direct here (because the
> attacker needs to try many different resources in an attempt to find the
> one resource that matches) so it is of somewhat lesser concern, in my
> opinion.
> 
> Therefore, I propose the following table. For this table to be useful, you
> need to mentally walk through some of the scenarios, such as "if the
> server receives a message of type chat for a full JID where the account
> does not exist, then it would silently drop the message; whereas if the
> server receives such a message and there is more than one resource with a
> non-negative priority but there is no matching resource, then it would
> deliver the message to the most-available resource or to all of the
> resources (if they have opted in for receiving all chat messages)".
> 
> The conventions "O", "E", "D", "S", "M", and "A" are defined in Section
> 8.3.4 of draft-ietf-xmpp-3921bis-09.
> 
> Finally, a word about the choices (e.g., "S/E"). If a server is more
> concerned about protecting against presence leaks, it would choose to
> silently drop messages sent to the full JID because some messages will
> always be delivered. If a server is more concerned about providing helpful
> feedback to (some) senders, it would choose to return errors in some cases
> (e.g., groupchat messages sent to full JIDs if there is no such active
> resource).
> 
> I'd like it if we could simplify this table, but I found it helpful to
> provisionally define the entire matrix (this matrix is different from the
> one currently in 3921bis, and the text in 3921bis does not match this
> matrix in all cases).
> 
> (I'm posting this via webmail so I hope it works! If not, perhaps someone
> could clean it up and repost, because I am out of time. I might have
> missed something so other corrections are welcome.)
> 
> +----------------------------------------------------------+
> | Condition        | Normal | Chat  | Groupchat | Headline |
> +----------------------------------------------------------+
> | ACCOUNT DOES NOT EXIST                                   |
> |  bare            |  S/E   |  S/E  |     E     |    S     |
> |  full            |  S/E   |  S/E  |    S/E    |   S/E    |
> +----------------------------------------------------------+
> | ACCOUNT EXISTS, BUT NO ACTIVE RESOURCES                  |
> |  bare            |  O/E   |  O/E  |     E     |    S     |
> |  full (no match) |  S/E   |  O/E  |    S/E    |   S/E    |
> +----------------------------------------------------------+
> | 1 NEGATIVE RESOURCE                                      |
> |  bare            |  O/E   |  O/E  |     E     |    S     |
> |  full match      |   D    |   D   |     D     |    D     |
> |  full no match   |  S/E   |  O/E  |    S/E    |   S/E    |
> +----------------------------------------------------------+
> | 1 NON-NEGATIVE RESOURCE                                  |
> |  bare            |   D    |   D   |     E     |    D     |
> |  full match      |   D    |   D   |     D     |    D     |
> |  full no match   |  S/E   |   D   |    S/E    |   S/E    |
> +----------------------------------------------------------+
> | 1+ NON-NEGATIVE RESOURCES                                |
> |  bare            |  M/A   |  M/A* |     E     |    A     |
> |  full match      |   D    |  D/A* |     D     |    D     |
> |  full no match   |  S/E   |  M/A* |    S/E    |   S/E    |
> +----------------------------------------------------------+
> 
> I will be mostly offline for the next few days because of travel and
> meetings, but I will try to reply to list discussion where possible.
> 
> Peter
>

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 10 Aug 2010 14:40:28 -0600

The diffs are here:

http://tools.ietf.org/rfcdiff?url2=draft-ietf-xmpp-3920bis-12.txt

http://tools.ietf.org/rfcdiff?url2=draft-ietf-xmpp-3921bis-10.txt

I was working on several tasks at once, so sanity checks would be
appreciated. :)

/psa

---

From: Florian Zeitz <[email-masked]>
Date: Wed, 11 Aug 2010 22:56:30 +0200

> I was working on several tasks at once, so sanity checks would be
s/1 NEGATIVE RESOURCE/1+ NEGATIVE RESOURCES/ in 3921bis is the only
thing I could spot when I had a quick look (and that was in there before).
However you have replaced some of the "When the to attribute
contains"-type of things with "In this case", which I occasionally found
more confusing than helpful. Most notably in Section 8.3.

Looking good otherwise ;)

--
Florian Zeitz

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 11 Aug 2010 15:14:01 -0600

> thing I could spot when I had a quick look (and that was in there before).

I think this is even more accurate:

1+ NEGATIVE RESOURCES BUT NOT NON-NEGATIVE RESOURCES

> more confusing than helpful. Most notably in Section 8.3.

I got tired of typing all that text. But I can add it.

> Looking good otherwise ;)

Thanks for the feedback!

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 11 Aug 2010 15:19:03 -0600

> 1+ NEGATIVE RESOURCES BUT NOT NON-NEGATIVE RESOURCES

s/NOT/NO/

---

From: Kevin Smith <[email-masked]>
Date: Fri, 13 Aug 2010 13:43:11 +0100

> harmonized text along the lines discussed below. I will submit them in a

I've finally got to this thread, sorry. I think going down this route
of Silently Ignore/Error to avoid attacks has unfortunately dug us
into two holes. One hole we know about - the tradeoff between giving
useful information to potential attackers vs. defending against the
attack and we probably can't do anything about. The other hole is that
we're now (if we silently ignore) delivering potentially degraded
service to people who already have all the information they'd have
been able to glean from the attack. I'm therefore suggesting (somewhat
hesitantly) that the way out of hole 2 is to further complicate the
rules - by saying that if the sender has presence, you never silently
ignore, but instead always error.

Comments?

/K

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 16 Aug 2010 14:11:55 -0600

On 8/13/10 6:43 AM, Kevin Smith wrote:
> attack and we probably can't do anything about. 

Do you mean we can't do anything to fend off the attack, or that we dug
this hole so long ago that we can't go back at this late date?

> The other hole is that
> been able to glean from the attack. 

Good point.

> I'm therefore suggesting (somewhat

Put another way: there is no presence leak if I have already authorized
you to see my presence (or have sent you directed presence).

Let me look at the text to see how we can incorporate this approach...

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 16 Aug 2010 15:07:48 -0600

> Let me look at the text to see how we can incorporate this approach...

Here's a counterpoint:

While it's nice to return helpful errors to those who already know (or
legitimately could know) the information that would be leaked to an
attacker, we might also need to look at the broader picture so that the
server's behavior is consistent.  Consider: if an attacker has one
contact at a server, then it could see that the server behaves one way
when it sends a message to that known contact (returning a helpful
error) and another way when it sends a message to an unknown entity
(silently ignoring the message). This is something that an especially
security-conscious deployment might want to factor into its server-wide
policies.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Florian Zeitz <[email-masked]>
Date: Mon, 16 Aug 2010 23:18:11 +0200

> security-conscious deployment might want to factor into its server-wide
Question: What information can an attacker derive from that?
AFAICT only that the server acts differently depending on whether he's
subscribed or not. That doesn't tell him if the message sent to someone
he's not subscribed to and he didn't get a reply for was a) delivered b)
stored in offline storage or c) ignored because the user doesn't exist.

--
Florian Zeitz

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 16 Aug 2010 15:42:57 -0600

> stored in offline storage or c) ignored because the user doesn't exist.

Hmm, I think you're right. I was getting overly paranoid...

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 16 Aug 2010 15:48:30 -0600

> Let me look at the text to see how we can incorporate this approach...

I suggest that we make the following adjustments.

First, in 3920bis we add the last paragraph of the following section
(everything else remains the same).

###

10.2.  General Considerations

   At high level, there are three primary considerations at play in
   server processing of XML stanzas, which sometimes are at odds but
   need to be managed in a consistent way:

   1.  It is good to deliver a stanza to the intended recipient if
       possible.

   2.  If a stanza cannot be delivered, it is helpful to inform the
       sender.

   3.  It is bad to facilitate directory harvesting attacks
       (Section 13.11) and presence leaks (Section 13.10.2).

   With regarding to possible delivery-related attacks, the following
   points need to be kept in mind:

   1.  From the perspective of an attacker, there is little if any
       effective difference between the server's (i) delivering the
       stanza or storing it offline for later delivery (see [XMPP-IM])
       and (ii) silently ignoring it (because an error is not returned
       immediately in any of those cases); therefore, in scenarios where
       a server delivers a stanza or places the stanza into offline
       storage for later delivery, it needs to silently ignore the
       stanza if that account does not exist.

   2.  How a server processes stanzas sent to the bare JID
       <localpart@domainpart> has implications for directory harvesting.

   3.  How a server processes stanzas sent to a full JID has
       implications for presence leaks.  However, the attack is less
       direct here (because the attacker needs to try many different
       resources in an attempt to find the one resource that matches) so
       it is of somewhat lesser concern.

   Naturally, presence is not leaked if the entity to which a user's
   server returns an error already knows the user's presence or is
   authorized to do so (e.g., by means of a presence subscription or
   directed presence), and a server does not enable a directory
   harvesting attack if it returns an error to an entity that already
   knows if a user exists (e.g., because the entity is in the user's
   contact list); these matters are discussed more fully in [XMPP-IM].

###

Second, I suggest that we add the following section to 3921bis.

###

8.1.  General Considerations

   [XMPP-CORE] discusses general considerations for stanza delivery, in
   particular the tradeoffs between (i) providing an acceptable level of
   service regarding stanza delivery and (ii) preventing directory
   harvesting attacks and presence leaks.  However, the concept of a
   directory harvesting attack does not apply if a contact is known to
   and trusted by a user (because the contact is in the user's roster as
   described under Section 2).  Similarly, the concept of a presence
   leak does not apply if a contact is authorized to know a user's
   presence (by means of a presence subscription as described under
   Section 3) or if the user has voluntarily sent presence to an entity
   (by means of directed presence as described under Section 4.6).
   Therefore, in cases where the following sections guard against
   directory harvesting attacks and presence leaks by providing an
   alternative of (a) silently ignoring a stanza or (b) returning an
   error, a server SHOULD return an error if the originating entity is
   in the user's roster (when the error would reveal whether the user's
   account exists) or is authorized to receive presence from the user or
   has received directed presence from the user (when the error would
   reveal the presence of a user's resource).

###

/psa