Subject: [xmpp] 'to' attribute on stanzas from server->client

From: Matthew Wild <[email-masked]>
Date: Fri, 15 Oct 2010 18:35:29 +0100

I should probably have posted this a while back, but I managed to
convince myself it was a non-issue. Unfortunately it cropped up again
today, so I thought I'd raise it here and let others see what they
think.

The specs are, I think, largely silent on what the 'to' address should
be on stanzas sent from the server to the client. I (naturally)
believe that Prosody's current handling is sensible in all places, but
since it isn't defined by the specs I keep being questioned on it by
client developers (a bug was even filed today).

There are a number of issues that get raised:

1) The spec says that if a stanza received from the client has no 'to'
address then it is addressed to the user's "account". Prosody replies
to such stanzas without a 'from' address, to mirror the request. It
also doesn't stamp a 'to' on the stanza. Since these stanzas are never
routed that seems superflous to me.

2) When a stanza is received to the bare JID, should the 'to' address
be rewritten for the full JID of each session it gets delivered to? We
don't rewrite.

3) Similarly, when a stanza comes in to a full JID for a resource that
is not online, should we rewrite the 'to' to the bare JID?

4) Finally, closely related to #2 and #3, the bug filed with us today
concerned messages in offline storage. Should they preserve the
original 'to', or get rewritten to the bare JID, or the full JID that
just came online?

Now my personal view on this is: why on earth do clients *care* what
the 'to' address is?! They received the stanza, the server delivered
it to them because it believed that was the right thing to do. The
client already knows it's full JID... what IS the problem?

To answer that I think the problem is simply that the rules are not
(clearly) specified.

Additonally I think preserving the original 'to' in all cases allows
clients to detect stanzas that were addressed to other resources for
example. I don't have a use case for this, but it seems more useful
than 1) stripping 'to' entirely or 2) always rewriting it to the
client's full JID, which the client knows full well already.

I think this may touch both 3920bis and 3921bis, but I haven't studied
them extensively yet in this light. I can do so and give proposed text
if it would be helpful.

Thoughts?

Matthew

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 15 Oct 2010 13:49:33 -0600

On 10/15/10 11:35 AM, Matthew Wild wrote:
> address then it is addressed to the user's "account". 

By which we mean implicitly to the user's bare JID.

> Prosody replies

Seems sensible.

> be rewritten for the full JID of each session it gets delivered to? We

It depends.

If the client sends a message stanza to the bare JID of its account,
that message will be processed just like any other message to the bare
JID (e.g., it might end up being delivered to all resources, a few
resources, one resource, or even no resource). When the server delivers
the stanza, it retains the 'to' address of the original stanza.

If the client sends a presence stanza to the bare JID of its account,
the presence stanza will be broadcast to all interested resources, but
the 'to' address of the original stanza will be retained (just as for
any presence stanza sent to the bare JID).

If the client sends an IQ get or set to the bare JID of its account, it
seems to me that the 'from' address of that IQ get or set is implicitly
the full JID of the sending client, so the 'to' address of the IQ result
or error that the server sends in response would be the full JID.

> is not online, should we rewrite the 'to' to the bare JID?

If I understand this scenario correctly, the client is sending a stanza
to one of its own full JIDs, but the destination resource is now gone.
Correct? If so, it seems to me that the 'to' address of the error stanza
that the server delivers to the sending client would be the full JID of
the sending client -- the fact that the destination resource is gone
doesn't change the full JID of the sending client.

> original 'to', or get rewritten to the bare JID, or the full JID that

IMHO it is best to keep the original message just as it was (there is no
reason to modify the 'to' address, and it might even be helpful for the
client that finally receives the message from offline storage to know
the destination JID intended by sending entity.

> client's full JID, which the client knows full well already.

Agreed.

> them extensively yet in this light. I can do so and give proposed text

I think the explanations I wrote above provide the skeleton for proposed
text. I'd be happy to work with you on wordsmithing.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Dave Cridland <[email-masked]>
Date: Fri, 15 Oct 2010 21:15:17 +0100

On Fri Oct 15 18:35:29 2010, Matthew Wild wrote:
> be on stanzas sent from the server to the client.

Agreed - 8.1.1 talks about client-to-server and server-to-server, but  
not server-to-client.

> routed that seems superflous to me.

We stamp with the full jid of the session - if it has one - on any  
stanza we generate to a client. Obviously with XEP-0078 responses,  
that's not the case - I think we don't stamp, but I'd have to check.

> be rewritten for the full JID of each session it gets delivered to?  
The spec sayeth thou SHALT NOT:

   In all cases, the server MUST NOT rewrite the 'to' attribute (i.e.,
   it MUST leave it as <user@domain> rather than change it to
   <user@domain/resource>).

> is not online, should we rewrite the 'to' to the bare JID?
I was expecting this to be SHALT NOT again, but I can't find a  
similar paragraph to the above.

> original 'to', or get rewritten to the bare JID, or the full JID  
Pass. I think at that point anything is reasonable, since this is a  
redelivery.

> client already knows it's full JID... what IS the problem?
Some people have too much time?

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Florian Zeitz <[email-masked]>
Date: Fri, 15 Oct 2010 22:41:30 +0200

> any presence stanza sent to the bare JID).
Okay, now you thoroughly confused me (which might be me, because that
has happened too often lately, but still):
If presence is send without a 'to' attr (as it normally is) it is
broadcast to all interested resources with those resources full JID as
'to' addr.
Now no 'to' attr implies to bare JID, why would the 'to' addr of the
original stanza be retained when this is made explicit?

> the destination JID intended by sending entity.
+1

>> client's full JID, which the client knows full well already.
I'm not sure I agree. To turn this around: "Why would a client parse
something that was addressed to someone else?".
E.g. I have previously mentioned the social network that tells a client
one JID during resource binding, but uses a completely different one for
everything else. If clients were to ignore the 'to' attribute this would
be valid (well doable) behavior and I personally think that's quite bad.
Also almost all clients utterly fail against that service (at least for
the roster get, not sure if we have text that causes this), so that
change wouldn't be documenting the "status quo".
It's probably sensible to define that stanzas that have a 'to' attribute
where the bare portion of the JID matches MUST be handled.

--
Florian Zeitz

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 15 Oct 2010 14:52:29 -0600

> original stanza be retained when this is made explicit?

If Romeo sends directed presence from <[email-masked]> to
<[email-masked]> and she has multiple interested resources (say,
"balcony" and "chamber"), her server will broadcast the directed
presence to all of those resources but it won't change the 'to' address
for those delivered stanzas, instead the 'to' will be her bare JID.

Now, why would things be any different if Juliet sends directed presence
from <[email-masked]/balcony> to her bare JID? I grant that it's
silly to send directed presence to your bare JID, but I think such a
stanza would be handled just like any other directed presence stanza.

> something that was addressed to someone else?".

By "someone else" do you mean the bare JID or a resource other than the
one bound for this client?

> where the bare portion of the JID matches MUST be handled.

That seems reasonable, although I'm trying to come up with exceptions
and counter-examples so I think SHOULD might be enough.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 15 Oct 2010 14:53:57 -0600

> If Romeo sends directed presence from <[email-masked]> to

Sorry, that should have been <[email-masked]/orchard> or somesuch.

---

From: Florian Zeitz <[email-masked]>
Date: Fri, 15 Oct 2010 23:13:35 +0200

> stanza would be handled just like any other directed presence stanza.
That seems to have mostly been me misremembering. I had thought we
specify that no 'to' always implies "To the users bare JID". In fact
that's only the case for <message/> though, so that's alright.

> one bound for this client?
Your choice really. I think a stanza to a "wrong" bare JID should only
have been delivered to you in error, so why handle it. But I'd not be
surprised if someone chose to argue the same for the bound resource.
Personally I think "ignoring" (where the '"' is large enough to allow
"ignore" to be defined as: display the message and note that it was
originally addressed to another resource) the resource is fine though.

> and counter-examples so I think SHOULD might be enough.
That would be fine with me.

Florian

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 15 Oct 2010 21:45:27 -0600

On 10/15/10 2:15 PM, Dave Cridland wrote:
> I was expecting this to be SHALT NOT again, but I can't find a similar
> paragraph to the above.

How about this?

###

8.1.1.1.  Client-to-Server Streams

   The following rules apply to inclusion of the 'to' attribute in
   stanzas sent from the client to the server over an XML stream
   qualified by the 'jabber:client' namespace.

   1.  A stanza with a specific intended recipient (e.g., a conversation
       partner, a remote service, the server itself, even another
       resource associated with the user's bare JID) MUST possess a 'to'
       attribute whose value is an XMPP address.

   2.  A stanza sent from a client to a server for direct processing by
       the server as described in [XMPP-IM] for rosters (e.g., presence
       sent to the server for broadcasting to other entities) MUST NOT
       possess a 'to' attribute.

   The following rules apply to inclusion of the 'to' attribute in
   stanzas sent from the server to the client over an XML stream
   qualified by the 'jabber:client' namespace.

   1.  If the server has received the stanza from another connected
       client or from another server, the server MUST NOT modify the
       'to' address before delivering the stanza to the client.

   2.  If the server has itself generated the stanza (e.g., a response
       to an IQ stanza of type "get" or "set", even if the stanza did
       not include a 'to' address), the stanza SHOULD include a 'to'
       address, which MUST be the full JID of the client.

###

/psa

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 15 Oct 2010 22:00:42 -0600

> That would be fine with me.

We don't have any text right now in 3920bis about client handling of
incoming stanzas. Currently we assume that the client trusts the server
to not deliver stanzas that are intended for another account. This
client checking of the 'to' address seems like a faux security measure
to me ("protecting" the client against stanzas that were originally
addressed a different authorized resource of the same account). Do we
really need to add some text saying "there's no need for the client to
check the 'to' address of stanzas it receives from the server"? I mean,
if your server is sending you malicious stanzas of some kind (which it
could easily do by rewriting every 'to' address anyway!), then you have
much bigger problems.

That said, I suppose we could add a short paragraph to the Security
Considerations about the extent to which the client can or should trust
the server.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Matthew Wild <[email-masked]>
Date: Sat, 16 Oct 2010 05:05:46 +0100

I probably shouldn't be responding to emails at this time of the
night/morning, but here goes...

>       'to' address before delivering the stanza to the client.

+1.

>       address, which MUST be the full JID of the client.

+0.5

I would rather we specify allowed behaviour than not, but I'm still
not convinced that including 'to' in these cases ought to be a SHOULD.
After all, this is completely superfluous information - the client
already knows its JID. I would rather it be treated more like the
'from' attribute, and implicitly the receiving client's full JID if
missing. Anyone with me?

Matthew

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 15 Oct 2010 22:16:16 -0600

On 10/15/10 10:05 PM, Matthew Wild wrote:
> night/morning, 

No, you SHOULD NOT!

> but here goes...

It is indeed superflous information, so MAY is probably more appropriate:

   2.  If the server has itself generated the stanza (e.g., a response
       to an IQ stanza of type "get" or "set", even if the stanza did
       not include a 'to' address), the stanza MAY include a 'to'
       address, which MUST be the full JID of the client; however, if
       the stanza does not include a 'to' address then the client MUST
       treat it as if the 'to' address were included with a value of
       the client's full JID.

Yes, no, maybe?

/psa

---

From: Matthew Wild <[email-masked]>
Date: Sat, 16 Oct 2010 05:20:23 +0100

>       treat it as if the 'to' address were included with a value of

+another 0.5, thanks!

It must be a good sign when we're down to nit-picking on these trivialities :)

Good night!

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 15 Oct 2010 22:26:23 -0600

>>       treat it as if the 'to' address were included with a value of

Let's see...

  0.5
+ 0.5
= 1.0

> It must be a good sign when we're down to nit-picking on these trivialities :)

It's a sign that we're obsessive about XMPP. Perhaps that's not a good
thing. ;-)

> Good night!

Isn't it almost morning for you?!?

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Florian Zeitz <[email-masked]>
Date: Sat, 16 Oct 2010 14:18:21 +0200

> could easily do by rewriting every 'to' address anyway!), then you have
My concern was actually different. A client developer could guess that
it's "okay" to check the 'to' address (for whatever reason). After
having slept over it I'm actually convinced that it is in fact safe to
ignore this. However that you *have to* ignore it (or at least not check
against the full JID) is not entirely clear IMHO.
The text proposed in the other branch of this thread addresses this to
some extent (albeit implicitly).
However if people are writing bug reports against servers about this it
might be worth a Implemenation Note.

--
Florian Zeitz

---

From: Peter Saint-Andre <[email-masked]>
Date: Sat, 16 Oct 2010 20:11:54 -0600

> might be worth a Implemenation Note.

Look, it's fine for someone to write a paranoid client. I happen to
think it's stupid to do so, because if you don't trust your server at
such a fundamental level then you have bigger problems, but I don't
think we can say that a client MUST NOT try to check the 'to' address of
incoming stanzas. However, I think we could say something like this:

   Implementation Note: Because it is the server's responsibility to
   deliver only stanzas that were addressed to the client's full JID or
   the user's bare JID, there is no need for the client to check the
   'to' address of incoming stanzas.

/psa

---

From: Paul Aurich <[email-masked]>
Date: Sat, 16 Oct 2010 22:01:35 -0700

> change wouldn't be documenting the "status quo".

For what it's worth, my recollection of the issue with the social
network-that-we're-not-naming-and-shaming was that they stamped their IQ
response with a "from" attribute of what *they* thought the bare JID of
the account was (which wasn't what they told the client it was during
session binding), and clients validate that someone sending them roster
stanzas is allowed to do so (i.e. is from the account's bare JID or the
full JID of this session, the later option due to a quirk in RFC3921, IIRC).

At least, that was the primary issue with Pidgin (I can't think of any
specific places we validate the "to" attribute).

~Paul

---

From: Paul Aurich <[email-masked]>
Date: Sat, 16 Oct 2010 22:01:51 -0700

On 2010-10-16 19:11, Peter Saint-Andre wrote:
> incoming stanzas.

+1 to both points, although I personally wouldn't complain about
strongly discouraging them unless they can quote all three RFCs from
memory *in verse* ;)

> However, I think we could say something like this:
> 

I may have missed part of the conversation where it was decided the
server /should/ alter the 'to', but what about the case where the server
is delivering a message from offline storage that was originally
addressed to a difference resource?

~Paul

---

From: Florian Zeitz <[email-masked]>
Date: Sun, 17 Oct 2010 14:35:04 +0200

> specific places we validate the "to" attribute).

That is absolutely correct. Large parts of that mail really fall in the
"things I shouldn't have written" category. Sleep deprived and bad cold
is not a good combination it seems.
Sorry for that.

--
Florian Zeitz

---

From: Florian Zeitz <[email-masked]>
Date: Sun, 17 Oct 2010 15:05:01 +0200

> strongly discouraging them unless they can quote all three RFCs from
I also agree. If it wasn't clear: By now I'm also definitely against
adding any normative text about this.

> addressed to a difference resource?
An implementation note would be nice (as I said before), however I'd
suggest:
   Implementation Note: Because it is the server's responsibility to
   deliver only stanzas that were addressed appropriately, there is no
   need for the client to check the 'to' address of incoming stanzas.
   As the 'to' address may be a full JID different to the client's due
   to e.g. offline message delivery it is in fact suggested to check at
   most the bare portion of the 'to' address.

--
Florian Zeitz

---

From: Peter Saint-Andre <[email-masked]>
Date: Sun, 17 Oct 2010 09:23:35 -0600

>> addressed to a difference resource?

Offline storage is out of scope for rfc3920bis and 3921bis -- we simply
reference XEP-0160 for details:

http://xmpp.org/extensions/xep-0160.html

>    most the bare portion of the 'to' address.

Thanks for the proposed text. I'd phrase it like so:

   Implementation Note: Because it is the server's responsibility to
   deliver only stanzas that were addressed appropriately, there is no
   need for the client to check the 'to' address of incoming stanzas.
   Because the 'to' address might be the user's bare JID, the client's
   current full JID, or even a full JID with a different resourcepart
   (e.g., in the case of so-called "offline messages" as described in
   [XEP-01060]), if the client does check the 'to' address then it is
   suggested to check at most the bare JID portion, not the full JID.

/psa

---

From: Peter Saint-Andre <[email-masked]>
Date: Sun, 17 Oct 2010 09:26:39 -0600

>    suggested to check at most the bare JID portion, not the full JID.

Well, except for the typo. :)

---

From: Florian Zeitz <[email-masked]>
Date: Sun, 17 Oct 2010 17:59:34 +0200

> Well, except for the typo. :)

Sounds good. Thank you.

---

From: Paul Aurich <[email-masked]>
Date: Sun, 17 Oct 2010 18:24:23 -0700

>> Well, except for the typo. :)

Same here.

---

From: Matthew Wild <[email-masked]>
Date: Mon, 18 Oct 2010 13:02:04 +0100

>>> Well, except for the typo. :)

And here too :)

Matthew

---

From: Dave Cridland <[email-masked]>
Date: Mon, 18 Oct 2010 13:31:27 +0100

> > routed that seems superflous to me.

Are we really sure that not having a to on a S2C stanza is sensible?

I don't have any strong feelings, but I'm concerned that if there  
exist clients which choke on this, I'd rather err on the side of  
caution and mandate that we do.

I think Prosody is unique in this behaviour outside XEP-0078 code, so  
I suspect it might be easier to "fix" Prosody rather than "fix" the  
clients.

Dave.

---

From: Dave Cridland <[email-masked]>
Date: Mon, 18 Oct 2010 13:32:47 +0100

> >>> Well, except for the typo. :)

I had to check I'd not missed ~800 XEPs being published over the  
weekend.

I'm basically happy with this, though.

Dave.

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 18 Oct 2010 10:30:00 -0600

On 10/18/10 6:31 AM, Dave Cridland wrote:
> I don't have any strong feelings, but I'm concerned that if there exist
> clients which choke on this, I'd rather err on the side of caution and
> mandate that we do.
> 
> I think Prosody is unique in this behaviour outside XEP-0078 code, so I
> suspect it might be easier to "fix" Prosody rather than "fix" the clients.

I would guess that most early servers (and probably many recent ones)
simply swap the 'to' and 'from' attributes when replying to stanzas like
IQs used for service discovery and roster management. Thus I'd expect
that most clients already handle "s2c" stanzas with no 'to' address.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Matthew Wild <[email-masked]>
Date: Mon, 18 Oct 2010 17:55:24 +0100

> suspect it might be easier to "fix" Prosody rather than "fix" the clients.

This is hardly a widespread issue, in fact I can't name any clients
that are broken now - as far as I know the only actual report of a
broken client in use was Jappix, and the developers have fixed that (I
still have no idea why they were checking it, but they now implicitly
set it to their JID if it is missing).

The other reports tend to be from people writing their own libraries
or working with low-level ones. Frankly I don't mind these people
learning that they really have no need to check the 'to' as a client
in XMPP. As far as I'm concerned the spec is simply clarifying the
current state of things.

Matthew

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 18 Oct 2010 11:05:34 -0600

> in XMPP. As far as I'm concerned the spec is simply clarifying the

That's my understanding as well.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Matthew Wild <[email-masked]>
Date: Mon, 18 Oct 2010 18:08:42 +0100

> that most clients already handle "s2c" stanzas with no 'to' address.

Indeed, this is why Prosody doesn't include a "to" in such requests.
Rather than special-casing things like XEP-0077/78 we only stamp the
'from' on a stanza prior to routing. Since private requests to the
server are never routed as such, but responded to directly, they don't
reach the stamping and pre-routing validity checks (has the client
bound a resource, etc.).

Largely unrelated to the issue at hand, but justifcation for Prosody's
behaviour: Prosody deals with all "private" requests prior to routing.
This allows us to cleanly separate routed stanzas from
client-generated "private" stanzas. This completely removes the
possibility of clients requesting other clients' rosters, or (as a
hypothetical example *cough*) modifying other users' passwords.

Regards,
Matthew

---

From: Paul Aurich <[email-masked]>
Date: Mon, 18 Oct 2010 22:04:56 -0700

On 2010-10-18 05:32, Dave Cridland wrote:
> I had to check I'd not missed ~800 XEPs being published over the weekend.

Should the XSF start planning for the X1K problem? ;)

~Paul

---

From: Dave Cridland <[email-masked]>
Date: Tue, 19 Oct 2010 10:38:36 +0100

> Should the XSF start planning for the X1K problem? ;)

The XSF's preferred citation style is 4 digits, allowing for another  
9721 XEPs before we hit a problem. Nevertheless, our finest minds are  
feverishly working on the problem.

Dave.