Subject: [xmpp] extensibility of existing XMPP elements

From: Kurt Zeilenga <[email-masked]>
Date: Tue, 4 Aug 2009 08:27:54 -0700

While XMPP spec's discussion of extended content (Section 9.4 of  
3920bis) says "XMPP uses XML namespaces (see [XML-NAMES]
to extend the basic stanza syntax for the purpose of providing  
additional functionality", the subsequent discussion is
limited to extension through addition of child elements.

There is no discussion of whether one may extend XMPP elements through  
additional attributes.   The specification should be
clarified as whether such extensions are allowed or not.  If allowed,  
the specification should include some guidance especially
in regards to what implementations should do in face of attributes  
they don't recognize.

(Note that I am not arguing that such extensions are or are not  
allowed, or what should or should not be allowed.  I'm just
asking for the specification to be clear on what is allowed or not.)

-- Kurt

---

From: Jack Moffitt <[email-masked]>
Date: Tue, 4 Aug 2009 10:53:04 -0600

> clarified as whether such extensions are allowed or not.  If allowed, the
> specification should include some guidance especially
> in regards to what implementations should do in face of attributes they
> don't recognize.

I believe several XEPs do you extension attributes, so we should
probably add that this is allowed.

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 04 Aug 2009 12:58:31 -0600

On 8/4/09 9:27 AM, Kurt Zeilenga wrote:
> clarified as whether such extensions are allowed or not.  

Nothing in the spec has ever forbidden extended attributes instead of
extended elements. However, there is a tradition in the XMPP community
of using only elements for the various extensions, not attributes. I
freely grant that this custom might have emerged because of limitations
in the earliest implementations.

> If allowed,

I would assume that they would ignore unrecognized attributes, just as
they currently ignore unrecognized elements.

Peter

- --
Peter Saint-Andre
https://stpeter.im/

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 04 Aug 2009 20:02:00 +0100

>they currently ignore unrecognized elements.
I agree with Kurt that this needs to be explicitly stated in the spec.

---

From: Joe Hildebrand <[email-masked]>
Date: Tue, 04 Aug 2009 13:52:08 -0600

> they currently ignore unrecognized elements.

As long as you add "and MAY discard unrecognized elements, particularly on
stanza elements", then I'm fine with it.

In other words: feel free to send them, but don't expect me to forward them
along.

-- 
Joe Hildebrand

---

From: Jack Moffitt <[email-masked]>
Date: Tue, 4 Aug 2009 14:07:57 -0600

> In other words: feel free to send them, but don't expect me to forward them

Yuck.  Doesn't that defeat the whole purpose?  How about we formalize
where namespaced things can go so that you can not have these on the
top level stanza, but any child element is guaranteed to have the
payload forwarded along?  I suppose we have to leave some leeway in
there for stripping stuff out for mobile clients.  Perhaps that's what
you meant?

jack.

---

From: Dave Cridland <[email-masked]>
Date: Tue, 04 Aug 2009 21:12:38 +0100

> there for stripping stuff out for mobile clients.  Perhaps that's  

Right - the stanza element is for the consumption by the hop, and the  
peer may (and in some cases almost always will) tweak or entirely  
replace the attributes before forwarding, but never the contents.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Randall Gellens <[email-masked]>
Date: Tue, 4 Aug 2009 14:12:28 -0700

At 9:12 PM +0100 8/4/09, Dave Cridland wrote:

>  Right - the stanza element is for the consumption by the hop, and 
> the peer may (and in some cases almost always will) tweak or 
> entirely replace the attributes before forwarding, but never the 
> contents.

So extension attributes MAY be sent in any stanza, including 
top-level ones, but unrecognized attributes in a top-level stanza MAY 
be discarded?

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly selected tag: ---------------
The chance of forgetting something is directly proportional
to.....to........uh..............

---

From: Joe Hildebrand <[email-masked]>
Date: Tue, 04 Aug 2009 15:59:43 -0600

On 8/4/09 3:12 PM, "Randall Gellens" <[email-masked]> wrote:

>> contents

Right.  I wouldn't expect namespaced attributes on the stanza to make it
through all existing implementations, for example.

> top-level ones, but unrecognized attributes in a top-level stanza MAY

Yes, plus:
"therefore extension attributes on stanza elements SHOULD be avoided for
maximum interoperability"

-- 
Joe Hildebrand

---

From: Kurt Zeilenga <[email-masked]>
Date: Tue, 4 Aug 2009 15:02:54 -0700

> through all existing implementations, for example.

I'm more curious about namespaced attributes on children of the  
stanza, such as <body/>.

-- Kurt

---

From: Joe Hildebrand <[email-masked]>
Date: Tue, 04 Aug 2009 23:27:53 -0600

> I'm more curious about namespaced attributes on children of the

We've always told people not to muck with the pre-defined stuff.  Perhaps
instead of saying stanza, we could say anything in jabber:client (or
jabber:component:accept of course).

-- 
Joe Hildebrand

---

From: Kurt Zeilenga <[email-masked]>
Date: Wed, 5 Aug 2009 06:48:53 -0700

> jabber:component:accept of course).

I'm torn a bit on this.  I would like for the entity originally adding  
a child element to a stanza to be able to include extension  
attributes, properly name spaced, and hope that these would get passed  
through.   In particular, I would to be able to add 'xml:id'  
attributes to child elements so that one could unambiguously identify  
elements in a stanza.  (Because 'xml:id' uniqueness requirements,  
values might have to be more unique than unique within stanza.  But I  
digress.)

As has been noted, such attributes simply might not get passed.  If  
so, then their useful is limited.  However, is the extension that  
making use of such an attribute has a strong value, I suspect that  
would drive servers passing of at least that extensions attributes  
(even if the extension is not otherwise implemented).

But I guess I would be okay with:
    "Extension attributes on stanza and non-extension child elements  
SHOULD be avoided for interoperability reasons."

as this leaves the door open enough...

-- Kurt

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 12 Aug 2009 12:25:54 -0600

On 8/5/09 7:48 AM, Kurt Zeilenga wrote:
> I'm torn a bit on this.  I would like for the entity originally adding a
> child element to a stanza to be able to include extension attributes,
> properly name spaced, and hope that these would get passed through.   In
> particular, I would to be able to add 'xml:id' attributes to child
> elements so that one could unambiguously identify elements in a stanza. 
> (Because 'xml:id' uniqueness requirements, values might have to be more
> unique than unique within stanza.  But I digress.)
> 
> As has been noted, such attributes simply might not get passed.  If so,
> then their useful is limited.  However, is the extension that making use
> of such an attribute has a strong value, I suspect that would drive
> servers passing of at least that extensions attributes (even if the
> extension is not otherwise implemented).
> 

Putting together everything discussed in this thread, I suggest the
following text:

   Stanzas themselves (i.e., the <iq/>, <message/>, and <presence/>
   elements qualified by the "jabber:client" or "jabber:server"
   namespace) and child elements of stanzas MAY also include attributes
   that are qualified by XML namespaces other than "jabber:client",
   "jabber:server", or "http://etherx.jabber.org/streams".  For the sake
   of backward compatibility and maximum interoperability, an entity
   that generates a stanza SHOULD NOT include such attributes in the
   stanza itself or in child elements of the stanza that are qualified
   by the "jabber:client" and "jabber:server" namespaces (e.g., the
   message <body/> element).

Peter

- --
Peter Saint-Andre
https://stpeter.im/

---

From: Jack Moffitt <[email-masked]>
Date: Wed, 12 Aug 2009 14:51:13 -0600

>   message <body/> element).

Don't several XEPs use attributes currently?  I was happy to say that
extended attributes on the top level element MAY be removed/ignored by
the server, but I fail to see what is backward-incompatible with
including them on child elements?

XEP 206 uses extended attributes at the top level (although it's
extending httpbind <body/> elements).

There are all kinds of good reasons to use attributes.  xml:lang is
one that is used in lots of examples on /message/body elements.

It seems to me that implementations which do not recognize these
elements will simply keep forwarding them along and ignore them, just
like they do with unknown children.

jack.

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 12 Aug 2009 15:11:45 -0600

On 8/12/09 2:51 PM, Jack Moffitt wrote:
> Don't several XEPs use attributes currently?  

Mostly SOAP (XEP-0072), I'd guess.

> I was happy to say that

Perhaps some testing is in order?

> extending httpbind <body/> elements).

Right, that's different.

> one that is used in lots of examples on /message/body elements.

Well, xml:lang is grandfathered in by RFC 3920. Probably it makes to
exempt all xml:foo attributes, since XMPP is supposed to be an XML
application...

> like they do with unknown children.

That strikes me as a reasonable expectation.

Peter

- --
Peter Saint-Andre
https://stpeter.im/

---

From: Ralph Meijer <[email-masked]>
Date: Thu, 13 Aug 2009 11:50:26 +0200

On Wed, 2009-08-12 at 12:25 -0600, Peter Saint-Andre wrote:
> [..]

This is not correct. Attributes without a prefix DO NOT inherit the
namespace of the containing element. They are in the empty namespace.
This is documented here: http://www.w3.org/TR/REC-xml-names/#defaulting 
As it is worded now, this would be ok:

  <presence myfancyattribute="yay!"/>

and this wouldn't be:

  <presence client:myfancyattribute="yay!"
            xmlns:client="jabber:client"/>

-- 
Groetjes,

ralphm

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 13 Aug 2009 08:14:29 -0600

>             xmlns:client="jabber:client"/>

The text I proposed does not talk about prefixing, but about whether an
attribute is qualified by a namespace other than the default namespace
or the streams namespace (perhaps the term "bound to" would be clearer
than the term "qualified by"?).

In your first example, the attribute 'myfancyattribute' is qualified by
the default namespace according to the defaulting rules you have quoted,
but it is not defined in the schema for the default namespace (assuming
the default is 'jabber:client' or 'jabber:server') and therefore I think
it is invalid content.

In your second example, the entity is unnecessarily defining the
attribute 'myfancyattribute' as qualified by the 'jabber:client'
namespace, yet that is a mere assertion because that attribute is not to
be found in the schema. Here again, invalid content.

The example that people are concerned about is something like this
(which is valid but not recommended):

   <presence foo:bar='baz' xmlns:foo='http://example.org/foo'/>

The content that Kurt would like to include is something like this (see
http://www.w3.org/TR/xml-id/ for reference):

   <message>
     <body xml:id='qux'>content</body>
   </message>

It strikes me that we might make an exception for attributes that are
bound to the namespace name http://www.w3.org/XML/1998/namespace since
we assume that all XMPP applications are XML applications, but that we
don't necessarily want to generalize beyond that.

Peter

- --
Peter Saint-Andre
https://stpeter.im/

---

From: Jack Moffitt <[email-masked]>
Date: Thu, 13 Aug 2009 10:03:24 -0600

> the default is 'jabber:client' or 'jabber:server') and therefore I think

The text Ralph pointed to specifically says attributes that are
unprefixed are not associated with any namespace, regardless of
whether there is a default namespace in scope or not.  "The
interpretation of unprefixed attributes is determined by the element
on which they appear", whatever that means.  This is certainly
counter-intuitive and not at all what I had previously understood.

Does this mean that the DTD for jabber:client could say that th to
attribute on a <message/> element is really from the jabber:server
namespace?  I'm guessing this is why xmlns doesn't need a prefix.

I will have to go read deeper to figure out what the conformance
requirements are.  I'm guessing they must be the same or there would
be no confusion :)

> don't necessarily want to generalize beyond that.

I do think we need to grandfather in the
http://www.w3.org/XML/1998/namespace namespace.

jack.

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 13 Aug 2009 22:56:14 -0600

> counter-intuitive and not at all what I had previously understood.

Ralph was right, I was wrong. Here is the relevant text:

"Default namespace declarations do not apply directly to attribute
names; the interpretation of unprefixed attributes is determined by the
element on which they appear.... The namespace name for an unprefixed
attribute name always has no value."

Ralph interpreted this to mean that "Attributes without a prefix DO NOT
inherit the namespace of the containing element. They are in the empty
namespace." I'm not sure this really matters, because the text says that
"the interpretation of unprefixed attributes is determined by the
element on which they appear", which to me sounds like *effectively*
inheriting the namespace of the containing element. I think this is what
the Namespaces in XML spec means when it says that "The namespace name
for an unprefixed attribute name always has no value", which I interpret
as equivalent to what Ralph called "the empty namespace" (whatever that
is!). I realize that this sounds like a purely philosophical topic, but
in practice I don't think it matters whether an unprefixed attribute has
an empty namespace, a namespace name with no value, or whatever; what
matters is the practical interpretation of an unprefixed attribute,
which "is determined by the element on which [it] appear[s]".

> Does this mean that the DTD for jabber:client could say that the to

Among other things, it means that the second 'type' attribute in the
following example is not qualified by the default namespace, because
"the interpretation of unprefixed attributes is determined by the
element on which they appear"...

<message type='headline'>
  <foo xmlns='http://example.com/foo'
       type='bar'/>
</message>

I don't think it means that the 'to' attribute on a <message/> stanza in
a stream whose default namespace is "jabber:client" is really qualified
by the "jabber:server" namespace, because in this case the <message/>
stanza is qualified by the default namespace and therefore the "to"
attribute is also qualified by that namespace (since it is unprefixed).

As a reminder, here is the text I suggested:

  "jabber:server", or "http://etherx.jabber.org/streams";.  For the sake

Ralph says that according to the text I provided, the following stanza
is acceptable...

... whereas the following stanza is not acceptable ...

I suggest that these two are practically equivalent, for the reasons I
mentioned above. However, I think that the text can be clarified
appropriately by changing "attributes" to "prefixed attributes" and
making a few additional clarifications, as follows:

  Stanzas themselves (i.e., the <iq/>, <message/>, and <presence/>
  elements qualified by the "jabber:client" or "jabber:server"
  namespace declared as the default namespace for the stream)
  and child elements of such stanzas MAY also include prefixed
  attributes that are qualified by XML namespaces other than
  the default namespace for the stream or the reserved "xml"
  prefix.  For the sake of backward compatibility and maximum
  interoperability, an entity that generates a stanza SHOULD NOT
  include such attributes in the stanza itself or in child elements
  of the stanza that are qualified by the default namespace (e.g.,
  the message <body/> element).

> requirements are.  I'm guessing they must be the same or there would

In fact I think they might be reasonable, but not clearly expressed.

> http://www.w3.org/XML/1998/namespace namespace.

Agreed. See above.

Peter

- --
Peter Saint-Andre
https://stpeter.im/

---

From: Waqas Hussain <[email-masked]>
Date: Fri, 14 Aug 2009 11:04:54 +0500

> attribute is also qualified by that namespace (since it is unprefixed).

The 'to' attribute is not qualified by the default stream namespace,
but the default empty "" namespace.

>            xmlns:client="jabber:client"/>

And that's fine. The two are not equivalent.

>  the message <body/> element).

Why make a special case for prefixed attributes qualified by the
default stream namespace? We don't use them in any specification, and
they wouldn't make sense otherwise. And again, they are not the same
as non-prefixed attributes.

> In fact I think they might be reasonable, but not clearly expressed.

Namespace prefixes, particularly for attributes cause a lot of
confusion. It might help to 'roll-out' the prefixes when thinking
about them.

Think of

as

 <{jabber:client}presence {}myfancyattribute="yay!"/>

and

 <presence client:myfancyattribute="yay!" xmlns:client="jabber:client"/>

maps to

  <{jabber:client}presence {jabber:client}myfancyattribute="yay!"/>

(expansion notation stolen from http://www.jclark.com/xml/xmlns.htm -
which might be a useful read)

As should be apparent, both a namespace aware, and namespace unaware
XML processor (conforming ones at least) would see both of the above
as different. There is no reason for attributes qualified by
"jabber:client" to get any special consideration.

>> http://www.w3.org/XML/1998/namespace namespace.

That depends. It's pretty much an implicit requirement (or it could be
explicit, I didn't look it up right now) for XML processors. Expat in
namespace aware mode for example reports the namespace with attributes
(i.e., it reports finding "http://www.w3.org/XML/1998/namespace|lang"
and not "xml:lang").

Also, while we are talking about prefixed attributes on stanzas, what
about namespace prefix declarations? They are not the same for a
namespace aware XML processor. Expat for example does not report
finding "xmlns" or "xmlns:*" attributes (because they are special,
qualified by the magical "http://www.w3.org/2000/xmlns/" namespace,
which can neither be seen, nor used).

Also, why prevent prefixed attributes on children of stanzas? Prefixed
attributes are pretty much in the spirit of XML namespaces, and I
don't think there are any interop problems. Heck, being able to stick
attributes in arbitrary locations without conflicts is the sole reason
prefixed attributes exist. Are there any known problems with any
implementation? I think it's unlikely, since you would pretty much
have to work at it to make it fail.

--
Waqas Hussain

---

From: Joe Hildebrand <[email-masked]>
Date: Fri, 14 Aug 2009 14:24:45 -0600

> have to work at it to make it fail.

The most important thing is that you can't use a prefix that was declared
outside the stanza (for example, on a stream:stream element), since that
context will be lost when the stanza is resent through multiple servers.

A lesser issue at this point was that we used to have servers that were
almost completely namespace unaware, that were likely to serialize prefixed
content oddly when it was resent.  That may not be as much of an issue now,
but we should do some testing for real-world backward compatibility before
loosening up the prefix restriction.

Regardless, if we do allow prefixes, it may be difficult to guarantee that
the prefix string would stay constant through a series of parse/serialize
operations.  If prefixes are allowed, we should point out that processors
SHOULD try to maintain prefixes when serializing, but receivers MUST NOT
rely on the prefix strings having any particular value.

-- 
Joe Hildebrand

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 14 Aug 2009 16:15:36 -0600

> as non-prefixed attributes.

We make a special case for "extensions" in the sense of "extended
content" as described in 3920/3920bis. In the case of elements this is
clear. In the case of attributes this is not clear because of the crazy
no-namespace rules in the namespaces spec. Therefore we need to make
this as clear as we can, which is what we're trying to do in this
thread. If you have text suggestions, feel free to provide them on the
list. I don't understand the exact nature of your suggestions.

> which might be a useful read)

Yes, that's good.

> "jabber:client" to get any special consideration.

They don't. The intent of the text I provided was to say that if you
want to include "extension attributes" then you need to be careful, and
an "extension attribute" is an attribute that is qualifed by a namespace
other than the default namespace for the stream. What we're trying to
specify is exactly what we mean by an "extension attribute", as we have
done for "extension elements". If the current text is not clear, then we
need to make it clear. Does removing the word "prefixed" help? I don't
really think so.

> have to work at it to make it fail.

See Joe's email. I don't think we want to prevent prefixed attributes,
but you need to be careful about how you handle them.

Peter

- --
Peter Saint-Andre
https://stpeter.im/

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 14 Aug 2009 16:37:03 -0600

The text for this section now reads as follows...

***

9.4.  Extended Content

   While the message, presence, and IQ stanzas provide basic semantics
   for messaging, availability, and request-response interactions, XMPP
   uses XML namespaces (see [XML-NAMES] to extend the basic stanza
   syntax for the purpose of providing additional functionality.

   A message or presence stanza MAY contain one or more optional child
   elements specifying content that extends the meaning of the message
   (e.g., an XHTML-formatted version of the message body as described in
   [XEP-0071]), and an IQ stanza of type "get" or "set" MUST contain one
   such child element.  Such a child element MAY have any name and MUST
   possess a namespace declaration (other than "jabber:client", "jabber:
   server", or "http://etherx.jabber.org/streams") that defines all data
   contained within the child element.  Such a child element is called
   an "extension element".

   Similarly, "extension attributes" are allowed.  That is: a stanza
   itself (i.e., the <iq/>, <message/>, and <presence/> elements
   qualified by the "jabber:client" or "jabber:server" namespace
   declared as the default namespace for the stream) and any child
   element of such a stanza (whether a child element qualifed by the
   default namespace or an extension element) MAY also include one or
   more attributes that are qualified by XML namespaces other than the
   default namespace or the reserved "xml" prefix (including the "empty
   namespace" if the attribute is not prefixed).  For the sake of
   backward compatibility and maximum interoperability, an entity that
   generates a stanza SHOULD NOT include such attributes in the stanza
   itself or in child elements of the stanza that are qualified by the
   default namespace (e.g., the message <body/> element).

   An extension element or extension attribute is said to be EXTENDED
   CONTENT and the namespace name for such an element or attribute is
   said to be an EXTENDED NAMESPACE.

   Because an XML stanza is the primary unit of meaning in XMPP, any
   prefixed element or attribute included in a stanza MUST use a prefix
   that is declared in the stanza itself or a child element of the
   stanza, not outside the context of the stanza (e.g., not on the
   stream header).

   Routing entities (typically servers) SHOULD try to maintain prefixes
   when serializing XML stanzas for processing, but receiving entities
   MUST NOT rely on the prefix strings having any particular value.

   Support for any given extended namespace is OPTIONAL on the part of
   any implementation.  If an entity does not understand such a
   namespace, the entity's expected behavior depends on whether the
   entity is (1) the recipient or (2) an entity that is routing the
   stanza to the recipient.

   Recipient:  If a recipient receives a stanza that contains a child
      element it does not understand, it MUST silently ignore that
      particular XML data, i.e., it MUST NOT process it or present it to
      a user or associated application (if any).  In particular:
      *  If an entity receives a message or presence stanza that
         contains XML data qualified by a namespace it does not
         understand, the portion of the stanza that qualified by the
         unknown namespace MUST be ignored.
      *  If an entity receives a message stanza whose only child element
         is qualified by a namespace it does not understand, it MUST
         ignore the entire stanza.
      *  If an entity receives an IQ stanza of type "get" or "set"
         containing a child element qualified by a namespace it does not
         understand, the entity MUST return an IQ stanza of type "error"
         with an error condition of <service-unavailable/>.
   Router:  If a routing entity (typically a server) handles a stanza
      that contains a child element it does not understand, it MUST
      ignore the associated XML data by routing or delivering it
      untouched to the recipient.

***

This text is in the -01 draft, which I have just submitted. It can be
further cleaned up in -02.

Peter

- --
Peter Saint-Andre
https://stpeter.im/