Subject: [xmpp] #28: prefix required for streams namespace

From: "xmpp issue tracker" <[email-masked]>
Date: Mon, 14 Jun 2010 21:17:31 -0000

#28: prefix required for streams namespace
--------------------------------+-------------------------------------------
 Reporter:  stpeter@…           |       Owner:  stpeter@…         
     Type:  defect              |      Status:  new               
 Priority:  major               |   Milestone:                    
Component:  3920bis             |     Version:                    
 Severity:  In WG Last Call     |    Keywords:                    
--------------------------------+-------------------------------------------
 Section 4.5.1 states:

 > A streams namespace declaration is REQUIRED in all XML stream
 > headers, and the name of the streams namespace MUST be
 > 'http://etherx.jabber.org/streams'. If this rule is violated, the
 > entity that receives the offending stream header MUST return a
 > stream error to the sending entity....

 Waqas Hussain commented: "Is this requiring that we detect if a *prefix*
 is defined? If yes, then this effectively forbids using Expat (and many
 other parsers) in namespace aware mode (which is a fairly serious issue
 with the spec)."

-- 
Ticket URL: <http://trac.tools.ietf.org/wg/xmpp/trac/ticket/28>
xmpp <http://tools.ietf.org/xmpp/>

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 23 Jun 2010 21:59:59 -0600

On 6/14/10 3:17 PM, xmpp issue tracker wrote:
>  is defined? 

In essence, yes, because the streams namespace is not allowed to be the
default namespace.

> If yes, then this effectively forbids using Expat (and many

Please explain.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Waqas Hussain <[email-masked]>
Date: Thu, 24 Jun 2010 22:24:54 +0500

On Thu, Jun 24, 2010 at 8:59 AM, Peter Saint-Andre <[email-masked]> wrote:
>>  > A streams namespace declaration is REQUIRED in all XML stream
>>  > headers, and the name of the streams namespace MUST be
>>  > 'http://etherx.jabber.org/streams'. If this rule is violated, the
>>  > entity that receives the offending stream header MUST return a
>>  > stream error to the sending entity....
>>

That text requires that XMPP applications only use XML parsers which
report prefix declarations to the application. XML-NAMES does not
require this of conforming XML processors. Is there any advantage in
placing this requirement on XMPP applications?

--
Waqas Hussain

---

From: Peter Saint-Andre <[email-masked]>
Date: Sun, 27 Jun 2010 14:32:20 -0600

> placing this requirement on XMPP applications?

Other than the fact that it has always been this way?

Given the fact that Jabber/XMPP technologies have always assumed that
the streams namespace is not the default namespace (and remember that
these technologies were developed in 1998/1999, when XML namespaces were
quite new and not well understood), I'm not sure that we can fix this
now. Is there perhaps a better way to word this restriction? For
example, could we phrase this in terms of namespaces rather than
prefixes? But then if the streams namespace is different from the
default namespace (which is always true) then that will imply the need
for a streams namespace declaration. I don't see any way around the
problem here...

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Dave Cridland <[email-masked]>
Date: Mon, 28 Jun 2010 10:22:44 +0100

> for a streams namespace declaration. I don't see any way around the

A fully namespace aware parser can still be used to detect the  
default namespace by injecting a <ns/> dummy element into the input  
after the stream header and seeing what namespace it comes out as.

This comes out of the hacks-are-us book of hacky hacks, of course,  
but it will work.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Matthew Wild <[email-masked]>
Date: Mon, 28 Jun 2010 10:38:14 +0100

>> for a streams namespace declaration. I don't see any way around the

Is there any problem to solve at all?

I don't see why we need to require the server to require a client to
be backwards-compatible with an old spec.

For outgoing streams we should just say that for backwards
compatibility, an entity SHOULD use a prefix of 'stream' for the
namespace 'http://etherx.jabber.org/streams' on outgoing streams, and
the default namespace should be 'jabber:client' or 'jabber:server' on
stream headers that it generates.

For incoming streams the only thing that should matter to a modern
compliant server is that the namespace on the actual stream tag,
stream elements, and stanzas are valid. What declarations the client
puts in the stream header and what prefixes it uses should be of no
concern outside the server's XML parser.

> A fully namespace aware parser can still be used to detect the default
> namespace by injecting a <ns/> dummy element into the input after the stream
> header and seeing what namespace it comes out as.
>
> This comes out of the hacks-are-us book of hacky hacks, of course, but it
> will work.
>

I propose SRV lookups to decide whether the implementation is one
known to send invalid stream namespaces, though this could allow for
DoS of s2s between servers by attackers able to spoof DNS records,
until DNSSEC is widely implemented.

Matthew

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 28 Jun 2010 08:09:49 -0600

> Is there any problem to solve at all?

Waqas says there is. :)

> concern outside the server's XML parser.

Color me confused.

Waqas objected to the very fact that you need to declare the streams
namespace. Ignoring the prefix madness, it seems to me (unless I am
missing something completely obvious) that you simply must declare the
streams namespace in the stream header if the default namespace is other
than the streams namespace, which it always is. So how can you avoid the
need for declaring the streams namespace? Therefore I don't see how we
can remove or modify this text, to which Waqas objected:

   A streams namespace declaration is REQUIRED in all XML stream
   headers, and the name of the streams namespace MUST be
   'http://etherx.jabber.org/streams'.  If this rule is violated, the
   entity that receives the offending stream header MUST return a stream
   error to the sending entity, which SHOULD be <invalid-namespace/>
   (although some existing implementations send <bad-format/> instead).

Similarly, because the default namespace is other than the streams
namespace, we probably need to prefix elements qualified by that
namespace within the stream:

   The element names of the <stream/> element and its <features/> and
   <error/> children MUST be qualified by the streams namespace prefix
   in all instances.  If this rule is violated, the entity that receives
   the offending element MUST return a stream error to the sending
   entity, which SHOULD be <bad-format/>.

You could argue that there's no need to prefix those elements, because
we could use so-called "prefix-free canonicalization":

      <error xmlns='http://etherx.jabber.org/streams'>
        <invalid-namespace
            xmlns='urn:ietf:params:xml:ns:xmpp-streams'/>
      </error>

I'd be fine with that approach.

As to the prefix name for the streams namespace, we've already removed
the hardcoding to "streams", see here:

      Interoperability Note: For historical reasons, an implementation
      MAY accept only the 'stream:' prefix for these elements.  If an
      entity receives a stream header with a streams namespace prefix it
      does not accept, it MUST return a stream error to the sending
      entity, which SHOULD be <bad-namespace-prefix/> (although some
      existing implementations send <bad-format/> instead).

That is softer than your proposed "SHOULD" to set the prefix to "streams".

So exactly what is Waqas objecting to?

> until DNSSEC is widely implemented.

Very funny. :P

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Matthew Wild <[email-masked]>
Date: Mon, 28 Jun 2010 16:50:55 +0100

> Color me confused.

Ok, I perhaps took the wrong tack based on what you write below. Note
that I haven't spoken to Waqas (he hasn't been online today) and I
shan't speak for him, but...

>   (although some existing implementations send <bad-format/> instead).

Unless waqas is reading this as requiring you to use a prefix, I don't
see anything to object to here.

It may perhaps be better worded along the lines of "The root stream
element must have the namespace 'http://...'". In the same way as I
spoke about prefixes in my last mail, all we really care about is the
resulting elements, and that they have the correct namespace.

Perhaps Waqas can suggest some equivalent text he'd be happier with.

> we could use so-called "prefix-free canonicalization":

Yes, I think all that needs changing here is s/prefix//.

>      existing implementations send <bad-format/> instead).

Yes, I didn't realise this was done, I assumed it wasn't, I'd missed
it, and Waqas was complaining about it - hence my earlier post
focusing on prefix names.

> So exactly what is Waqas objecting to?

Let's wait for him to come online and tell us :)

Regards,
Matthew

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 28 Jun 2010 10:40:23 -0600

> resulting elements, and that they have the correct namespace.

True. I'll see if I can reword that paragraph (although I think the
practical implication is that you'll need namespace declarations).

> Perhaps Waqas can suggest some equivalent text he'd be happier with.

OK.

> Yes, I think all that needs changing here is s/prefix//.

Most definitely -- an element cannot be *qualified* by a prefix, only by
a namespace. :)

> Let's wait for him to come online and tell us :)

Yep. :)

But until then I'll fix up the text a bit and perhaps post again.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 28 Jun 2010 11:38:14 -0600

Thanks for the continued feedback. Revised text at the bottom of this
message.

> practical implication is that you'll need namespace declarations).

See below.

> But until then I'll fix up the text a bit and perhaps post again.

Here's what I have now in my working copy. These are mostly slight
revisions as discussed in this thread. I addition, I have:

1. Moved all the text about declarations and prefixes to a new
subsection, in an effort to reduce confusion.

2. Moved the text about mandatory-to-implement default namespaces to a
new subsection at the end of this section, in an effort to improve the flow.

###

4.5.  Namespaces

   Readers are referred to the specification of XML namespaces
   [XML-NAMES] for a full understanding of the concepts used in this
   section.

4.5.1.  Streams Namespace

   The root <stream/> element ("stream header") MUST be qualified by the
   namespace 'http://etherx.jabber.org/streams' (the "streams
   namespace").  If this rule is violated, the entity that receives the
   offending stream header MUST return a stream error to the sending
   entity, which SHOULD be <invalid-namespace/> (although some existing
   implementations send <bad-format/> instead).

   The element names of the <stream/> element and its <features/> and
   <error/> children MUST be qualified by the streams namespace in all
   instances.  If this rule is violated, the entity that receives the
   offending element MUST return a stream error to the sending entity,
   which SHOULD be <invalid-namespace/> (although some existing
   implementations send <bad-format/> instead).

4.5.2.  Default Namespace

   The default namespace for data sent over the stream MUST be other
   than the streams namespace.  The default namespace MUST be the same
   for the initial stream and the response stream so that both streams
   are qualified consistently.  The default namespace applies to all
   first-level child elements sent over the stream unless explicitly
   qualified by another namespace.

4.5.3.  Other Namespaces

   Either party to a stream MAY send data qualified by namespaces other
   than the default namespace and the streams namespace.  For example,
   this is how data related to TLS negotiation and SASL negotiation are
   exchanged, as well as XMPP extensions such as server dialback
   [XEP-0220] and stream management [XEP-0198].

4.5.4.  Namespace Declarations and Prefixes

   Because the default namespace is other than the streams namespace,
   the following statements are true:

   1.  The stream header needs to contain a namespace declaration for
       both the default namespace and the streams namespace.
   2.  The streams namespace declaration needs to include a namespace
       prefix for the streams namespace.

      Interoperability Note: For historical reasons, an implementation
      MAY accept only the prefix 'stream' for the streams namespace
      (resulting in prefixed names such as <stream:stream> and <stream:
      features>).  If an entity receives a stream header with a streams
      namespace prefix it does not accept, it MUST return a stream error
      to the sending entity, which SHOULD be <bad-namespace-prefix/>
      (although some existing implementations send <bad-format/>
      instead).

   By definition, the namespace declaration for the default namespace
   will not include a namespace prefix (e.g., "xmlns='jabber:client'").
   Furthermore, an implementation MUST NOT generate namespace prefixes
   for elements qualified by the default namespace if the default
   namespace is 'jabber:client' or 'jabber:server'.

   Namespaces declared in a stream header MUST apply only to that stream
   (e.g., the 'jabber:server:dialback' namespace used in server dialback
   [XEP-0220]).  In particular, because XML stanzas intended for routing
   or delivery over streams with other entities will lose the namespace
   context declared in the header of the stream in which those stanzas
   originated, namespaces for extended content within such stanzas MUST
   NOT be declared in that stream header (see also Section 8.4).  If
   either party to a stream declares such namespaces, the other party to
   the stream SHOULD close the stream with a stream error of <invalid-
   namespace/>.  In any case, an entity MUST ensure that such namespaces
   are properly declared (according to this section) when routing or
   delivering stanzas originating from such a stream over streams with
   other entities.

4.5.5.  Mandatory-to-Implement Default Namespaces

   XMPP as defined in this specification uses two default namespaces:
   'jabber:client' and 'jabber:server'.  These namespaces are nearly
   identical but are used in different contexts (client-to-server
   communication for 'jabber:client' and server-to-server communication
   for 'jabber:server').  The only difference between the two is that
   the 'to' and 'from' attributes are OPTIONAL on stanzas sent over XML
   streams qualified by the 'jabber:client' namespace, whereas they are
   REQUIRED on stanzas sent over XML streams qualified by the 'jabber:
   server' namespace.  Support for these default namespaces implies
   support for the common attributes (Section 8.1) and basic semantics
   (Section 8.2) of all three core stanza types (message, presence, and
   IQ).

   An implementation MAY support default namespaces other than 'jabber:
   client' or 'jabber:server'.  However, because such namespaces would
   define applications other than XMPP, they are to be defined in
   separate specifications.

   For historical reasons, an implementation MAY refuse to support any
   other default namespaces.  If an entity receives a stream header with
   a default namespace it does not support, it MUST return an <invalid-
   namespace/> stream error.

   Client implementations MUST support the 'jabber:client' default
   namespace.

   Server implementations MUST support both the 'jabber:client' default
   namespace (when the stream is used for communication between a client
   and a server) and the 'jabber:server' default namespace (when the
   stream is used for communication between two servers).

      Implementation Note: Because a client sends stanzas over a stream
      whose default namespace is 'jabber:client', if the server to which
      the client is connected needs to route a client-generated stanza
      to another server then it MUST "re-scope" the stanza so that its
      default namespace is 'jabber:server' (i.e., it MUST NOT send a
      stanza qualified by the 'jabber:client' namespace over a stream
      whose default namespace is 'jabber:server').  Similarly, a routing
      server MUST "re-scope" a stanza received over a server-to-server
      stream (whose default namespace is 'jabber:server') so that the
      stanza is qualified by the 'jabber:client' namespace before
      sending it over a client-to-server stream (whose default namespace
      is 'jabber:client').

###

/psa

---

From: Matthew Wild <[email-masked]>
Date: Mon, 28 Jun 2010 19:19:27 +0100

>   implementations send <bad-format/> instead).

This paragraph (above) could be problematic. How do I know if I
receive a stream features stanza with an incorrect namespace? If I
receive:
   <features xmlns="http://example.com/">
       <starttls xmlns="urn:xmpp:starttls"/>
   </features>

how am I to know that it is meant to be stream:features, and that it
isn't just some new protocol I don't understand? This makes this rule
rather hard to enforce.

>      sending it over a client-to-server stream (whose default namespace

+1 to this text from me, with the above issue resolved, it's much better.

Regards,
Matthew

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 28 Jun 2010 13:20:59 -0600

> isn't just some new protocol I don't understand? This makes this rule

Yes it does. Indeed, it's unenforceable. We can only phrase it in terms
of sending, not receiving (and I don't know how valuable that is) --
unless we want to say that no first-level child of the root <stream/>
element can be <features/> or <error/>, which would be an ugly hack. So
I suggest that we delete the second paragraph in Section 4.5.1.

> +1 to this text from me, with the above issue resolved, it's much better.

Super, thanks.

I'm going to mark this issue as provisionally resolved.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: "xmpp issue tracker" <[email-masked]>
Date: Mon, 28 Jun 2010 19:25:44 -0000

#28: prefix required for streams namespace
--------------------------------+-------------------------------------------
 Reporter:  stpeter@…           |        Owner:  stpeter@…         
     Type:  defect              |       Status:  closed            
 Priority:  major               |    Milestone:                    
Component:  3920bis             |      Version:                    
 Severity:  In WG Last Call     |   Resolution:  fixed             
 Keywords:                      |  
--------------------------------+-------------------------------------------
Changes (by stpeter@…):

  * status:  new => closed
  * resolution:  => fixed

Comment:

 Provisionally resolved with this text:

 ###

 4.5.  Namespaces

 4.5.1.  Streams Namespace

 4.5.2.  Default Namespace

 4.5.3.  Other Namespaces

 ###

-- 
Ticket URL: <http://trac.tools.ietf.org/wg/xmpp/trac/ticket/28#comment:1>
xmpp <http://tools.ietf.org/xmpp/>

---

From: Waqas Hussain <[email-masked]>
Date: Tue, 29 Jun 2010 21:10:56 +0500

> But until then I'll fix up the text a bit and perhaps post again.

Okay, thanks for the wait folks :)

My objection wasn't for specifying what the sender should send. It was
for what the receiver MUST not accept.

Taking an example from Peter's other email:

<stream xmlns='http://etherx.jabber.org/streams'>
 <message xmlns='jabber:client'>
   <body>foo</body>
 </message>
</stream>

Now, I'm perfectly fine with the prefix being a SHOULD for the sender
(for compatibility reasons, since many servers require a prefix). But
with the text in the draft, an entity which understood and accepted
this would be non-compliant (since it MUST return a stream error,
according to the draft). That strikes me as wrong. An entity may
reject this with an error if it can't deal with it, but an entity
which can deal with this shouldn't be forced to reject it. And an
entity which does accept this shouldn't be considered non-compliant.

--
Waqas Hussain

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 29 Jun 2010 16:49:50 -0600

> entity which does accept this shouldn't be considered non-compliant.

Waqas, have you read the text in the latest release candidate (and later
messages in this thread)? I think that text addresses your concerns.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/