Subject: [xmpp] 3920bis: handling of silent peers

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 19 Jul 2010 21:50:15 -0600

Both Philipp Hancke and Florian Zeitz have provided comments about the
handling of silent peers (Section 4.4 of 3920bis):

http://www.ietf.org/mail-archive/web/xmpp/current/msg01349.html

http://www.ietf.org/mail-archive/web/xmpp/current/msg01367.html

I suggest the following text to clarify the matters under discussion:

###

4.4.  Handling of Silent Peers

   When an entity that is a party to a stream has not received any XMPP
   traffic from its stream peer for some period of time, the peer might
   appear to be silent.  There are several reasons why this might
   happen:

   1.  The peer is idle and simply has not sent any XMPP traffic over
       its XML stream to the entity.

   2.  The XML stream is broken despite the fact that the underlying TCP
       connection is alive.

   3.  The underlying TCP connection is dead.

   These three conditions are best handled separately, as described in
   the following sections.

      Implementation Note: For the purpose of handling silent peers, we
      treat a two unidirectional TCP connections as conceptually
      equivalent to a single bidirectional TCP connection (see
      Section 3.3); however, implementers need to be aware that, in the
      case of two unidirectional TCP connections, responses to traffic
      at the XMPP application layer will come back from the peer on the
      second TCP connection.

4.4.1.  Idle Peer

   If the stream peer is idle at the XMPP application layer, it is
   acceptable for the entity to ieither close the stream using the
   handshake described under Section 4.3 or to return a stream error
   (e.g., <resource-constraint/> if the entity has reached a limit on
   the number of open TCP connections or <policy-violation/> if the
   connection has exceeded a local timeout policy).  However, it is
   RECOMMENDED to be liberal in accepting idle peers, since experience
   has shown that doing so improves the reliability of communication
   over XMPP networks and that it is typically more efficient to
   maintain a stream between two servers than to aggressively timeout
   such a stream.

4.4.2.  Broken Stream

   In this case, the peer never responds to XMPP traffic that the entity
   sends, whether normal stanzas or specialized stream-checking traffic
   such as the application-level pings defined in [XEP-0199] or the more
   comprehensive stream management protocol defined in [XEP-0198].  It
   is appropriate for the entity to terminate a broken stream using the
   <connection-timeout/> stream error described under Section 4.7.3.4.

4.4.3.  Dead Connection

   If the underlying TCP connection is dead, stream-level checks (e.g.,
   [XEP-0199] and [XEP-0198]) are ineffective.  Therefore it is
   unnecessary to close the stream with or without an error, and it is
   appropriate instead to simply close the TCP connection.

   One common method for checking the TCP connection is to send a space
   character (U+0020) between XML stanzas, which is allowed for XML
   streams as described under Section 11.7; the sending of such a space
   character is properly called a WHITESPACE KEEPALIVE (the term
   "whitespace ping" is often used, despite the fact that it is not a
   ping since no "pong" is possible).

4.4.4.  Use of Checking Methods

   Implementers are advised to support whichever stream-checking and
   connection-checking methods they deem appropriate, but to carefully
   weigh the network impact of such methods against the benefits of
   discovering broken streams and dead TCP connections in a timely
   manner.  The length of time between the use of any particular check
   is very much a matter of local service policy and depends strongly on
   the network environment and usage scenarios of a given deployment and
   connection type; at the time of writing, it is RECOMMENDED that any
   such check be performed not more than once every 5 minutes and that,
   ideally, such checks will be initiated by clients rather than
   servers.  Those who implement XMPP software and deploy XMPP services
   are encouraged to seek additional advice regarding appropriate timing
   of stream-checking and connection-checking methods, particularly when
   power-constrained devices are being used (e.g., in mobile
   environments).

###

Further feedback is welcome.

Peter

- -- 
Peter Saint-Andre
https://stpeter.im/

---

From: Ralph Meijer <[email-masked]>
Date: Tue, 20 Jul 2010 14:09:05 +0200

On Mon, 2010-07-19 at 21:50 -0600, Peter Saint-Andre wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> [..]

This seems to mostly assume that that one stream is the only way to
reach the recipient domain. In deployments where multiple streams
between the sender domain and recipient domain are possible, XEP-0199
and XEP-0198 don't seem effective [*] for determining the state of a
particular stream. Of course one /can/ still consider such a stream
redundant if replies come in through another stream, but I think the
prose is a bit vague in that respect.

[*] I'm not sure about XEP-0198. That spec does mentions that it can be
used for server-to-server streams, but there are no examples and it is
not clear in which stream request and responses can be sent.

--
ralphm

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 21 Jul 2010 12:37:54 -0600

> prose is a bit vague in that respect.

The prose is indeed vague, and I think deliberately so. The point about
multiple streams between the sender domain and recipient domain is
rather subtle and I'm not sure if we need to say much about it here, but
text is welcome if you can figure out a way to clarify what we have now.

> not clear in which stream request and responses can be sent.

We'll need to ask one of the maintainers of that spec to fix it up. :)

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 21 Jul 2010 16:36:03 -0600

> text is welcome if you can figure out a way to clarify what we have now.

Thinking about this further, I realize that we might want to add a note
earlier in the document about multiple streams (e.g., as used for s2s
between large service providers), then add a sentence in Section 4.4
about how the use of multiple streams complicates matters because you
can't say that a particular initial stream is necessarily correlated
with a particular response stream.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 21 Jul 2010 16:51:02 -0600

> with a particular response stream.

How is this?

[right before Section 4.1]

      Implementation Note: Although this specification speaks of a
      single stream in each direction, nothing prevents implementations
      from negotiating multiple streams in each direction (and, in
      practice, large XMPP service providers often use multiple streams
      for server-to-server connectivity).

[right before Section 4.4.1]

      Implementation Note: For the purpose of handling silent peers, we
      treat a two unidirectional TCP connections as conceptually
      equivalent to a single bidirectional TCP connection (see
      Section 3.3); however, implementers need to be aware that, in the
      case of two unidirectional TCP connections, responses to traffic
      at the XMPP application layer will come back from the peer on the
      second TCP connection.  In addition, the use of multiple streams
      in each direction (which is a common deployment choice for server-
      to-server connectivity among large XMPP service providers) further
      complicates application-level checking of XMPP streams and their
      underlying TCP connections, because there is no necessary
      correlation between any given initial stream and any given
      response stream.

/psa

---

From: Ralph Meijer <[email-masked]>
Date: Thu, 22 Jul 2010 09:24:33 +0200

>       correlation between any given initial stream and any given

Yeah, this should cover it. Thanks.

--
ralphm

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 22 Jul 2010 10:04:14 -0600

> Yeah, this should cover it. Thanks.

Super. Once I-D submission opens up again on July 26, I shall submit
draft-ietf-xmpp-3920bis-11 (incorporating the small changes we've
discussed in the working group), at which point it will be ready for
IETF Last Call.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Justin Karneges <[email-masked]>
Date: Thu, 22 Jul 2010 14:12:51 -0700

>       correlation between any given initial stream and any given

This is a little confusing.  I think you mean to say unidirectional "stanza" 
stream instead of TCP connection.  Even with a unidirectional stanza stream 
(normal s2s in one direction, today), you still have a bidirectional TCP 
connection and two XML streams.

XEP-0198 interactions (ack requests / responses) are per stanza stream, 
whether unidirectional or bidirectional.  It may be worth restoring the 
XEP-0198 stream-level <ping> feature, since indeed with XEP-0199 pings you 
can't easily test a specific s2s stream if you have more than one.

-Justin

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 22 Jul 2010 15:28:18 -0600

> connection and two XML streams.

Well, it depends. :)

A stream is bidirectional for the purposes of stream negotiation, but
once negotiation is complete it might be (2) bidirectional as is typical
for c2s or (2) unidirectional as is typical for s2s.

In the s2s case, you might have (a) two unidirectional streams over the
same TCP connection or (b) one unidirectional stream over the first TCP
connection for application-level communication in one direction plus
another unidirectional stream over a second TCP connection for
application-level communication in the other direction. Strictly
speaking those TCP connections are not unidirectional, but they are
treated as such from the XMPP perspective.

I don't know that the neologism "stanza-stream" really clarifies the
matter, but if necessary we can add some further clarifying text along
the lines of the foregoing paragraph. However, I think we're reaching
the point of diminishing returns in this text, which is mostly
informational anyway.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Justin Karneges <[email-masked]>
Date: Thu, 22 Jul 2010 16:10:04 -0700

On Thursday 22 July 2010 14:28:18 you wrote:
> >>>>>> Both Philipp Hancke and Florian Zeitz have provided comments about
> >>>>>> the handling of silent peers (Section 4.4 of 3920bis):
> >>>>>>
> >>>> The prose is indeed vague, and I think deliberately so. The point
> >>>> about multiple streams between the sender domain and recipient domain
> >>>> is rather subtle and I'm not sure if we need to say much about it
> >>>> here, but text is welcome if you can figure out a way to clarify what
> >>>> we have now.
> >>>
> > This is a little confusing.  I think you mean to say unidirectional
> > "stanza" stream instead of TCP connection.  Even with a unidirectional
> > stanza stream (normal s2s in one direction, today), you still have a
> > bidirectional TCP connection and two XML streams.
>
> Well, it depends. :)
>

I'm not sure of a better way to put it.  The directionality applies to stanzas 
and not general communication, and these rules hold both during and 
after "negotiation".  For example, db:verify/db:result can be exchanged over 
a unidirectional s2s stream to add additional allowed domains, even after one 
is already active.

-Justin

---

From: Ralph Meijer <[email-masked]>
Date: Fri, 23 Jul 2010 10:35:06 +0200

> a unidirectional s2s stream to add additional allowed domains, even after one 

This is exactly why I wrote this elsewhere in this thread:

  [*] I'm not sure about XEP-0198. That spec does mentions that it can
  be used for server-to-server streams, but there are no examples and
  it is not clear in which stream request and responses can be sent.

As you are one of the authors of that specification, please change it to
reflect how it works exactly, instead of hinting in a general direction.

--
ralphm

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 23 Jul 2010 08:15:45 -0600

> reflect how it works exactly, instead of hinting in a general direction.

Matthew Wild and I will be revising XEP-0198 before too long, but at the
moment I'm focused on the RFC revisions, so if folks think we need to
further modify the text in 3920bis then let's get that settled. One
thing at a time. :)

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 23 Jul 2010 08:24:58 -0600

> a unidirectional s2s stream to add additional allowed domains, even after one 

It was to clarify this point that I recently tweaked section 3.3 of
3920bis, as follows:

###

3.3.  Directionality

   The TCP connection underlying an XML stream is either bidirectional
   (allowing stanzas to be sent in either direction at any time) or
   unidirectional (allowing stanzas to be send only in one, fixed,
   direction).

   Clients establish a single, bidirectional TCP connection to a server
   for a session, allowing XML stanzas to be sent from client to server
   and server to client.  Servers, for historical reasons, establish two
   unidirectional TCP connections between each other, one in each
   direction.  A future extension might allow servers to establish
   bidirectional connections given suitable feature negotiation;
   however, definition of such a feature is out of scope for this
   document.

   The directionality of TCP connection applies only to XML stanzas
   (Section 8) sent at the application layer.  Therefore during STARTTLS
   negotiation (Section 5) and SASL negotiation (Section 6) two servers
   would use one TCP connection, but after stream setup that TCP
   connection would be used only for the initiating server to send XML
   stanzas to the receiving server.  In order for the receiving server
   to send XML stanzas to the initiating server, the receiving server
   would need to reverse the roles and negotiate an XML stream from the
   receiving server to the initiating server over a separate TCP
   connection.

###

/psa

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 23 Jul 2010 08:31:42 -0600

>    receiving server to the initiating server over a separate TCP

Now, perhaps we do need a new term here -- I'm tempted to use the word
"link" or "hop" because "stanza stream" strikes me as strange. I'll give
it more thought on the plane today. :)

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Philipp Hancke <[email-masked]>
Date: Fri, 23 Jul 2010 16:54:53 +0200

Peter Saint-Andre wrote:
[...]
> It was to clarify this point that I recently tweaked section 3.3 of

No good because of those weird tcp directionality concept.

>    receiving server to the initiating server over a separate TCP

What about this modification:

Clients establish a single TCP connection to a server for a session, 
allowing XML stanzas to be sent from client to server and server to client.

Servers, for historical reasons, establish TCP connections between each 
other which allow an unidirectional exchange of XML stanzas from the 
initiating server to the receiving server. This directionality only 
applies to XML stanzas (Section 8) sent at the application layer.
In order for the receiving server to send XML stanzas to the initating 
server, the receiving server would need to reverse the roles and 
negotiate an XML stream to the initating server over a separate TCP 
connection.
A future extension might allow servers to establish connections which, 
given suitable feature negotation, allow a bidirectional exchange of 
stanzas; however, definition of such a feature is out of scope for this 
document.

###

---

From: Philipp Hancke <[email-masked]>
Date: Fri, 23 Jul 2010 17:49:49 +0200

>       for server-to-server connectivity).

The use of multiple streams for that purpose is a can of worms that you 
should not open.

>       correlation between any given initial stream and any given

FWIW, I don't think I would ever use a 0199-ping to test for idleness of 
a s2s peer but would go for whitespace writes or the old 0198 pings. I 
do not think this note is needed or useful.

4.4.1. in -11 (working copy?) is worse than the -10 version.
The purpose of this stuff is to describe when a s2s stream should be 
closed, putting it into 4.3 would make sense.
The initiating server SHOULD close the stream if it has not sent stanzas 
for a certain period of time.
The receiving server SHOULD wait for the initiating server to close the 
stream, but MAY close the connection using the closing handshake.

Implementing that is much easier than describing it :-)

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 23 Jul 2010 15:03:38 -0600

[Typed while in flight, delivery will be delayed.]

> stanzas; however, definition of such a feature is out of scope for this

That doesn't provide any of the clarifications people requested. I'd
prefer something like this (which I propose moving from the section on
TCP Binding to the section on XML Streams):

###

4.3.  Directionality

   An XML stream is always unidirectional, by which is meant that XML
   stanzas can be sent in only one direction over the stream (either
   from the initiating entity to the receiving entity or from the
   receiving entity to the initiating entity).

   Depending on the type of session that has been negotiated and the
   nature of the entities involved, the entities might use:

   o  Two streams over a single TCP connection; this is typical for
      client-to-server sessions, and a server MUST allow a client to use
      the same TCP connection for both streams.

   o  Two streams over two TCP connections, where one TCP connection is
      used for the stream in which stanzas are sent from the initiating
      entity to the receiving entity and the other TCP connection is
      used for the stream in which stanzas are sent from the receiving
      entity to the initiating entity; this is typical for server-to-
      server sessions.

   o  Multiple streams over two or more TCP connections; this approach
      is sometimes used for server-to-server sessions between two large
      XMPP service providers, but is not otherwise described in this
      document.

   This concept of directionality applies only to stanzas and explicitly
   does not apply to other first-level children of the stream root, such
   as elements used for TLS negotiation, SASL negotiation, server
   dialback [XEP-0220], and the stream management protocol [XEP-0198].
   In particular, during establishment of a server-to-server session,
   while completing STARTTLS negotiation (Section 5) and SASL
   negotiation (Section 6) two servers would use one TCP connection, but
   after the stream negotiation process is done that original TCP
   connection would be used only for the initiating server to send XML
   stanzas to the receiving server.  In order for the receiving server
   to send XML stanzas to the initiating server, the receiving server
   would need to reverse the roles and negotiate an XML stream from the
   receiving server to the initiating server over a separate TCP
   connection.

      Implementation Note: For historical reasons, a server-to-server
      session always uses two TCP connections.  Although a future
      extension might allow servers to use a single TCP connection after
      negotiation of a suitable feature, definition of such a feature is
      out of scope for this document.

      Informational Note: Although XMPP developers sometimes apply the
      terms "unidirectional" and "bidirectional" to the underlying TCP
      connection (e.g., calling the TCP connection for a client-to-
      server session "bidirectional" and the TCP connection for a
      server-to-server session "unidirectional"), strictly speaking a
      stream is always unidirectional (because the initiating entity and
      receiving entity always have a minimum of two streams, one in each
      direction) and a TCP connection is always bidirectional (because
      TCP traffic can be sent in both directions).  Directionality
      applies to the application-layer traffic sent over the TCP
      connection, not to the transport-layer traffic sent over the TCP
      connection itself.

###

/psa

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 23 Jul 2010 15:16:34 -0600

On 7/23/10 9:49 AM, Philipp Hancke wrote:
>>>>>> the

We're not opening that can, only warning people that it exists.

> 4.4.1. in -11 (working copy?) 

Yes, -11 is my working copy because it can't be submitted until July 26.

> is worse than the -10 version.

Possibly.

> for a certain period of time.

It's true that closing of the stream by the idle peer is not mentioned
anywhere. It might be helpful to add that sentence you've provided.

> stream, but MAY close the connection using the closing handshake.

IETF folks don't like the "SHOULD but MAY" construction. Therefore I
suggest the following:

###

   If the peer has not sent any stanzas for a certain period of time, it
   SHOULD close the stream using the handshake described under
   Section 4.4.  If an idle peer does not close the stream, the other
   party to the stream MAY either close the stream using the handshake
   described under Section 4.4 or return a stream error (e.g.,
   <resource-constraint/> if the entity has reached a limit on the
   number of open TCP connections or <policy-violation/> if the
   connection has exceeded a local timeout policy).  However, it is
   preferable to be liberal in accepting idle peers, since experience
   has shown that doing so improves the reliability of communication
   over XMPP networks and that it is typically more efficient to
   maintain a stream between two servers than to aggressively timeout
   such a stream.

###

> Implementing that is much easier than describing it :-)

Quite likely. :)

Peter

-- 
Peter Saint-Andre
https://stpeter.im/