Subject: [xmpp] 3921bis - not responding to probes.

From: Kevin Smith <[email-masked]>
Date: Wed, 21 Jul 2010 15:24:19 +0100

4.3.2 has responding to presence probes as a SHOULD where there are no
online resources. I'd propose we change this to MUST - I'm not
convinced by the 'number of stanzas' argument since a server
interested in reducing this is going to be better able to reduce the
number of probes sent if it knows it gets responses to all of them -
as it's going to make life simpler if you know you're going to get
responses to the stanzas you send.

If people don't buy the above, I suggest that at least this is changed
to a MUST if the user previously had directed presence for the
contact, as this is going to be important for detecting ghosts.

/K

---

From: Jonas Lindberg <[email-masked]>
Date: Wed, 21 Jul 2010 17:05:03 +0200

I disagree per previous discussion on this subject. This is an extremely
useful optimization for large scale services.

Jonas

On Wed, Jul 21, 2010 at 4:24 PM, Kevin Smith <[email-masked]> wrote:

> _______________________________________________
> xmpp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xmpp
>

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 21 Jul 2010 10:01:24 -0600

> useful optimization for large scale services.

Yes, we had discussion about this some months ago and my sense of the
list was SHOULD, not MUST. However, we might want to explain more fully
why a service would not respond to probes.

> On Wed, Jul 21, 2010 at 4:24 PM, Kevin Smith <[email-masked]
> <mailto:[email-masked]>> wrote:
> 

Wouldn't that force the service to keep a record, after the client's
presence session (and for how long after?) of the entities to which the
client sent directed presence (we need a more compact term for that
concept!).

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Kevin Smith <[email-masked]>
Date: Wed, 21 Jul 2010 17:33:15 +0100

> why a service would not respond to probes.

Ok - do we know how much this saves us, compared to the other
optimisation (not requesting presence when we already know it) that's
pretty much excluded by doing this? Given that you've already got the
s2s open by the point that you don't reply, you've already received a
probe, and already processed the probe, it doesn't seem to me that not
sending the reply if it's for an offline user gains much. Does anyone
have any metrics?

> client sent directed presence (we need a more compact term for that

Yes, it would.

/K

---

From: Peter Saint-Andre <[email-masked]>
Date: Sat, 24 Jul 2010 03:59:49 -0600

[Typed while in transit, delivery will be delayed.]

> sending the reply if it's for an offline user gains much. Does anyone

I don't have metrics, but perhaps Jonas does.

I think part of the challenge here is that if a server MUST always
respond to a probe then it takes the hit, whereas the optimization
you're arguing for won't help the server that's responding to probes and
will depend on the good behavior of the remote server (which the probed
server can't depend upon).

>> client sent directed presence (we need a more compact term for that

That seems problematic to me. Traditionally, directed presence has been
something that the server keeps track of during the life of the presence
session. That session ends when the client sends unavailable (or goes
offline), and at that point the server doesn't need to care about the
entities to which the client had previously sent directed presence.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Jonas Lindberg <[email-masked]>
Date: Mon, 26 Jul 2010 10:20:26 +0200

> I don't have metrics, but perhaps Jonas does.

Responding to presence probes for unavailable users would add an extra
~15% presence traffic in our use case, which is pretty significant.
The percentage to be much larger if one also accounts for the "other
optimization". (Disclaimer: this number is based on a quick back of
the envelope calculation, so I reserve the right to have made
errors..)

Perhaps more interesting than that number is considering the cost of
processing initial presence for users with large contact lists. Take
the scenario where a user with a 1000 subscribed friends logs in. Say
20% of these friends are connected and present. With the optimization,
servers responding to the probes need to process 200 probes to online
friends and send responses. The other 800 probes can be dropped as
soon as the receiver is identified as offline. But without the
optimization, 400% more probes need to be responded to, which in
addition to the cost of sending a response also may mean having to
load (or keep) policy and roster data for each of these offline users
in memory, to run business logic figuring out what the response should
look like (unavailable/unsubscribed) etc.

> entities to which the client had previously sent directed presence.

Agree that this sounds problematic.

---

From: Kevin Smith <[email-masked]>
Date: Wed, 28 Jul 2010 18:08:52 +0100

> look like (unavailable/unsubscribed) etc.

Thanks. I'm digesting.

/K

---

From: Dave Cridland <[email-masked]>
Date: Thu, 29 Jul 2010 17:36:51 +0100

> > look like (unavailable/unsubscribed) etc.

Actually, you need to send those unsubscribed whether or not the user  
is online, it's the first step in 3921bis. This means you already  
have to load in the rosters. That's a MUST, incidentally - so you're  
also arguing against that.

Also, thanks to PEP, servers currently maintain quite a substantial  
chunk of state for remote contacts which needs cleanup from time to  
time. If probes are unanswered, then we have to have timers per  
contact to decide if they're probably offline, and remove their  
subscriptions.

There's similar cleanup in other places.

For directed presence, as used in MUC, this is particularly  
important, as a directed probe is our sole means of seeing whether a  
full jid remains online or not. (Note that this is somewhat different  
- the response to the MUC probe is either unsubscribed or some  
presence). Again, this can be done with timers, but again, it feels  
very more more hacky.

Essentially, a response to a probe allows consistent behaviour in a  
number of areas where otherwise heuristics, or degraded behaviour,  
have to be currently deployed.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Jack Moffitt <[email-masked]>
Date: Thu, 29 Jul 2010 10:44:55 -0600

> For directed presence, as used in MUC, this is particularly important, as a
> directed probe is our sole means of seeing whether a full jid remains online
> or not. (Note that this is somewhat different - the response to the MUC
> probe is either unsubscribed or some presence). Again, this can be done with
> timers, but again, it feels very more more hacky.

Don't you always need timers since even if the spec says that you MUST
send a response, network conditions or poor implementations could
prevent receipt or sending? I think it is a necessary fact of life on
the Internet that our server code will be full of such timeout
conditions regardless of what the spec says.

I'm not saying you didn't have good arguments in general, but I am not
convinced there is much to lean on with the "my server code will be
hacky" argument.

jack.

---

From: Dave Cridland <[email-masked]>
Date: Thu, 29 Jul 2010 17:50:21 +0100

On Thu Jul 29 17:44:55 2010, Jack Moffitt wrote:
> important, as a

The cleanup is needed rarely, and if left out entirely, there's just  
a gentle build-up of duff subscriptions and MUC ghosts, which, for  
the most part, eventually get cleaned up.

The fact is, though, that if we have responses we can (at least  
generally) rely on, we'll see better service.

I guess what I'm saying is that this should remain at least  
recommended behaviour.

Dave.