Subject: [xmpp] #67: denial of service possible with large stanzas

From: "xmpp issue tracker" <[email-masked]>
Date: Tue, 22 Jun 2010 02:12:09 -0000

#67: denial of service possible with large stanzas
--------------------------------+-------------------------------------------
 Reporter:  stpeter@…           |       Owner:  stpeter@…         
     Type:  defect              |      Status:  new               
 Priority:  major               |   Milestone:                    
Component:  3920bis             |     Version:                    
 Severity:  In WG Last Call     |    Keywords:                    
--------------------------------+-------------------------------------------
 Regarding Section 8.5 (Stanza Size), Matthew Wild commented: "If the
 purpose of the stanza size limit is to prevent server resource
 consumption, replying with a stanza error is very tricky. Because the
 server would have to continue reading and parsing the stanza to know when
 it has finished, so it can send the reply. This can't be done without
 significant resource consumption in any of the off-the-shelf XML parsers I
 know. Although it sounds brutal, I suggest alternatively allowing the
 server to give a policy-violation error."

-- 
Ticket URL: <http://trac.tools.ietf.org/wg/xmpp/trac/ticket/67>
xmpp <http://tools.ietf.org/xmpp/>

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 23 Jun 2010 16:35:11 -0600

> #67: denial of service possible with large stanzas

Do you mean a <policy-violation/> stream error or a <policy-violation/>
stanza error? I can see reasons for returning either (if the limit is
40k and you send 60k I might warn you with a stanza error, whereas if
you send 600k I might terminate your connection).

Thus I propose:

8.5.  Stanza Size

   XMPP is optimized for the exchange of relatively large numbers of
   relatively small stanzas.  A client or server MAY enforce a maximum
   stanza size (where "size" is inclusive of all XML elements and
   markup).  The maximum stanza size MUST NOT be smaller than 10000
   bytes, from the opening "<" character to the closing ">" character.
   If an entity receives a stanza that exceeds its maximum stanza size,
   it MUST either return a <policy-violation/> stanza error (thus
   allowing the sender to recover) or close the stream with a <policy-
   violation/> stream error.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: "xmpp issue tracker" <[email-masked]>
Date: Wed, 23 Jun 2010 22:36:24 -0000

Type:  defect              |      Status:  assigned          
Changes (by stpeter@…):

  * status:  new => assigned

Comment:

 Provisionally addressed with the following text:

-- 
Ticket URL: <http://trac.tools.ietf.org/wg/xmpp/trac/ticket/67#comment:1>
xmpp <http://tools.ietf.org/xmpp/>

---

From: Matthew Wild <[email-masked]>
Date: Wed, 23 Jun 2010 23:41:42 +0100

>   violation/> stream error.

My point was that to still have an XMPP stream around on which to send
a stanza error, we have to have parsed the full 60K, 600K, whatever.
If the goal of the limit is to prevent DoS on the server, a stanza
error won't suffice because it places responsibility on the server to
keep the stream open and continue parsing the incoming data.

The proposed text is perfect, thanks (though did you also get my issue
about whether the minimum size is inclusive of the '<' and '>'? It
makes sense the size should include these, but I can't help reading it
wrongly sometimes if that's the case... :) ).

Matthew.

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 23 Jun 2010 16:47:48 -0600

> a stanza error, we have to have parsed the full 60K, 600K, whatever.

Correct.

> keep the stream open and continue parsing the incoming data.

Up to a point, yes. If the server is nice, it will have a soft limit and
a hard limit (I say 40k is the limit but I'll return a stanza error if
you send up to 100k whereas above that I'll return a stream error).

I don't know if that's reasonable or not, but I think it can be a matter
of implementation, which is what the proposed text essentially says.

> The proposed text is perfect, thanks 

Super.

> (though did you also get my issue

The foregoing snippet says:

   (where "size" is inclusive of all XML elements and markup)

IMHO "<" and ">" are part of the markup. From the XML specification:

[Definition: Markup takes the form of start-tags, end-tags,
empty-element tags, entity references, character references, comments,
CDATA section delimiters, document type declarations, processing
instructions, XML declarations, text declarations, and any white space
that is at the top level of the document entity (that is, outside the
document element and not inside any other markup).]

Given that XML elements are part of markup, we could just say:

   (where "size" is inclusive of all XML markup)

/psa

---

From: Matthew Wild <[email-masked]>
Date: Thu, 24 Jun 2010 00:02:23 +0100

Sent the below message to Peter, skipping the list by mistake.

On 23 June 2010 23:59, Matthew Wild <[email-masked]> wrote:
> On 23 June 2010 23:47, Peter Saint-Andre <[email-masked]> wrote:
> Yes, I think that's feasible.
>
> Understood, I guess that would be fine by me then. Thanks!
>
> Matthew
>

---

From: Justin Karneges <[email-masked]>
Date: Wed, 23 Jun 2010 16:22:38 -0700

> keep the stream open and continue parsing the incoming data.

Clients have a more difficult responsibility here, because if your server 
sends a large stanza to you, you /really/ don't want to have to send a 
policy-violation stream error to your server and lose your whole session.  
Until we have a way to negotiate a client's max size with the server, a 
client can't easily enforce a maximum size without DoS'ing itself.

Maybe we could have a small note, such as: "the maximum receive stanza size of 
a remote entity SHOULD be assumed to be 10000 bytes unless known to be 
otherwise."  This would be a rule for everyone to follow (c2s or s2s, either 
direction), but most importantly it is for the server->client path.  If your 
server receives a 12000 byte stanza from me, it should reject the stanza 
rather than deliver it on to your client, unless your server has some reason 
to believe your client is capable of receiving a 12K stanza (for example, 
with [1]).

-Justin

[1] http://xmpp.org/extensions/inbox/stanzalimits.html#request

---

From: Matthew Wild <[email-masked]>
Date: Thu, 24 Jun 2010 01:11:45 +0100

> to believe your client is capable of receiving a 12K stanza (for example,

I wouldn't be opposed to such text - it makes good sense, I really
agree that we need discoverable limits. As the text stands, people are
going to hard-code the 10000 limit in their servers, clients and
libraries (I don't need a crystal ball for that). So definining a
minimum limit seems sensible, though I do see the current text as
doing precisely that.

Matthew

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 23 Jun 2010 19:09:48 -0600

> minimum limit seems sensible, though I do see the current text as

If we're going to set limits (and I agree with Justin that developers
will hardcode whatever we mention in the spec), then I suggest that we
legislate something a bit larger. In my experience 64kb is a popular
number with service providers.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 23 Jun 2010 19:16:52 -0600

> number with service providers.

The resulting text:

   XMPP is optimized for the exchange of relatively large numbers of
   relatively small stanzas.  A client or server MAY enforce a maximum
   stanza size (where "size" is inclusive of all XML markup as defined
   in Section 2.4 of [XML]).  The maximum stanza size SHOULD NOT be
   smaller than 65536 bytes, from the opening "<" character to the
   closing ">" character.  If an entity receives a stanza that exceeds
   its maximum stanza size, it MUST either return a <policy-violation/>
   stanza error (thus allowing the sender to recover) or close the
   stream with a <policy-violation/> stream error.  The maximum stanza
   size acceptable to a remote entity SHOULD be assumed to be 65536
   bytes unless known to be otherwise.

/psa

---

From: "xmpp issue tracker" <[email-masked]>
Date: Thu, 24 Jun 2010 03:39:34 -0000

#67: denial of service possible with large stanzas
--------------------------------+-------------------------------------------
 Reporter:  stpeter@…           |        Owner:  stpeter@…         
     Type:  defect              |       Status:  closed            
 Priority:  major               |    Milestone:                    
Component:  3920bis             |      Version:                    
 Severity:  In WG Last Call     |   Resolution:  fixed             
 Keywords:                      |  
--------------------------------+-------------------------------------------
Changes (by stpeter@…):

  * status:  assigned => closed
  * resolution:  => fixed

-- 
Ticket URL: <http://trac.tools.ietf.org/wg/xmpp/trac/ticket/67#comment:2>
xmpp <http://tools.ietf.org/xmpp/>

---

From: Kevin Smith <[email-masked]>
Date: Thu, 24 Jun 2010 07:43:02 +0100

>   bytes unless known to be otherwise.

Ok - I see this is the thread with the discussion in. This text feels
very wrong to me - can someone explain what potential attack this
mitigates against, please (that wouldn't be performed equally well by
sending two half-sized stanzas)? I can see possible resource
exhaustion attacks if the stanza is so huge it can't be stored in
memory - but we're orders of magnitude out for almost all entities
here. As I see it, you're just going to end up bouncing valid and
sensible stanzas and it opens up interesting attacks, such as forcing
different members of a MUC to see a different set of messages.

/K

---

From: Matthew Wild <[email-masked]>
Date: Thu, 24 Jun 2010 14:17:30 +0100

On 24 June 2010 07:43, Kevin Smith <[email-masked]> wrote:

> sending two half-sized stanzas)?

Yes, I can. When a stanza is received by the server it is replied to
or routed, and discarded.

During reception of the stanza, the stanza is using server resources
(usually in the XML parser). In addition to RAM requirements, the
burden of serialization will be greater on the CPU in proportion with
the size of the stanza (serialization is already a documented
bottleneck in many server implementations). Breaking this resource
usage into smaller chunks would be beneficial.

> I can see possible resource
> here.

I disagree. Worst case scenario - 25K clients have allocated 65K each
(this purely the data, and not counting the state that would need
storing either)... by my calculations that's about 1.5GB. I know RAM
is cheap, but this is a significant amount of it. Even on jabber.org I
believe this would currently take us beyond our current margins for
free RAM.

Experience has also taught us that large stanzas are best avoided:
http://metajack.wordpress.com/2008/06/10/binary-data-is-xmpps-achilles-heel/

> As I see it, you're just going to end up bouncing valid and

Yes, though a MUC service could then limit stanza sizes to the minimum
limit in the spec, or track each user's server's limit and implement a
dynamic limit in the MUC. Having such restrictions in place (don't
forget that many servers already have such restrictions, but they
aren't standard nor discoverable) doesn't make things any easier, but
it's necessary for a smoothly-working network in my opinion.

And on a related note the current recommended size of in-band
bytestream payloads is just 4K... (maximum allowed is 65K).

I'm also not sure what you're arguing towards. Do you want any mention
of stanza size removed? Do you think it is wrong to have a defined
method for implementations to bounce over-sized stanzas? I understand
your individual concerns, but I find the current text a decent
compromise for all of them.

Matthew

---

From: Kevin Smith <[email-masked]>
Date: Thu, 24 Jun 2010 14:57:49 +0100

> usage into smaller chunks would be beneficial.

If CPU usage is proportional to stanza size, it makes no difference if
one large stanza, or two half-sized stanzas are sent - the CPU hit is
the same.

Note that I'm not arguing against maximum stanza sizes being a
possibility allowed by the RFC - but mandating it there means that
even if you're able and happy to route non-tiny stanzas, no-one will
ever send them to you.

> believe this would currently take us beyond our current margins for

Aside: RAM, possibly, but not address space - stuff would just get swapped out.

Note that this (every user on the system sending a large stanza at
exactly the same moment) is a very different case from a single user
sending a single large stanza. I can imagine a server wanting to keep
track of how much memory its using and start to refuse requests to do
stuff that will take it outside some implementation-defined limit
(<resource-constrained/>?). I believe this is a different problem to
solve.

> http://metajack.wordpress.com/2008/06/10/binary-data-is-xmpps-achilles-heel/

Yes - although Jack's findings were that when using XMPP to drive a
real-time game that large stanzas were a problem for user experience -
I don't think this  was posing a technical problem to their
infrastructure, nor that it necessarily applies as a generalisation to
server traffic for near-real-time chat for example.

> dynamic limit in the MUC.

With the current wording (All entities must assume the target has the
minimum possible limit), there's nothing the MUC can do. It can't
reject the stanza for being too big, because it's less than the
minimum size allowed to be rejected for being too big. It can't send
it on to the targets, because it'd be larger than the assumed size
limit for the targets.

> Having such restrictions in place (don't

I'm not convinced that it's necessary for a smooth-working network to
have these limits in the RFC. (I'm not arguing that there are no
scenarios where receiving too much data is bad).

> bytestream payloads is just 4K... (maximum allowed is 65K).

I think that's orthogonal to this discussion.

> of stanza size removed?

Only specific sizes, I think.

> Do you think it is wrong to have a defined
> method for implementations to bounce over-sized stanzas?

Certainly not.

> I understand

I think that if individual deployments want to have a maximum stanza
size that the RFC should not forbid this. I think that the RFC should
not be mandating figures for a maximum stanza size on the network. My
two suggestions would be to either leave the errors in the RFC, but
remove the numbers, and then have the XSF write up a XEP for
negotiating stanza size limits, or to include this in the RFC. My
preference is for the former, because I don't think it's vital for the
implementation of XMPP-CORE in a large number of situations.

/K

---

From: Matthew Wild <[email-masked]>
Date: Thu, 24 Jun 2010 16:37:32 +0100

> one large stanza, or two half-sized stanzas are sent - the CPU hit is

Yes, it is - but it doesn't tie the server up with one user/one
stream/one stanza for an undefined period of time - it's still a DoS
vector (though I appreciate it is dependent upon implementation
details, it would affect most implementations out there).

> even if you're able and happy to route non-tiny stanzas, no-one will

That's why we should push an extension through for discoverable limits.

> Aside: RAM, possibly, but not address space - stuff would just get swapped out.

Yes, we all know how great it is for a large service to depend on swap :)

> (<resource-constrained/>?). I believe this is a different problem to

Yes, that would be something a smart server could do. However then it
allows one user to use the server's resources up to that limit, and
DoS other users who will have stanzas bounced because they have no way
of knowing what their ever-changing size limit is.

> server traffic for near-real-time chat for example.

I agree, they were after low-latency stanza deliverance... I'd say
most people using XMPP are.

> it on to the targets, because it'd be larger than the assumed size

Yes, this is indeed tricky to solve - but is tangential to whether we
recommend a minimum size in the RFC - implementation-defined and
discoverable limits will have their own difficulties.

> I think that's orthogonal to this discussion.

I don't think it is directly related, no - but it is related. I
vaguely recall some community discussion around that choosing figure
for IBB, and it serves as an example of precedent on this issue.

> implementation of XMPP-CORE in a large number of situations.

I, however, do believe it is important. There are already real issues
on the network resulting from stanzas being dropped en-route by
implementations enforcing size limits. By leaving us with no "safe"
size as a guideline, these issues shall remain for a long time.

Regards,
Matthew

---

From: Kevin Smith <[email-masked]>
Date: Thu, 24 Jun 2010 17:03:22 +0100

> size as a guideline, these issues shall remain for a long time.

Ah, but that's different and I'm happy with it.
I'm happy to have in the RFC that the size should not be less than X
(64k is fine), so that an entity will always be able to send a stanza
at less than this size and know it'll be below limit. If you only mind
about defining a safe size for sending, I'm happy for

"The maximum stanza size SHOULD NOT be
  smaller than 65536 bytes, from the opening "<" character to the
  closing ">" character."

To stay and

"The maximum stanza
  size acceptable to a remote entity SHOULD be assumed to be 65536"

To either go, or be changed to

"The maximum stanza
  size acceptable to a remote entity SHOULD be assumed to be no less than 65536"

(which is equivalent to the first part).

/K

---

From: Matthew Wild <[email-masked]>
Date: Thu, 24 Jun 2010 18:23:41 +0100

> at less than this size and know it'll be below limit.

Riiight... I'm starting to see your logic now.

> If you only mind

Ok, that's clear what you want now, thanks. But why should any entity
*assume* another entity's size limit is greater than 65536? That would
be risky (especially now we have the option of closing the stream) if
it wants the stanza handled/routed and wants to keep its stream alive.

I understand there may be cases you /know/ sending a large stanza
would be ok. That in my opinion does not prevent the RFC from
specifying this as a "SHOULD". In fact the text isn't even saying the
SHOULD keep its stanzas small - it's just saying that it SHOULD assume
that to be the limit, and if it goes over without any indication that
it is safe to then it should be prepared to face the consequences.

Matthew

---

From: Kevin Smith <[email-masked]>
Date: Thu, 24 Jun 2010 18:57:05 +0100

> it wants the stanza handled/routed and wants to keep its stream alive.

If I want to send a 65k stanza from my home account to my jabber.org
account, there's just no-way to do that with the proposed text,
because even thought I know both servers will accept it, my server
doesn't know anything about jabber.org, so shouldn't send it, because
it's assuming that jabber.org has a 64k limit. This seems pointless.

> it is safe to then it should be prepared to face the consequences.

Right, but unless we define an "I know more than you do" child, every
hop has to know about the real limit, and then things get messy.
Although the text says that if you know differently you can assume
different limits, in practice on the Internet you're saying that this
is a hard limit. I don't think the RFC is the place to determine what
a deployment's resource availability is.

I still suggest that we drop the 'should assume' keep the 'can safely
assume is no less than'. There's no sensible way, that I can see, us
getting around sometimes hitting the limit unless we can query every
hop to ask both what its stanza size limit is and what its going to
add to the stanza (if its willing to tell us), doing some maths in the
sender etc. Given that, it doesn't seem unreasonable to attempt trying
the send and having that cause a too-large error, instead of some
intermediary rejecting it with a too-large error to save sending the
stanza and having a too-large error.

/K

---

From: Justin Karneges <[email-masked]>
Date: Thu, 24 Jun 2010 11:41:25 -0700

On Wednesday 23 June 2010 18:16:52 Peter Saint-Andre wrote:
>    The maximum stanza size SHOULD NOT be
>    closing ">" character.

Hmm, the previous revision said "MUST NOT" here instead of "SHOULD NOT".  The 
text should remain a "MUST NOT" in order to be worthwhile, I think.

By the way, 10000 bytes was chosen as the smallest maximum in the previous 
revision because that was jabberd1's original maximum, and so it seemed 
reasonable to assume that no existing implementation (client or server!) had 
a smaller maximum than that, even mobile apps.  However, it's less clear to 
me if 64K is a safe value.  Can anyone comment on this?

-Justin

---

From: Matthew Wild <[email-masked]>
Date: Thu, 24 Jun 2010 19:50:53 +0100

On 24 June 2010 18:57, Kevin Smith <[email-masked]> wrote:
> assume is no less than'.

One says that sending over 65K is risky, one says that sending under
65K is safe - I guess I still don't see a need for change.

> There's no sensible way, that I can see, us
> sender etc.

I don't think it's needed to go to those lengths. A sensible
implementation could limit stanzas received over c2s, and then have a
higher limit for s2s.

Except it couldn't, because according to the spec no server should
expect another server to allow it. In which case your wording makes
more sense.

So... +1 to "size acceptable to a remote entity SHOULD be assumed to
be no less than 65536" I guess (though Justin's mail came as I was
writing this, and deserves separate consideration).

> Given that, it doesn't seem unreasonable to attempt trying

Yes, that's fine - any error bounce would be routed correctly to the
sending client, so all would work.

Matthew

---

From: Matthew Wild <[email-masked]>
Date: Thu, 24 Jun 2010 20:14:29 +0100

> text should remain a "MUST NOT" in order to be worthwhile, I think.

I strongly disagree. If you know the recipient is happy to receive
large stanzas, or are willing to accept a stanza or stream error, I
see no reason you should be forbidden from sending anything you like.

> me if 64K is a safe value.  Can anyone comment on this?

Any limit from 10K-65K would be fine for me, I'd just like to see some
baseline laid down in the absence of any discoverable limit.

In fact in light of this I'd probably lean towards 10K. When we have a
standard for advertising limits then an implementations/deployments
can push it higher as they desire, but they wouldn't be able to lower
it with this text in place. In that sense it seems to make more sense
to be conservative here about what we essentially require
implementations/deployments to handle.

Matthew

---

From: Justin Karneges <[email-masked]>
Date: Thu, 24 Jun 2010 12:55:22 -0700

On Thursday 24 June 2010 12:14:29 you wrote:
> > Hmm, the previous revision said "MUST NOT" here instead of "SHOULD NOT".
> >  The text should remain a "MUST NOT" in order to be worthwhile, I think.
>

I think you misread my message.  It has nothing to do with restricting 
sending.

> > By the way, 10000 bytes was chosen as the smallest maximum in the
> > previous revision because that was jabberd1's original maximum, and so it
> > seemed reasonable to assume that no existing implementation (client or
> > server!) had a smaller maximum than that, even mobile apps.  However,
> > it's less clear to me if 64K is a safe value.  Can anyone comment on
> > this?
>

Agreed, I'm leaning on 10K as well, especially if we're going to make this a 
MUST (as I wrote above).

-Justin

---

From: Matthew Wild <[email-masked]>
Date: Thu, 24 Jun 2010 21:38:53 +0100

> I think you misread my message.  It has nothing to do with restricting

Sorry, I guess I did. However I think I'm still against it for the
same reasons. There may be cases where I have a small specialised
server and *know* my particular clients will never send stanzas over
4K in size. I see no reason to say this is no longer valid XMPP, do
you?

Matthew

---

From: Justin Karneges <[email-masked]>
Date: Thu, 24 Jun 2010 14:09:28 -0700

On Thursday 24 June 2010 13:38:53 you wrote:
> >> > Hmm, the previous revision said "MUST NOT" here instead of "SHOULD
> >> > NOT". The text should remain a "MUST NOT" in order to be worthwhile, I
> >> > think.
> >>

It's useful to have a guaranteed minimum in the absence of having to do any 
negotiations.  Kind of like how IPv4 has a minimum MTU of 576 bytes:

From RFC 791:
  All hosts must be prepared to accept datagrams of up to 576 octets (whether
  they arrive whole or in fragments).  It is recommended that hosts only send
  datagrams larger than 576 octets if they have assurance that the destination
  is prepared to accept the larger datagrams.

Funny, it looks a lot like our text. :)  I think it should be fine to have a 
minimum as long as it is conservative.

-Justin

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 24 Jun 2010 15:13:38 -0600

> 4K in size. I see no reason to say this is no longer valid XMPP, do

Specialized deployments could go lower and still offer valid XMPP, as
long as we make this a SHOULD. Careful wording is the key. :)

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Matthew Wild <[email-masked]>
Date: Thu, 24 Jun 2010 22:16:36 +0100

> long as we make this a SHOULD. Careful wording is the key. :)

Indeed, that's what I'm arguing - they wouldn't be if we changed
SHOULD to MUST as Justin is suggesting.

Matthew

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 24 Jun 2010 15:23:19 -0600

> SHOULD to MUST as Justin is suggesting.

If we make it 10k then someone is going to complain about how they have
a specialized deployment scenario where a max of 4k is all they can
allow. If we then make it 4k someone else is going to complain about
their even more specialized deployment scenario with a 1k limit. As
admin of the jabber.org service I can recall plenty of people asking if
they could send stanzas bigger than 500k (which was our original limit).
So much of this depends on the deployment scenario that we need to find
a middle ground, preferably based on operational experience. The
justification here is preventing denial of service attacks. Could
someone launch a denial of service attack by sending 500k stanzas? I'm
pretty sure they could. How about 64k? Less likely. I'm not sure that we
need to go all the way down to 10k or 5k or whatever, especially given
the existence of things like vCards and large service discovery results.

But I admit that I have not yet read the entire thread here...

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Peter Saint-Andre <[email-masked]>
Date: Sun, 27 Jun 2010 13:46:06 -0600

> the existence of things like vCards and large service discovery results.

Here again we had similar text in two places (always a bad idea).
Therefore I have removed Section 8.5 ("Stanza Size") and have adjusted
the text in Section 13.12 ("Denial of Service") as follows:

   4.  A server implementation SHOULD enable a server administrator to
       limit the size of stanzas it will accept from a connected client
       or peer server (where "size" is inclusive of all XML markup as
       defined in Section 2.4 of [XML], from the opening "<" character
       of the stanza to the closing ">" character).  An entity's maximum
       stanza size SHOULD NOT be smaller than 65536 bytes, and an entity
       SHOULD assume that the maximum stanza size acceptable to a stream
       peer or remote entity is 65536 bytes unless it has specific
       knowledge to the contrary.  If a connected resource or peer
       server sends a stanza that violates the upper limit, the
       receiving server MUST either return a <policy-violation/> stanza
       error (thus allowing the sender to recover) or close the stream
       with a <policy-violation/> stream error.

Continued feedback is welcome.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Kevin Smith <[email-masked]>
Date: Sun, 27 Jun 2010 22:11:58 +0100

> Continued feedback is welcome.

I think we had agreement earlier in the thread to remove the

"SHOULD assume that the maximum stanza size acceptable to a stream
peer or remote entity is 65536 bytes"

and replace with
"size acceptable to a remote entity SHOULD be assumed to
be no less than 65536" I guess " or to simply remove that text.

/K

---

From: Peter Saint-Andre <[email-masked]>
Date: Sun, 27 Jun 2010 15:16:58 -0600

> be no less than 65536" I guess " or to simply remove that text.

Given that we currently have no way to discover the maximum stanza size
for a remote entity, I'd be OK with deleting that clause.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Matthew Wild <[email-masked]>
Date: Mon, 28 Jun 2010 07:54:25 +0100

> for a remote entity, I'd be OK with deleting that clause.

Delete, or reduce the limit back to 10K (to match jabberd) and re-word
as Kevin suggests.

I think I'd be ok with either if we work on a XEP for discoverability
of this value.

Matthew

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 28 Jun 2010 07:56:05 -0600

> Delete, or reduce the limit back to 10K (to match jabberd) and re-word

Are we still using jabberd 1.x as the touchstone? That seems rather
atavistic at this point. And does jabberd 1.x really have a 10k limit as
the default? I don't recall that from when we ran it at jabber.org.

Is the objection to the 65536-byte recommendation, or to assuming that
implementations and deployments will typically have that as the limit?

> I think I'd be ok with either if we work on a XEP for discoverability

We had http://xmpp.org/extensions/inbox/stanzalimits.html but the XMPP
Council objected to publishing it long ago. Perhaps the current Council
could see fit to fixing that oversight. :)

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Jack Moffitt <[email-masked]>
Date: Mon, 28 Jun 2010 09:26:36 -0600

> server traffic for near-real-time chat for example.

Kevin has it right. It was no problem for the server to shuttle large
amounts of data around. However, because there is only one stream in
each direction, the latency on future stanzas when a big stanza is
being transferred makes it pretty bad for the user, or any machine
that is expecting fast responses.

Considering that a client cannot control all of the stanzas it will
receive, it is at the mercy of the network to be well behaved if it
wants to keep latency down.

It's perhaps best to address this issue another way in the future, but
it is a practical problem.

The other issue with large stanzas is responses to disco#items and
jabber:iq:roster queries. Those can get quite large, and without any
kind of built in reply splitting, can arbitrarily disable certain
kinds of interactions.

jack.

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 28 Jun 2010 10:06:14 -0600

> wants to keep latency down.

True.

> jabber:iq:roster queries. 

Don't forget vCards with large photos.

> Those can get quite large, and without any

Roster versioning (now in 3921bis) helps with the large roster problem.
We need a similar feature for disco#info results (e.g., large numbers of
chatrooms). But those are problems for another standards organization to
solve. :)

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Matthew Wild <[email-masked]>
Date: Mon, 28 Jun 2010 17:08:43 +0100

>>>>>>>>>>>> On Wednesday 23 June 2010 18:16:52 Peter Saint-Andre wrote:

That's what I call a discussion.

> the default? I don't recall that from when we ran it at jabber.org.

Ok:

- I would rather a recommended size limit than none, but I wouldn't
object to all mentions of a specific size being removed if consensus
takes it that way.

- I am in favour of Kev's new "size should be assumed to be no less
than" wording, but am again not going to complain if the current text
stays.

- If we keep the limit then I would rather it be quite small (we're
saying *all* XMPP entities to be able to handle this stanza size). I
know 65K isn't that large, but 10K is smaller. It's also the current
limit, why change? Finally Justin says he thinks it was the jabberd1
limit (though we have no evidence for that).

Based on this my preferred text would be:

"SHOULD assume that the maximum stanza size acceptable to a remote
entity SHOULD be assumed to be no less than 10000"

> implementations and deployments will typically have that as the limit?

I object to nothing :)

> could see fit to fixing that oversight. :)

Yes, I wasn't aware of that until it was mentioned in this thread. I'd
love to see it revived. TIme to find some old discussion logs :)

This will be my last post in this thread unless someone directs me a
question or it takes a new direction - I think I've said enough to
place my position on the issue now.

Regards,
Matthew

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 28 Jun 2010 10:26:34 -0600

> That's what I call a discussion.

Indeed. :)

>> the default? I don't recall that from when we ran it at jabber.org.

BTW, as a reminder, here is what's currently in -09rc1:

   4.  A server implementation SHOULD enable a server administrator to
       limit the size of stanzas it will accept from a connected client
       or peer server (where "size" is inclusive of all XML markup as
       defined in Section 2.4 of [XML], from the opening "<" character
       of the stanza to the closing ">" character).  An entity's maximum
       stanza size SHOULD NOT be smaller than 65536 bytes.  If a
       connected resource or peer server sends a stanza that violates
       the upper limit, the receiving server MUST either return a
       <policy-violation/> stanza error (thus allowing the sender to
       recover) or close the stream with a <policy-violation/> stream
       error.

> object to all mentions of a specific size being removed if consensus

I think that mentioning *some* size is helpful so that we have a baseline.

> than" wording, but am again not going to complain if the current text

See above -- that text is gone.

> know 65K isn't that large, but 10K is smaller. 

And 4k is smaller still. :)

Really, 10k is fine with me. It's not a hill for me to die on.

> It's also the current
> limit, why change? 

There is no current limit. We put 10k into earlier versions of 3920bis,
but that was not final. There was no mention of maximum stanza sizes in
3920.

> Finally Justin says he thinks it was the jabberd1

As Philipp pointed out to me via IM, the 10k number came from default
karma settings in jabberd 1.x -- that was not a stanza size limit.

> entity SHOULD be assumed to be no less than 10000"

Or, perhaps more simply:

   An entity's maximum stanza size SHOULD NOT be smaller than
   10000 bytes.

What implementations assume is up to them.

WFY?

---

From: Justin Karneges <[email-masked]>
Date: Mon, 28 Jun 2010 10:01:28 -0700

> karma settings in jabberd 1.x -- that was not a stanza size limit.

Hmm

>    An entity's maximum stanza size SHOULD NOT be smaller than

To drag this thread on: if we're going to use a SHOULD here instead of a MUST, 
then the number we pick doesn't matter all that much.  We may as well go for 
the 64K then, since even those that don't support a size that large wouldn't 
be noncompliant.

Personally I'd prefer a MUST, and a value so low that there would be no reason 
to go lower.  What value is that?  4K?  Certainly not lower than 576 bytes at 
least, since that's below IP constraint. ;)

We could even have two values here.  "SHOULD NOT be smaller than $idealsize 
bytes and MUST NOT be smaller than $baselinesize bytes".

-Justin

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 28 Jun 2010 11:53:13 -0600

> the 64K then, since even those that don't support a size that large wouldn't 

I'm fine with MUST and a smaller value.

> least, since that's below IP constraint. ;)

Not 4K -- that might interfere with in-band bytestreams (XEP-0047). 10K
seems quite reasonable.

> bytes and MUST NOT be smaller than $baselinesize bytes".

But then you probably won't be able to depend on the $idealsize, only
the $baselinesize. "MUST 10K" seems more straightforward.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Ben Campbell <[email-masked]>
Date: Tue, 29 Jun 2010 17:07:39 -0500

On Jun 24, 2010, at 4:09 PM, Justin Karneges wrote:

[...]

> minimum as long as it is conservative.

Without prejudicing this particular discussion, I would be careful holding up decisions made in very old RFCs as precedent. Believe it or not, the IETF has, on occasion, made mistakes ;-)

These days, text of that form is likely to provoke questions of the form of "How could an implementation ever have such an assurance?"  That doesn't necessarily mean the text is wrong, just that we should be prepared to answer that sort of question.

---

From: Dave Cridland <[email-masked]>
Date: Tue, 29 Jun 2010 23:12:30 +0100

On Tue Jun 29 23:07:39 2010, Ben Campbell wrote:
> Without prejudicing this particular discussion, I would be careful  
> holding up decisions made in very old RFCs as precedent. Believe it  
> or not, the IETF has, on occasion, made mistakes ;-)

Well, I believe it only claims to be infallible in conclave - unless  
that was something else.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Ben Campbell <[email-masked]>
Date: Tue, 29 Jun 2010 17:22:37 -0500

> the 64K then, since even those that don't support a size that large wouldn't 

That is a fairly watered down interpretation of SHOULD, in effect no stronger than MAY. 

I realize that that this was probably intended as sarcasm--but their has been some discussion in this thread that concerns me--namely the idea of expanding the compliance space by using weaker normative language. We've been burned before by product managers claiming their products are compliant if they implement (if you are lucky) all of the MUSTs but nothing else.

In the spirit of 2119, I would not consider myself compliant if I ignored a SHOULD unless I also complied with "... but the full implications must be understood and carefully weighed before choosing a different course."

---

From: Jehan Pagès <[email-masked]>
Date: Wed, 30 Jun 2010 14:09:29 +0900

Hi,

> the $baselinesize. "MUST 10K" seems more straightforward.

I don't think we should have both a MUST and a SHOULD because then
such insistance on maximum stanza size sounds to me like you have to
implement one.

- As a first point, I would prefer to remind that a maximum stanza
size is not mandatory (for most small server, I agree it is useful to
prevent attacks, but bigger servers may be better suited to receive
very big stanzas. Moreover we should not assume knowing all the
possible use cases of XMPP). And for this point, I definitely prefer
the previous wording we had « A client or server MAY enforce a maximum
stanza size » than the latter one « A server implementation SHOULD
enable a server administrator to limit the size of stanzas » which
seems to imply that having a maximum size is preferred.

- Then as a second point, write down that if you want to have such
size limit, it "MUST NOT be smaller than 10k" (I am good with this
last wording).

- Also as a last point, I think I saw either in this thread or in
another (but there are so many messages, I cannot find back the
message) a discussion about returning the server's maximum limit as
some kind of contextual information with the error. I think this is a
good idea in order for a client to be able to fix its behavior when it
receives the policy-violation stream/stanza error (which by itself
does not give enough information for the client's implementation to
understand what went wrong, except by guess). The problem of this is
that we need such contextual information to be precisely defined in
the RFC so that it can be understood and interpreted by a machine (and
the stanza error's description does not plan such a use case, which
would yet be very useful for many kind of errors).

Either this or a way to discover the maximum size limit of a specific
server's installation in the service discovery (which would be in fact
the most appropriate way as it is better not to make the size limit
error from the beginning, and after all such a server configuration
can be considered "information about the entity"). But I guess you
might be reluctant to adding such new information to disco's answers.
If so, at least then a contextual information with the error would
worth it.

Jehan

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 30 Jun 2010 10:18:10 -0600

> seems to imply that having a maximum size is preferred.

This says only that a server implementation SHOULD provide a setting
that enables an administrator to set a max stanza size.

In general, MUST/SHOULD/MAY (etc.) apply to software implementations,
not deployed services. The XMPP specifications are now always strict
about this rule, but I'd prefer that we follow it as much as possible.
For instance, a rule like "a server MAY enforce a maximum stanza size"
is ambiguous -- does it mean that a deployed service ("server") is
allowed to limit the stanza size, or that a software implementation is
allowed to have a default maximum stanza size?

> some kind of contextual information with the error. 

http://xmpp.org/extensions/inbox/stanzalimits.html

> I think this is a

We could do this using XEP-0128 or the inbox proposal cited above, but
that's a matter for the XSF, not the IETF. Please bring this up on the
[email-masked] list if desired.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 30 Jun 2010 10:20:06 -0600

> about this rule, but I'd prefer that we follow it as much as possible.

s/now/not/