Subject: [Sipping] new draft on overload control in SIP

From: Jonathan Rosenberg <[email-masked]>
Date: Sat, 25 Feb 2006 15:29:52 -0500

Folks,

I've just submitted an I-D on the topic of overload control in SIP. 
Until it appears in the archives, you can pick it up at:

http://www.jdrosen.net/papers/draft-rosenberg-sipping-overload-reqs-00.txt

The draft talks about the general problem of overload, and points out 
some serious problems with the current 503 mechanism. In particular, the 
current mechanism can amplify network load substantially during periods 
of overload. The draft calls out requirements for a solution.

I believe this is an important topic. Comments are welcome.

Thanks,
Jonathan R.
-- 
Jonathan D. Rosenberg, Ph.D.                   600 Lanidex Plaza
Cisco Fellow                                   Parsippany, NJ 07054-2711
Cisco Systems
[email-masked]                              FAX:   (973) 952-5050
http://www.jdrosen.net                         PHONE: (973) 952-5000
http://www.cisco.com

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP

---

From: "James M. Polk" <[email-masked]>
Date: Sat, 25 Feb 2006 15:13:56 -0600

Jonathan

Excellent ID

This problem needs to be highlighted

I'm biased, but I also believe this effort ought to account for networks, 
or SIP elements, that use the Resource Priority header to handle congestion 
in an element, even if only in passing that some messages indicated by 
domain or nework or element policy can get processed with elevated 
priority-values.

I fully understand the RPH indication will not be deployed/available 
everywhere, but it will be used in some implementations and domains, and 
those may see this document not accounting for RPH functionality.

Perhaps text saying how RPH would work if deployed, or why it wouldn't work 
if deployed should be in the next version IMO to have a discussion point, 
for completeness of the overall topic of congestion handling.

RPH aside, I've been thinking for about 2 years of a new OVERLOAD Request 
to address this topic, but I cannot get past the obvious 
security/misuse/DDOS considerations of "how do I shut down a SIP network? I 
use a standardized Method"

ugh

Regarding the problems outlined near Figure 3, would something within (or 
an extension to) the History Info header help?

At 03:29 PM 2/25/2006 -0500, Jonathan Rosenberg wrote:
>Folks,
>
>I've just submitted an I-D on the topic of overload control in SIP. Until 
>it appears in the archives, you can pick it up at:
>
>The draft talks about the general problem of overload, and points out some 
>serious problems with the current 503 mechanism. In particular, the 
>current mechanism can amplify network load substantially during periods of 
>overload. The draft calls out requirements for a solution.
>

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Mon, 27 Feb 2006 10:05:10 -0600

> I believe this is an important topic. Comments are welcome.

Jonathan: This is a long time in the making.  A few observations
on what we've seen in deployments; this may trigger more requirements.

One, in some networks, we've seen the opposite of what you have
in Figure 2.  That is, P1 (or a load distributer or a SBC) views S1
through S3 as one logical node.  So, even if S1 issues a 503/R-A, it
considers the whole cluster busy and will not send traffic to
S2 and S3.  It seems appropriate to have a requirement that
load balancers should have a more granular view on the downstream
elements that it is distributing the load to.

Second, we have observed beneficial results if certain requests are
allowed, even during overload conditions.  Specifically, requests
like BYEs, (gulp) CANCELs, and SUBS w/ Expires=0 since these tend
to free up resources on the overloaded host.  Thus, another
requirement may be along the lines of an overloaded host may
processing a request if that request frees up resources.

And finally, when a mechanism is put in place, it should be of
MUST strength.  One of the problem we've faced is that many
implementations do not implement 503/R-A since it is a SHOULD
strength.  Some implementations that honor it tend not to send a
request in the same *dialog* to the overloaded host, but have no
such qualms on sending *new* requests to the overloaded host.
Thus, another requirement -- as transparent as it sounds -- may be
the fact that overloaded condition applies to an entire host and
not a transaction or dialog targeted to that host.  As a corollary,
a request that free's up resources on the overloaded host may be
allowed (account for the requirement in the previous paragraph).

Thanks,

- vijay
-- 
Vijay K. Gurbani  vkg@{lucent.com,research.bell-labs.com,acm.org}
Bell Laboratories, Lucent Technologies, Inc.
2701 Lucent Lane, Rm. 9F-546, Lisle, Illinois 60532 (USA)
_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP

---

From: Juha Heinanen <[email-masked]>
Date: Mon, 27 Feb 2006 22:14:36 +0200

Vijay K. Gurbani writes:

 > strength.  

i agree.  recently i complained to a major vendor that jonathan well
knows about an issue that in rfc3261 is SHOULD, and got promptly back
reply that because the feature is SHOULD and not MUST, they are not
going to do anything about it.  based on these kind of experiences, i
have come to the conclusion that RFCs MUST not contain any other
requirements than MUST requirements, because in practise SHOULDs and
MAYs are worth of nothing.

-- juha

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Mon, 27 Feb 2006 16:01:08 -0800

> MAYs are worth of nothing.

Before actually reading RFC 2119 I thought that "Be conservative in what
you send and liberal in what you receive" was to be applied to the
SHOULD word, i.e. that a SHOULD was a MUST when you apply it to what you
send, and a MAY when you apply it to what to receive.

I was young.

-- 
Marc Petit-Huguenin
Home: [email-masked]
Work: [email-masked]

---

From: "Dan Wing" <[email-masked]>
Date: Mon, 27 Feb 2006 17:43:27 -0800

> MAYs are worth of nothing.

It could help the situation if documents containing SHOULDs had discussion
text explaining the cases where and why it's a SHOULD rather than a MUST
(and likewise, why it's a MAY instead of a SHOULD).  However, doing this
takes a lot of effort.

Even if you did make everything a MUST, vendors will ignore MUSTs anyway.
For example commonly-deployed NATs and personal firewalls cause hosts to
exist on the Internet which violate the host requirement RFC (RFC1122) by
not responding to pings or TCP SYNs.  The argument for non-compliance will
change -- rather than today's "it's just a SHOULD, quit bugging us", it will
morph to "our product works in a different environment than the environment
considered by the author so we don't follow that MUST.  Quit bugging us."

I've seen it from both sides - our products trying to interact with someone
else who didn't follow a MUST, and our products not following a MUST.  It is
especially bothersome when a standards author is an employee of the same
company that is in violation, but all too often that doesn't directly
translate into an ability to affect code in products.

-d

---

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 27 Feb 2006 21:27:50 -0500

inline.

> (and likewise, why it's a MAY instead of a SHOULD).  However, doing this

I agree completely with Dan here. I've done this in various places, but 
its very difficult to do. The point though, is that ultimately, 
implementors do what they want and/or can get away with. The only way to 
really sway people is to clearly explain the pros and cons of a SHOULD 
choice (or even a MUST that you don't think they'll like). The prose is 
worth a million MUSTs.

-Jonathan R.

---

From: Juha Heinanen <[email-masked]>
Date: Tue, 28 Feb 2006 09:58:43 +0200

> send, and a MAY when you apply it to what to receive.

the last week's thing i was referring to is display name in From header
from anonymous caller, which according to rfc3261 SHOULD be "Anonymous".
if major vendor puts where something else, how would your liberalism
help to detect that the caller wants to be anonymous?

-- juha

---

From: Juha Heinanen <[email-masked]>
Date: Tue, 28 Feb 2006 10:03:22 +0200

> Even if you did make everything a MUST, vendors will ignore MUSTs

that is not the case with this major vendor i'm talking about.  their
engineer replied that he cannot issue a bug report on ANYTHING that is
SHOULD or MAY in ANY rfc.  he can do it, if rfc has MUST.

 > considered by the author so we don't follow that MUST.  Quit bugging

than according to consumer protection laws in any decent country, the
customer would at least be able to get his money back, but you are
unlikely to have any luck with SHOULDs and MAYs.

- juha

---

From: Juha Heinanen <[email-masked]>
Date: Tue, 28 Feb 2006 10:06:57 +0200

Jonathan Rosenberg writes:

 > implementors do what they want and/or can get away with.

this major vendor has OFFICIAL policy that they don't accept bug reports
against SHOULD and MAY violations. it ha nothing to do with implementors
who write the code.  i have got an impression that the implementors of
this major company are not ALLOWED to fix such bugs even if they want
to.

-- juha

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 28 Feb 2006 08:20:52 -0500

> SHOULD or MAY in ANY rfc.  he can do it, if rfc has MUST.

That's pretty short sighted. But in some sense it is understandable as 
long as the draft isn't clear about what conditions do (and don't) 
justify violation of the SHOULD. It puts more of a premium on doing that.

	Paul

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Tue, 28 Feb 2006 08:59:18 -0600

> choice (or even a MUST that you don't think they'll like). The prose is 

At the same time, I don't think we can simply sweep certain extenuating
cases under the rug.  While I agree that explaining why a behavior is
SHOULD vs. MUST is prone to error, we should -- at the very least --
learn from our mis-steps.  Given that we have a fair amount of rfc3261-
deployment, implementation, and operational experience, we can now
choose to provide a much stronger normative language on certain
features.  We were not able to do this 4-5 years ago when we did
not have the experience that we do now.

To solve overload control we know that anything short of a MUST will
cause problems.  Thus, if we still went ahed with a SHOULD or MAY,
we have no one to blame but ourselves when we see interoperability
problems (not to mention DoS attacks becuase someone's proxy did
not implement O/C since it was a SHOULD).

I am sure there are a few other cases.  One thing that comes to
mind is the interaction of loop detection and Max-Forwards (M-F).
If implementations leave it to M-F to terminate loops, then the
kind of attacks demonstrated in draft-lawrence-maxforward-problems
are possible.  Thus, leave loop detection as a SHOULD, but warn
people with a documented information RFC that this is what will
happen if you did not implement loop detection.

Again, we now have more data to make informed decisions going
forward, and wherever possible, we should leverage the experience
to tighten up the normative language in SIP.

Thanks,

- vijay

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 28 Feb 2006 10:39:33 -0500

This particular feature in SIP - using a display name of "Anonymous" was 
quite a bad idea in the end. I have proposed a fix to this which would 
make a consistent and unambiguous way for a URI to indicate its anonymous:

http://www.jdrosen.net/papers/draft-rosenberg-sip-identity-privacy-00.txt

-Jonathan R.

---

From: "Dan Wing" <[email-masked]>
Date: Tue, 28 Feb 2006 07:48:41 -0800

> SHOULD or MAY in ANY rfc.  he can do it, if rfc has MUST.

That seems to call for wording like:

  Implementations MUST do <foo>, unless <condition>.

> than according to consumer protection laws in any decent country, 
> the customer would at least be able to get his money back, but 
> you are unlikely to have any luck with SHOULDs and MAYs.

Getting your money back doesn't get interoperability, though. 
Interoperability is what interests me.

Most consumer-grade devices don't claim compliance with RFCs, 
at least not that I have seen on their external packaging or
their user documentation.

-d

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Tue, 28 Feb 2006 10:02:47 -0600

> their user documentation.

Not to berate the point, but many consumer devices do claim
compliance with IEEE standards.  And I am sure the same problems
we face here in IETF over protocol inter-operability occur in
IEEE as well.  Unfortunately, I do not follow IEEE standards closely
so I am unable to gauge how they ensure successful and
ubiquitous interoperability.  Maybe someone else who is plugged
into the IEEE standards can provide more input.

Thanks,

- vijay

---

From: Dean Willis <[email-masked]>
Date: Tue, 28 Feb 2006 17:12:47 -0600

> this major company are not ALLOWED to fix such bugs even if they want

You have my blessing to call them idiots, even if I happen to work  
there. Of course, I doubt you ever needed my blessing for that . . .  
What's the Finnish word for idiot, anyhow? I'm sure I've overheard it  
in some of those standards discussions after I've said something  
"controversial", but it would be nice to know. One can never know too  
many insults, they're such handy words.

Seriously, we have historically badly over-used the requirements  
language of RFC 2119. We've often used it to describe how things  
normally work. It should be used only to explain the consequences of  
deviations from normalcy. Note that simply saying "MUST do X" or  
"SHOULD do Y" doesn't really explain these consequences.

IMHO, each usage of 2119 language outside of a "pure requirements  
document" (and arguably even in those) needs to be accompanied by  
justification.

MUST: If you deviate from this, the indicated dire consequences are  
very likely to occur, and no work-around or alternative is defined.  
The default dire consequences here are "protocol simply won't work",  
but usually more is needed.

SHOULD: If you deviate from this (which you might need to do for some  
use cases), you need to take care of some extra points (described  
here), or the indicated dire consequences are very likely to occur.

MAY: Here's some extra stuff you might choose to do, if you've dealt  
with the consequences, which including making sure that the other  
affected elements are prepared to deal with this optional behavior.  
If you fail to verify that other affected elements are prepared to  
deal with the optionality, you must be prepared to deal with them  
either ignoring the optionality, rejecting the request, or failing in  
an unpredictable way, possibly catastrophic.

Good example, from draft-ietf-sipping-uri-services-05.txt:

    Attackers may attempt to send a URI-list containing URIs whose host
    parts route to the victims of the DoS attack.  These victims do not
    need to be SIP nodes; they can be non-SIP endpoints or even routers.
    If this attack is successful, the result is that an attacker can
    flood with traffic a set of nodes, or a single node, without needing
    to generate a high volume of traffic itself.

    There are several measures that need to be taken to prevent this  
type
    of attack.  The first one is keeping unauthorized users from using
    URI-list services.  So, URI-list services MUST NOT perform any
    request explosion for an unauthorized user.  URI-list services MUST
    authenticate users and check whether they are authorized to request
    the service before performing any request fan-out.

This actually places one requirement, redundantly using a MUST and a  
MUST NOT with opposing descriptions, backed up by a description of  
what will go wrong if you fail to meet the requirement --  
susceptibility to a range of attacks.

Bad example from draft-ietf-sipping-uri-list-message-05.txt:

    A UAC that wants to create a multiple-recipient MESSAGE request MUST
    create a MESSAGE request according to RFC 3428 [8] Section 4.  The
    UAC SHOULD populate the Request-URI with the SIP or SIPS URI of the
    MESSAGE URI-list service.  In addition to the regular instant  
message
    body, the UAC SHOULD add a URI-list body whose Content-Disposition
    type is 'recipient-list', specifed in the Framework and Security
    Considerations for SIP URI-list Services [11].  This body contains a
    URI-list with the recipients of the MESSAGE.  The URI-list body MAY
    also include the 'capacity' extension to the URI-list specified in
    Section 4.1.  The UAC MUST also include the 'recipient-list-message'
    option-tag, defined in Section 5, in a Require header field.

Let's look at the "requirements" in the above.

MUST #1 (create a message request): Why MUST we create a message  
request? What failure of interoperability occurs if we don't create a  
message request? We're really trying to say that this service only  
understands MESSAGE requests as defined in RFC 3428, and that it  
won't work if you were to use some other kind of request.

SHOULD #1 (populate the request URI): What are the alternatives? What  
would we want to put in the request URI other than the URI of the  
target? What happens if we were to put something else here, and how  
would we compensate if we did? Really, we're not trying to define a  
requirement here -- we're just describing a use case.

SHOULD #2 (add a URI-list body): Again, what are the alternatives?  
We're not defining any here. We're just trying to describe a use case.

MAY #1 (also include the 'capacity' extension): Why would we want to  
do that? Fortunately, the "why" is defined in the referenced section  
4.1. But are we stating an aspect of optionality that affects  
interoperability? Nope. We're just describing some information that,  
if present in the request, will be used in processing the request at  
the recipient, and if it's not there, it won't be.

MUST #2 (also include the 'recipient-list-message" option tag): It  
would be nice to say WHY we do this, although we use this in SIP so  
much we seem to expect the reader to know. In short, we're using the  
extension-negotiation mechanism of SIP to make sure that the  
recipient understands this extension. If we include the option-tag,  
then a recipient of this request will reject the request with an  
informative error if it does not understand the extension.

My suggested rewrite:

    A UAC that wants to create a multiple-recipient MESSAGE request
    creates a MESSAGE request according to RFC 3428 [8] Section 4.  The
    UAC populates the Request-URI with the SIP or SIPS URI of the
    MESSAGE URI-list service.  In addition to the regular instant  
message
    body, the UAC adds a URI-list body whose Content-Disposition
    type is 'recipient-list', specifed in the Framework and Security
    Considerations for SIP URI-list Services [11].  This body contains a
    URI-list with the recipients of the MESSAGE.  The URI-list body may
    also include the 'capacity' extension to the URI-list specified in
    Section 4.1.  The UAC MUST also include the 'recipient-list-message'
    option-tag, defined in Section 5, in a Require header field.

Note that the only 2119 language retained here is the final MUST on  
extension negotiation. And only the hope that the negotiation  
mechanism is well-understood justifies our not saying more here. Were  
I to re-write this again, I would change the closing sentence to be:

    The UAC MUST also include the 'recipient-list-message'
    option-tag (defined in Section 5) in a Require header field,
    which will result in rejection of this request by any UAS
    that does not implement this extension.

Now, Miguel didn't take all my suggestions. Instead, he produced the  
following in -07

    A UAC that wants to create a multiple-recipient MESSAGE request
    creates a MESSAGE request that MUST be formatted according to RFC
    3428 [8] Section 4.  The UAC populates the Request-URI with the SIP
    or SIPS URI of the MESSAGE URI-list service.  In addition to the
    regular instant message body, the UAC adds a URI-list body whose
    Content-Disposition type is 'recipient-list', specified in the
    Framework and Security Considerations for SIP URI-list Services  
[11].
    This body contains a URI-list with the recipients of the MESSAGE.
    Target URIs in this body MAY also be tagged with the 'capacity' and
    'anonymize' attributes specified in the XML Format Extension for
    Representing Capacity Attributes in Resource Lists [12].  The UAC
    MUST also include the 'recipient-list-message' option-tag,  
defined in
    Section 5, in a Require header field.

He still has two MUSTs that one has to dereference (but there ARE  
references given), and a MAY (which is also referenced). All the 2119  
language that doesn't have a real functional requirement attached to  
it has been removed. While it's not as pretty as mine (opinion), it's  
at least testable for compliance, and the important aspects of the  
"required" behavior can at least be extracted from the specification.

--
Dean

---

From: Dean Willis <[email-masked]>
Date: Tue, 28 Feb 2006 17:15:32 -0600

>   Implementations MUST do <foo>, unless <condition>.

That's a textbook SHOULD formulation. The vendor here appears to be  
misreading the spec or misunderstands the implications of 2119 language.

--
Dean