Subject: [MEDIACTRL] MUST vs. SHOULD vs MAY

From: "Mary Barnes" <[email-masked]>
Date: Sun, 23 Nov 2008 15:36:02 -0600

Hi folks,

Based on the debate on this topic between myself and Keith, I went
digging for the past discussion on this topic in gen-art. While gen-art
certainly isn't the "owner" of RFC 2119, we likely do spend more time on
this topic than WGs in general. 

This goes back to RFC 2119, which suggests to use "MUST" sparingly.   

This is described in section 6: "Guidance in the use of these
Imperatives"
" Imperatives of the type defined in this memo must be used with care
and sparingly. In particular, they MUST only be used where it is
actually required for interoperation or to limit behavior which has
potential for causing harm (e.g., limiting retransmisssions) For
example, they must not be used to try to impose a particular method on
implementors where the method is not required for interoperability."

Noting also that in the description of "MAY" RFC 2119 has specific
recommendations in terms of interoperability:
Section 5. MAY"
"This word, or the adjective "OPTIONAL", mean that an item is truly
optional. One vendor may choose to include the item because a particular
marketplace requires it or because the vendor feels that it enhances the
product while another vendor may omit the same item. An implementation
which does not include a particular option MUST be prepared to
interoperate with another implementation which does include the option,
though perhaps with reduced functionality. In the same vein an
implementation which does include a particular option MUST be prepared
to interoperate with another implementation which does not include the
option (except, of course, for the feature the option provides.)"
------------------------------------------------------------------------
---------------------------------------------------------

While I fully agree that some implementors will very, very often
primarily focus on MUSTs and might just not add any code whatsoever for
SHOULDs, MAYS, etc. - those implementations are broken, per section 5.
We can't be protocol police and I have spent time telling developers
that they need to consider SHOULDs - in particular ensure that if they
don't follow the recommendation that they do ensure the NOT really
applies to their situation.  

It's completely impossible for us to write a spec that ensures 100%
interoperability (our documents are defining the protocol and they are
NOT interop specs) - that's the reason for SIPits, etc. and why it is
highly unlikely that anyone can plug in another vendor's equipment and
everything works perfectly the first time around. And, we are not the
protocol police and  no matter how we write a specification there is no
way for us to enforce 100% compliance to all MUSTs, MAYs, SHOULDs etc. I
do realize this makes life difficult for Product managers, s/w managers,
other SDOs, etc., that's the way it is. 

As a point in case, I would love to hear from SIPit as to whether ANYONE
has even implemented every single MUST in RFC 3261 or whether they
properly follow the spec in terms of SHOULDs.

Regards,
Mary. 

_______________________________________________
MEDIACTRL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/mediactrl
Supplemental Web Site:
http://www.standardstrack.com/ietf/mediactrl

---

From: Eric Burger <[email-masked]>
Date: Sun, 23 Nov 2008 17:13:43 -0500

Part of the problem is RFC 3261 has well over 500 MUSTs. I don't think  
mediactrl is getting that complex.  The SIPit experience is anytime  
there is a SHOULD, bad things happen.  Yes, some folks do not  
implement MUSTs. They break. Marketplace fixes.

On Nov 23, 2008, at 4:36 PM, Mary Barnes wrote:

> digging for the past discussion on this topic in gen-art. While gen- 
> art certainly isn't the "owner" of RFC 2119, we likely do spend more  
> time on this topic than WGs in general.
>
> " Imperatives of the type defined in this memo must be used with  
> care and sparingly. In particular, they MUST only be used where it  
> is actually required for interoperation or to limit behavior which  
> has potential for causing harm (e.g., limiting retransmisssions) For  
> example, they must not be used to try to impose a particular method  
> on implementors where the method is not required for  
> interoperability."
>
> optional. One vendor may choose to include the item because a  
> particular marketplace requires it or because the vendor feels that  
> it enhances the product while another vendor may omit the same item.  
> An implementation which does not include a particular option MUST be  
> prepared to interoperate with another implementation which does  
> include the option, though perhaps with reduced functionality. In  
> the same vein an implementation which does include a particular  
> option MUST be prepared to interoperate with another implementation  
> which does not include the option (except, of course, for the  
> feature the option provides.)"
>
> ---------------------------------------------------------------------------------------------------------------------------------
>
> primarily focus on MUSTs and might just not add any code whatsoever  
> for SHOULDs, MAYS, etc. - those implementations are broken, per  
> section 5.  We can't be protocol police and I have spent time  
> telling developers that they need to consider SHOULDs - in  
> particular ensure that if they don't follow the recommendation that  
> they do ensure the NOT really applies to their situation.
>
> interoperability (our documents are defining the protocol and they  
> are NOT interop specs) - that's the reason for SIPits, etc. and why  
> it is highly unlikely that anyone can plug in another vendor's  
> equipment and everything works perfectly the first time around. And,  
> we are not the protocol police and  no matter how we write a  
> specification there is no way for us to enforce 100% compliance to  
> all MUSTs, MAYs, SHOULDs etc. I do realize this makes life difficult  
> for Product managers, s/w managers, other SDOs, etc., that's the way  
> it is.
>
> As a point in case, I would love to hear from SIPit as to whether  
> ANYONE has even implemented every single MUST in RFC 3261 or whether  
> they properly follow the spec in terms of SHOULDs.
>
> Regards,
> Mary.
>
>

---

From: "Mary Barnes" <[email-masked]>
Date: Mon, 24 Nov 2008 08:38:06 -0600

Yeah - so SIP may well be a worst case scenario in terms of complexity
and thus potential for interop. My point was that the SHOULDs are just
as important as MUSTs in terms of consideration in product development
to ensure the probability of successful interop.  Defining something to
be a MUST rather than a SHOULD just to ensure it will be implemented is
not the right thing to do. Now, if something in the basic protocol
really won't work if it's not implemented, then certainly it ought to be
a MUST. But, if it's something which might not need to be implemented in
certain environments, then a SHOULD (with the appropriate conditions
described) is the right choice. 

So, I don't see MUSTs ensuring interoperability any more than SHOULDs.
Based on my experience as the protocol matures, original MUSTs often
become SHOULDs because folks realize reasonable situations for which the
MUST really is too restrictive (and inappropriate).  That has certainly
been my experience with our product and past product development that
I've done. 

Just my opinions - YMMV - certainly mediactrl applicability and scope is
fairly straightforward - that in itself dramatically improves the
chances for successful interop.

Mary. 

-----Original Message-----
From: Eric Burger [mailto:[email-masked]] 
Sent: Sunday, November 23, 2008 4:14 PM
To: Barnes, Mary (RICH2:AR00)
Cc: [email-masked]
Subject: Re: [MEDIACTRL] MUST vs. SHOULD vs MAY

Part of the problem is RFC 3261 has well over 500 MUSTs. I don't think
mediactrl is getting that complex.  The SIPit experience is anytime
there is a SHOULD, bad things happen.  Yes, some folks do not implement
MUSTs. They break. Marketplace fixes.

> particular marketplace requires it or because the vendor feels that it

> enhances the product while another vendor may omit the same item.
> include the option, though perhaps with reduced functionality. In the 
> same vein an implementation which does include a particular option 
> MUST be prepared to interoperate with another implementation which 
> does not include the option (except, of course, for the feature the 
> option provides.)"
>
> ----------------------------------------------------------------------
> -----------------------------------------------------------
>
> section 5.  We can't be protocol police and I have spent time telling 
> developers that they need to consider SHOULDs - in particular ensure 
> that if they don't follow the recommendation that they do ensure the 
> NOT really applies to their situation.
>

> protocol police and  no matter how we write a specification there is 
> no way for us to enforce 100% compliance to all MUSTs, MAYs, SHOULDs 
> etc. I do realize this makes life difficult for Product managers, s/w 
> managers, other SDOs, etc., that's the way it is.
>

---

From: "Mary Barnes" <[email-masked]>
Date: Tue, 25 Nov 2008 12:40:07 -0600

I think we all agree on MUSTs always meaning MUST. The problem I have
found is that sometimes folks don't consider all reasonable deployments
or why a MUST might not apply to certain situations. If a SHOULD is
soft, then perhaps it ought to really be a MAY. But, again, either of
these terms, in particular SHOULD ought to always be properly qualified.
And, yes, it can be a pain to qualify SHOULDs, but being too liberal
with MUSTS can make a spec non-compliant from the get go in certain
environments. And, all SHOULDs ought to be properly handled. 

My view is that it's rare that "soft" SHOULDs transition to MUSTs.  My
experience is that MUSTs often get down-graded to SHOULDs as the
protocol progresses, which isn't necessarily a bad thing, but often
catches folks by surprise. My suggestion is to more seriously consider
whether you really want a MUST and query for any applicable SHOULDs
while the doc is in the early stages of development. My experience in
development (of both protocol specs and SW) is that the earlier
something wrong or inconsistent is detected, the far less likely it is
to cause problems down the road.  We went down this rathole with HELD
because we didn't consider VPNs and other intermediaries that we can't
control in the network. This is where we changed several MUSTs to
SHOULDs.  Now, we did end up splitting some requirements into two
sentences - a first part being a MUST and the second being a SHOULD in
certain conditions - although I'm not sure that will fly in the end once
it goes through IESG. And, again, my feedback on this topic is based
upon what seems to happen fairly often with gen-art reviews. 

Again, I know the deployment model for MEDIACTRL is quite restrictive at
this point, but the problem may be if someone does try to deploy it in a
less restrictive environment where they can't implement all the MUSTs
you end up with interop problems from the get go. 

I am just sharing my experiences (and some opinions), so the WG is free
to ignore. But, with my own documents, you're likely to see far more
SHOULDs than MUSTs (except for core protocol elements and the handling
thereof) and I don't want Keith bugging me later ;)

Regards,
Mary. 

-----Original Message-----
From: Alan Hawrylyshen [mailto:[email-masked]] On Behalf Of
Alan Hawrylyshen
Sent: Tuesday, November 25, 2008 12:00 PM
To: Barnes, Mary (RICH2:AR00)
Cc: Eric Burger; [email-masked]
Subject: Re: [MEDIACTRL] MUST vs. SHOULD vs MAY

> [snip] So, I don't see MUSTs ensuring interoperability any more than 
> SHOULDs.
> become SHOULDs because folks realize reasonable situations for which 
> the MUST really is too restrictive (and inappropriate).

I think it is fair to say that this is what Keith was driving at: it is
difficult work to decide 'a priori' what elements of the specification
should be MUST and must be SHOULD.
However, this work is time and effort well spent.

>  That has certainly
> Just my opinions - YMMV - certainly mediactrl applicability and scope 
> is fairly straightforward - that in itself dramatically improves the 

 From my implementation experience, you MUST implement the MUSTs and in
all likelihood, if the spec has been done reasonably, you will have a
workable (but minimal) implementation. The SHOULDs add value and
robustness and with luck, the implementor understands WHY based on the
specification (and requirements) documents.

And yes, YMMV. I still think we should ensure that we've managed to
consider the protocol enough to drive out the 'soft SHOULDs' ...

Alan

--
Alan Hawrylyshen

---

From: Alan Hawrylyshen <[email-masked]>
Date: Tue, 25 Nov 2008 11:59:59 -0600

On Nov 24, 2008, at 08:38 , Mary Barnes wrote:

> [snip]
> the
> MUST really is too restrictive (and inappropriate).

I think it is fair to say that this is what Keith was driving at: it  
is difficult work to decide 'a priori' what elements of the  
specification should be MUST and must be SHOULD.
However, this work is time and effort well spent.

> chances for successful interop.

 From my implementation experience, you MUST implement the MUSTs and  
in all likelihood, if the spec has been done reasonably, you will have  
a workable (but minimal) implementation. The SHOULDs add value and  
robustness and with luck, the implementor understands WHY based on the  
specification (and requirements) documents.

Alan

--
Alan Hawrylyshen

---

From: "DRAGE, Keith \(Keith\)" <[email-masked]>
Date: Wed, 26 Nov 2008 19:27:01 +0100

I really need to get round to looking at these documents properly and
supplying specific comments on the individual RFC 2119 language - I'm
sure that would be far more helpful to the editors. I think I am pretty
close to Mary's last comments on this.

1)	The first and overriding rule must be that RFC 2119 statements
must all be well formed. We use MUST when we want MUST and SHOULD when
we want SHOULD and MAY when we want MAY, and we should be able to
justify every usage. What I have missed in this discussion is that in
the past, I have detected cases where we should not have the RFC 2119
language at all, i.e. write the sentence as ordinary description text.
(Oh and in some cases exactly the same requirement appears once or even
twice in the document due to poor structure, which adds to the count).

2)	We all agree that SHOULD is always qualified, i.e. give guidance
on when it is expected not to apply, or why the condition can be relaxed
from a full SHOULD. Otherwise we could just as well write MAY and have
zero impact on what gets implemented in the field.

3)	I have certainly also detected instances where SHOULD was
written where it was correct to write "IF somewhat complicated condition
THEN MUST...". We do not serve the community by not fully detailing the
condition.

4)	The RFC 2119 language is there for interoperability. This may be
interoperability within the document, but it may also be for forward or
backward compatibility, and interoperability in those conditions.
Therefore a sending condition may be a MUST because we expect other
extensions in the future to depend on that MUST being there. Maybe we
should qualify these with "For reasons of forward compatibility, the ...
MUST..." but I am not sure that will persuade implementors to fulfil
them any better than letting them appear in the raw.

regards

Keith

> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Mary Barnes
> Sent: Tuesday, November 25, 2008 6:40 PM
> To: Alan Hawrylyshen
> Cc: [email-masked]
> I think we all agree on MUSTs always meaning MUST. The 
> problem I have found is that sometimes folks don't consider 
> all reasonable deployments or why a MUST might not apply to 
> certain situations. If a SHOULD is soft, then perhaps it 
> ought to really be a MAY. But, again, either of these terms, 
> in particular SHOULD ought to always be properly qualified.
> And, yes, it can be a pain to qualify SHOULDs, but being too 
> liberal with MUSTS can make a spec non-compliant from the get 
> go in certain environments. And, all SHOULDs ought to be 
> properly handled. 
> 
> My view is that it's rare that "soft" SHOULDs transition to 
> MUSTs.  My experience is that MUSTs often get down-graded to 
> SHOULDs as the protocol progresses, which isn't necessarily a 
> bad thing, but often catches folks by surprise. My suggestion 
> is to more seriously consider whether you really want a MUST 
> and query for any applicable SHOULDs while the doc is in the 
> early stages of development. My experience in development (of 
> both protocol specs and SW) is that the earlier something 
> wrong or inconsistent is detected, the far less likely it is 
> to cause problems down the road.  We went down this rathole 
> with HELD because we didn't consider VPNs and other 
> intermediaries that we can't control in the network. This is 
> where we changed several MUSTs to SHOULDs.  Now, we did end 
> up splitting some requirements into two sentences - a first 
> part being a MUST and the second being a SHOULD in certain 
> conditions - although I'm not sure that will fly in the end 
> once it goes through IESG. And, again, my feedback on this 
> topic is based upon what seems to happen fairly often with 
> gen-art reviews. 
> 
> Again, I know the deployment model for MEDIACTRL is quite 
> restrictive at this point, but the problem may be if someone 
> does try to deploy it in a less restrictive environment where 
> they can't implement all the MUSTs you end up with interop 
> problems from the get go. 
> 
> I am just sharing my experiences (and some opinions), so the 
> WG is free to ignore. But, with my own documents, you're 
> likely to see far more SHOULDs than MUSTs (except for core 
> protocol elements and the handling
> From: Alan Hawrylyshen [mailto:[email-masked]] On 
> Behalf Of Alan Hawrylyshen
> I think it is fair to say that this is what Keith was driving 
> at: it is difficult work to decide 'a priori' what elements 
> of the specification should be MUST and must be SHOULD.
>  From my implementation experience, you MUST implement the 
> MUSTs and in all likelihood, if the spec has been done 
> reasonably, you will have a workable (but minimal) 
> implementation. The SHOULDs add value and robustness and with 
> luck, the implementor understands WHY based on the 
> And yes, YMMV. I still think we should ensure that we've 
> managed to consider the protocol enough to drive out the 
> 'soft SHOULDs' ...
> 
> Alan
> 
> 
> 
> --
> Alan Hawrylyshen
> 
> 
> 
>