Subject: RE: MUSTs and SHOULDs [was RE: [Sipping] new draft on overloadcontrol in SIP]

From: "Jean-Francois Mule" <[email-masked]>
Date: Tue, 28 Feb 2006 13:15:27 -0700

Writing testable requirements is important and hard, and it takes
several iterations after running code is available and interops tests
occur. But there is a reason why RFC2119 is in the normative reference
section of many documents: do we just put it defacto because it comes
with the xml2rfc sample xml source or because it means something?

Some organizations and providers actually tag requirements using
well-known software to generate requirements' database which help us
trace formal requirements in test cases (PICS db) and in automated test
procedures.

>From my own experience (as a doer tagging requirements and as team lead
who forced folks to write test case to learn how to better write
requirements), I have tried two paths to test product compliance with
SIP RFCs. You can
  a) put a MUST req in your own spec to the RFC (as a service provider
would put a MUST in their RFPs) and leave the RFC alone assuming the RFC
is self sufficient on the requirement levels.
or
  b) put a MUST req in your own specs to the RFC, import the RFC text in
a req db tracking system or annex and reclassify some requirements 
(it goes from changing a critical lower case must to MUST, to upgrading
a MAY into SHOULD, or SHOULD into MUST - that said, it should not be
about downgrading requirements from MUST to MAY, RFC erratas should be
used to suggest that).

As usual, it depends on the RFC author and the content of the RFC, but
in my experience, b) leads to better product interoperability. 

Dan wrote:
> That seems to call for wording like:
> 
>   Implementations MUST do <foo>, unless <condition>.
We call that conditional MUST in some companies and they add some values
in between strict MUST and SHOULD.

Jonathan wrote:
> I agree completely with Dan here. I've done this in various places,
> but
> its very difficult to do. The point though, is that ultimately,
> implementors do what they want and/or can get away with. The only way
> to
> really sway people is to clearly explain the pros and cons of a SHOULD
> choice (or even a MUST that you don't think they'll like). The prose
> is worth a million MUSTs.

The fact that implementors may do what they want should not be an excuse
for not looking twice at the requirement level. Prose can be interpreted
in many ways by readers while a testable MUST requirement MUST lead to
one interpretation.

Jean-Francois.

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP