Subject: Re: Last Call: <draft-housley-two-maturity-levels-08.txt> (Reducing the Standards Track to Two Maturity Levels) to BCP

From: John C Klensin <[email-masked]>
Date: Thu, 01 Sep 2011 11:22:53 -0400

Hi.

The recent discussion about DISCUSS and DS/IS documents has
inspired me to go back and think about the "two maturity levels"
draft again.  Sadly, it hasn't changed my mind but has, in some
respects, reinforced and strengthened my earlier view that this
is not a good idea and is not harmless.

To avoid repeating myself and making this note too long, I won't
repeat myself but would like much of the content of my notes at 
https://www.ietf.org/ibin/c5i?mid=6&rid=49&gid=0&k1=933&k2=59000&tid=1314888873
https://www.ietf.org/ibin/c5i?mid=6&rid=49&gid=0&k1=933&k2=59008&tid=1314888873
and
https://www.ietf.org/ibin/c5i?mid=6&rid=49&gid=0&k1=933&k2=59018&tid=1314888873
to be considered as part of this response.  

However, in particular and independent of any of the other
issues, there is just no evidence that the two maturity level
change will make any difference at all.  The problems (and
possible solutions) lie elsewhere and, if one examines the
evolution of the various drafts of that proposal, it is clear
that the community, and probably the authors, understand that it
will make no significant difference other than to spare the IESG
one extra review in a case that rarely occurs (more rarely in
some areas than others).

So the question becomes whether this change, which "might" help
(that is as close as anyone can get -- we are still without a
compelling argument) is, at worst, harmless.  I don't think it
is, for at least two reasons:

(1) We have ample historical evidence that making a change like
this will block consideration of more substantive changes,
changes that might actually improve things.  People will argue
against consideration of such changes on basis of community
exhaustion (and will probably be right).  Others will argue that
we need to see if this experiment works rather than pushing
ahead with other changes that might create confusion about where
any effects came from or they will speculate that other changes
might mess this one up.  They might be right too -- certainly if
the IETF sets a "two levels" direction, it would be unwise to
immediately propose a "one level and lots of annotation and
explanation" model.

(2) Having procedures that we don't use exactly, or that the
community doesn't follow, merely means that we haven't updated
those procedures (for whatever reason).  We live with it and so
do ITU-T, ISO, IEEE, and so on with procedures they have altered
but not yet captured in formal documentation.  But to change
procedures from something we don't follow to something that
won't work is silly, should be embarrassing, and exposes us to
increased vunerability to attacks from others (including other
SDOs who want to take over IETF's place in the world).   It is
worth the risk to change procedures and risk new ones not
working if we have good reasons to believe that they will work,
or that they will bring significant benefits, or both.  But a
change to something for which there is no evidence that it will
work, and some evidence to believe that it solves the wrong
problem, should be a non-starter.

If we want to fix the standards track, then let's start by
seeing if we have consensus about what is broken (I think we
probably do and that the discussion of this draft has had the
immensely useful side-effect of producing useful discussion on
that topic).  Let's try some bold experiments in turning things
around, e.g., having the IESG move toward actually following the
intent of RFC 2026 wrt acceptance of documents at Proposed
(discussed in the recently-expired
draft-bradner-restore-proposed), encouraging some areas --
ideally those with the lowest rates of advancement beyond PS--
experiment with different considerations for advancement to DS
and IS (discussed in the "Discuss criteria ..." thread and
especially in the third mail message cited above).  Or let's ask
the question of whether broad-category maturity levels actually
serve our needs, and those of the community, well in our present
environment of increasingly complex and option-laden standards
(from the standpoint of maturity level categories, the only
really good standard is one that contains no requirements other
than "MUST" and "MUST NOT"  -- no SHOULD or MAY variations) and
move to one maturity level supplemented by more Applicability
Statements or commentary and annotation external to the protocol
specifications themselves.

   regards,
     john