Subject: draft-housley-two-maturity-levels-06

From: Dave Cridland <[email-masked]>
Date: Thu, 05 May 2011 18:22:36 +0100

On balance, whilst I appreciate the aims of this document, I think  
the proposals are not suitable for adoption.

1) This document radically lowers the quality of Proposed Standards.  
Given that to the wider world, an RFC is an RFC, I think this  
represents a mistake. Instead, in common with how one actually  
specifies the standards we write, we should base any modification  
first and foremost on what is observed in practise.

2) I note that it now takes a simple two week last call to go from a  
lowered-quality PS to a full IS.

This is interesting, and I think it's worthwhile examining how this  
might have been applied in recent actions.

Consider RFC 3920 - recently updates, this has been in force for over  
6 years until its recent replacement.

For 5 and a half of those, it could have been an Internet Standard,  
under these rules - XMPP has very wide deployment with a number of  
independent implementations working interoperably.

But it's worse than that - because RFC 3920 generated no small amount  
of noise in its Last Call, including a rework, if memory serves, of  
its SASL profile. Under the proposed rules for PS, this scrutiny may  
well not have happened, and we might either have had interoperability  
problems, or - more likely - simply a worse standard.

Neither of the two note above seem beneficial to increasing the  
quality of the standards we produce - quite the opposite. Neither of  
the above two seem to reflect the reality of what we do and how it's  
perceived by the wider internet community, and nor do they appear to  
take this into account in any way. Together, they represent a real  
danger that radically lower-quality specifications may get stamped  
with significantly higher approval.

I do, however, find merit in Keith Moore's suggestion of a  
lightweight labelling process for I-Ds; this I can see would have  
been highly beneficial in a number of cases, and largely acts as a  
formalization and communication of the status quo; that is, we often  
*do* recommend that people track a draft closely.

I am, incidentally, largely in favour of the central tenet of  
reducing the standards-track to just PS and IS; I think the  
implementation outlined in this proposal is, however, broken.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Dave CROCKER <[email-masked]>
Date: Thu, 05 May 2011 10:31:33 -0700

On 5/5/2011 10:22 AM, Dave Cridland wrote:
> On balance, whilst I appreciate the aims of this document, I think the proposals
> are not suitable for adoption.
>

What, specifically, are the parts of the proposal that you believe will lower 
the quality of a Proposed Standard?

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Dave Cridland <[email-masked]>
Date: Fri, 06 May 2011 09:31:30 +0100

On Thu May  5 18:31:33 2011, Dave CROCKER wrote:
> What, specifically, are the parts of the proposal that you believe  
> will lower the quality of a Proposed Standard?

The parts unmentioned in the document, in effect.

It states:

2.1. The First Maturity Level: Proposed Standard

   The stated requirements for Proposed Standard are not changed; they
   remain exactly as specified in RFC 2026 [1].  Various influences  
have
   made publishing a Proposed Standard much harder than the stated
   requirements in RFC 2026.  The intention of the two-tier maturity
   ladder is to restore practice to the requirements for Proposed
   Standard from RFC 2026, and stop performing more scrutiny than
   intended in IETF working groups and the IESG.  No new requirements
   are introduced; no existing published requirements are relaxed.

RFC 2026 essentially defines a PS document as being a first cut,  
likely to change, and as such unsuitable for production deployment.  
In particular:

   Implementors should treat Proposed Standards as immature
   specifications.  It is desirable to implement them in order to gain
   experience and to validate, test, and clarify the specification.
   However, since the content of Proposed Standards may be changed if
   problems are found or better solutions are identified, deploying
   implementations of such standards into a disruption-sensitive
   environment is not recommended.

So this clearly states that should we revert to RFC 2026, then we  
should immediately consider that HTTP, IMAP, and XMPP, for instance,  
should not be deployed in disruption-sensitive environments. Are you  
really sure that's what you think our consensus opionion is? It  
certainly isn't mine.

I don't think this reflects the current status quo in the slightest.  
I think our current PS maturity is essentially considered production  
quality, although we anticipate that clarifications in the  
specification may be required, and consultation with other  
implementors is likely to be beneficial.

In particular, for better or worse, the wider internet technical  
community - ie, the people who are aware of, but not involved in, the  
IETF - expect our PS documents to have high quality, and would be  
caught unawares by a sudden shift back to this stance.

So to make this clear, I think that as the requirements and scrutiny  
of PS documents have increased, the quality has as a result, and -  
undoubtedly more important - the expected quality has also. I am  
suprised that you don't see this as quite apparent.

We have, in general, raised the bar over the years for PS documents,  
and - whether or not you think this was a good idea - this horse has  
left the station, and it's too late to put the lid back on - the  
wider world now expects this.

Dave.

---

From: Dave CROCKER <[email-masked]>
Date: Fri, 06 May 2011 08:08:54 -0700

> The parts unmentioned in the document, in effect.
...
> remain exactly as specified in RFC 2026 [1].
...
> RFC 2026 essentially defines a PS document as being a first cut, likely to
> change, and as such unsuitable for production deployment. In particular:

You appear to be saying that the new document lowers quality by continuing to 
use the same basic criteria and qualifiers for Proposed that we've used for many 
years.

Forgive me, but I do not understand how that logic works.  How can the new 
process document lower quality by holding the established criteria for Proposed 
stable?

It appears that your actual concern is not about the new document, but rather 
the existing process specification (RFC 2026).

d/
-- 

   Dave Crocker

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 6 May 2011 12:01:01 -0400

> >The parts unmentioned in the document, in effect.
             ^^^^^^^^^^^

> You appear to be saying that the new document lowers quality by
> continuing to use the same basic criteria and qualifiers for
> Proposed that we've used for many years.

I think he is saying that there is are _de facto_ criteria that are
neither called out in 2026 nor in this draft, and that those criteria
are the running code, so the documentation ought to be made to match.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Scott Brim <[email-masked]>
Date: Fri, 6 May 2011 12:04:35 -0400

Dave: the issue is that PS was previously not seen as a finished product,
now it has much more exalted status, but the criteria have not changed.
On May 6, 2011 11:09 AM, "Dave CROCKER" <[email-masked]> wrote:
> _______________________________________________
> Ietf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf

---

From: Martin Rex <[email-masked]>
Date: Fri, 6 May 2011 18:15:05 +0200 (MEST)

I am strongly opposed to this 2 document maturity level proposal.

The real problem tha the IETF is regularly facing are constituencies that
will fight hard against specifications getting updated, improved or fixed,
once they have been published as RFC. 

Dave Cridland wrote:
> Dave CROCKER wrote:
> > Dave Cridland wrote:
>                             The intention of the two-tier maturity
>    intended in IETF working groups and the IESG.

The proposal literally says "stop performing more scurtiny than
intended".  The lower quality of the documents at Proposed is
an explicit goal of this proposal.  When this proposal is adopted,
good quality for documents at Proposed is only going to happen
by accident, but no longer by peer review and scrutiny.

I believe that this proposal is based on a premise that is inverse
to reality.

The real causality is:

  - very few Working Groups go through the effort on fixing or
    improving documents once they're published as RFCs, which,
    for standards track documents usually means, they stay at
    proposed for years

  - the widespread adoption of protocols & specifications on the
    internet happens based on specification that are at Proposed,
    if it happens at all.

And the logical consequence of Working Groups not updating and fixing
Proposed Document specification within a year after publication as RFC,
in order to improve interoperability and the quality of the specification
on which the software is based that runs the internet -- is to apply
more thorough peer review and scrutiny at that point of the
standardization process that can not be evaded by WG constituencies,
before the adoption of the specification as Proposed Standard.

I recently saw a proposal (by a WG chair) to adopt a new charter item
following a list of 3 WG documents at or heading for PS:

  The WG will attempt to avoid gratuitous changes to these protocols.

I do not believe that such a statement is motivated by the three
maturity levels for documents, but rather caused by constituencies
and emotional engagement by individuals with documents they authored
or contributed to, that do not want specification to be updated,
fixed or changed after these have been published as RFC.

> implementors is likely to be beneficial.

Some IETF participants do, some don't, which results in controversy.

My understanding of the IETF standardization process, in particular
with respect to specification at PS is "updating a spec to improve
interoperability is good, reducing complexity is good, i.e.
making stuff optional that is hardly used or creates interop
problems.  And when there are several different extensibility
options in a protocol, some of which with very high interop
in the installed base and some of which with poor interop in the
installed base, then redefining useful features to use the
more interoperable extensibility is useful.

However, there sometimes are constituencies in IETF WGs that take the
position "the PS spec MUST NOT be updated or changed, unless proven
beyound all doubt, that the spec does not work at all".

> caught unawares by a sudden shift back to this stance.

+1

I assume that for every IETF spec implementer within the IETF, there
are 10 or more implementors that do not participate the IETF.
And it is for those others that we should produce, and update/improve
our specifications, at least once in a while.

> wider world now expects this.

+1

-Martin

---

From: Dave CROCKER <[email-masked]>
Date: Fri, 06 May 2011 09:25:49 -0700

> once they have been published as RFC.

That seems particularly true about possible changes to RFC 2026.

It is, indeed, a problem...

d/
-- 

   Dave Crocker

---

From: Dave CROCKER <[email-masked]>
Date: Fri, 06 May 2011 09:27:16 -0700

> are the running code, so the documentation ought to be made to match.

Oh.  But then that doesn't mean that the /current/ draft lowers quality but that 
the existing process has exhibited the lower.

In that case, it seems like the concrete way to resolve such concerns is to 
propose specific text to add to the current draft?

d/

-- 

   Dave Crocker

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 6 May 2011 12:50:07 -0400

On Fri, May 06, 2011 at 09:27:16AM -0700, Dave CROCKER wrote:
> Oh.  But then that doesn't mean that the /current/ draft lowers
> quality but that the existing process has exhibited the lower.

No, it does not.  

RFC 2026 has a low bar of entry to get something published as PS.

As a matter of fact, we don't do what's in 2026.  Things published as
PS are often quite mature.  Since just about nothing moves past PS
anyway, people regard it as the effective final level of
standardization and therefore require high degrees of review before
publication.  And once something is published at PS, it is all but
impossible to change it in any but the most trivial ways, because of
all the contracts that refer to that RFC and require conformance.  So
in fact a document published as PS has met criteria more stringent
than RFC 20206 would have one believe.

The present draft explicitly says that the actual criteria as actually
published in RFC 2026 are the right ones, and that additional
unwritten conventions and so on are to be removed.  

Therefore, if one thinks those more stringent criteria make for higher
quality, then the current draft will lower quality.  

I think we could state this all more neutrally by saying only that the
existing actual criteria do not match the published criteria, and that
therefore when the current draft says "go back to RFC 2026 published
criteria" is is in fact changing the criteria.  Then we don't have to
have an argument about whether the quality is better, since I don't
know how that would be measured anyway.  But in any case, the document
flat out admits that it is changing these criteria:

   Various influences have made publishing a Proposed Standard much
   harder than the stated requirements in RFC 2026.  The intention of
   the two-tier maturity ladder is to restore practice to the
   requirements for Proposed Standard from RFC 2026, and stop
   performing more scrutiny than intended in IETF working groups and
   the IESG.

So we should not pretend that the draft doesn't change criteria for
PS.  It just reiterates a position about what should happen.  It
remains to be seen what will actually happen.

And again, I have no horse in this race.  I don't believe the document
will make any difference at all, but for that very reason I don't care
whether it is adopted.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: John C Klensin <[email-masked]>
Date: Fri, 06 May 2011 13:14:31 -0400

--On Friday, May 06, 2011 18:15 +0200 Martin Rex <[email-masked]>
wrote:

> The real problem tha the IETF is regularly facing are
> constituencies that will fight hard against specifications
> getting updated, improved or fixed, once they have been
> published as RFC. 

Martin,

That is an interesting observation/ claim.  Because I don't
think I've seen that happen, I'd appreciate some examples.  

I have seen:

	* Insistence that any updated version of a document
	address all known outstanding issues, including
	insistence that the revised document not move forward
	until specific changes for which there is no clear
	consensus be included.  In other words, if there is
	community consensus for changes A and B to document X,
	but no consensus that change C is needed (or
	appropriate, or even correct) some people will try to
	block approval of an X-bis that incorporates only
	changes A and B until everyone agrees to C.  That is a
	very difficult problem in any standards body.  We have
	sometimes gotten around it by explicitly noting that
	issue C remains controversial, but rarely without some
	resistance.  But the problem isn't resistance to updates
	and improvements, only controversy about whether the
	changes are sufficient.
	
	* Insistence by various groups, often (at least
	apparently) only the IESG, that any document being
	revised be altered to conform to multiple requirements
	for content or editorial style that are more recent than
	the original document and in spite a no evidence that
	that older style or organization was causing any
	problems.  That clearly makes it more difficult to get
	an update with actual corrections, etc., issued than it
	might be.  But I've not seen anything that I interpret
	as evidence of resistance to change or updates once an
	RFC is published, only procedural and editorial
	requirements for publication of updated documents that
	sometimes create barriers that seem unnecessary and
	inappropriate.

	* Certainly WGs and others tend to run out of energy
	once initial RFCs are published.   If no one is willing
	to produce a new or updated draft and persuade others to
	review it, then revisions are not going to happen.  But
	your comment implies active opposition to such
	revisions: "no energy" is not active opposition.
	Indeed, while I don't think it is justified in today's
	environment, IIR part of what caused the 2026 "advance
	or die" provision was the belief that, if no one cared
	enough about a given specification to advance it, maybe
	we didn't need it any more.

As someone who has been responsible for several RFCs and who has
worked on updates to a large subset of them, I've seen a lot of
pushback asking for "more", but little or no pushback for the
idea of reviewing and revising specs.

While I see those issues as significant, none of them have
anything to do with this particular proposal: they apply equally
regardless of how many steps there are in the Standards Track,
apply to documents being recycled in grade, and even apply to
non-standards-track materials.

So you may be seeing things that I'm not (behaviors do differ
between areas and even among protocol families within an area)
but, if you are seeing active pushback on making revisions, I'd
be interested in hearing specifics about that, what you would
propose to do about it, and why this document is either helpful
or unhelpful in that regard.

regards, 
    john

---

From: Dave Cridland <[email-masked]>
Date: Fri, 06 May 2011 18:19:36 +0100

> remains to be seen what will actually happen.
Thanks for explaining that; it is indeed what I'm trying to say.

> will make any difference at all, but for that very reason I don't  

I would rather not make the situation worse, especially without  
documentation which shows we have a real understanding of the  
de-facto standards process.

Ideally, I'd rather we first - before any attempts are made to change  
the standards process - document what it *is*.

Then, we should have a clearer understanding of the problems we're  
attempting to solve, and it should prove a lot less contentious.

Dave.

---

From: Eliot Lear <[email-masked]>
Date: Fri, 06 May 2011 20:28:06 +0200

Martin,

I think you may actually be arguing for a 1 step process.

Eliot

---

From: "John Levine" <[email-masked]>
Date: 6 May 2011 18:36:30 -0000

>are the running code, so the documentation ought to be made to match.

This suggests that perhaps we should rename "Proposed Standard" to
"Not a Standard But Might Be One Later," promote the PS published
under the overstrict rules to DS, and we're done.

I'm not sure whether I'm serious or not.

R's,
John

---

From: Barry Leiba <[email-masked]>
Date: Fri, 6 May 2011 17:12:35 -0400

> I'm not sure whether I'm serious or not.

Whether you are or not.., the only way to do this is to stop calling
them "RFC"s.  Maybe we should have a "PROP" series for PS docs, and
only give them "RFC" numbers later, when they progress.

Barry

---

From: "John R. Levine" <[email-masked]>
Date: 6 May 2011 17:29:08 -0400

> only give them "RFC" numbers later, when they progress.

Well, you know, the "Not a Standard But Might Be One Later" really are 
requesting comments.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

---

From: Dave Cridland <[email-masked]>
Date: Fri, 06 May 2011 22:31:28 +0100

> only give them "RFC" numbers later, when they progress.

This is not far off Scott Bradner's 2004 suggestion of "Stable  
Snapshots" of I-Ds.

It's also like the (much more versatile) labelling proposal Keith  
Moore made here.

Dave.

---

From: Barry Leiba <[email-masked]>
Date: Fri, 6 May 2011 17:37:20 -0400

John said...
> Well, you know, the "Not a Standard But Might Be One Later" really are

Yes, well, we all know that "RFC" has lost any real sense of its
expansion long ago.  It certainly has done so in the eyes of most of
the world, for whom "RFC" means "Internet standard".

Dave Cridland sid...
> It's also like the (much more versatile) labelling proposal Keith Moore made
> here.

Perhaps, but Keith's labelling proposal is still not sufficient if we
CALL them "RFC"s.  The *only* way to make people not look at them as
"already standard" is by NOT giving them RFC numbers.

Now, I'm not sure that's what we should do.  But I *am* sure it's the
only way to do it.

Barry

---

From: Dave CROCKER <[email-masked]>
Date: Fri, 06 May 2011 14:49:26 -0700

> Well, you know, the "Not a Standard But Might Be One Later" really are

seems like that is rather cumbersome phrasing for a label.  perhaps we can come 
up with a term that is shorter but implies a similar status with regard to now 
and later.

hmmm.  what about the word 'proposed'?

d/

-- 

   Dave Crocker

---

From: Dave Cridland <[email-masked]>
Date: Fri, 06 May 2011 23:14:01 +0100

> the world, for whom "RFC" means "Internet standard".
Actually, for most of the world, it means Rugby Football Club. But  
yes, for the majority of folk who know about protocol specs, then RFC  
means Standard.

> Moore made
Right, which is why both Scott and Keith's proposals involve I-Ds,  
not RFCs.

> Now, I'm not sure that's what we should do.  But I *am* sure it's  

I think it represents the sanest approach proposed so far.

Dave.

---

From: Martin Rex <[email-masked]>
Date: Sat, 7 May 2011 00:58:21 +0200 (MEST)

> the world, for whom "RFC" means "Internet standard".

Correct.  It really does not matter how *WE* call it, it is not going
to affect in any way what the world does with it as long as it has
the "published" status.

In many areas 90% of the implementors are outside of the IETF.
And if they get the task to implement and ship some protocol in
a product -- they will check the RFC index for a document describing
the protocol.  And if there is exactly one version of a protocol
(and ultimately just one RFC document describing it), then they
going to use exactly that document for their implementation,
indepent of what label&boilerplate the IETF has slapped on the
front page.  It really does not matter whether it reads
"Informational", "Proposed Standard", "Draft Standard" or "Standard".

They need to implement it, there is just one document describing it,
so really, what does this matter?

And frankly, large parts of the internet run with protocols and
specification that are "Informational" and "Proposed", and a
non-marginal part even runs on I-Ds.

The large majority out there just doesn't care how many document
maturity levels the IETF uses.

And the folks that prefer more numerous and easier I-D -> RFC transitions
are likely the "early adopters" which are frequently shipping products
based on I-Ds, and often the members of constituencies that would love
to completely lock down a specification as soon as they ship a product
based on it.

-Martin

---

From: Mykyta Yevstifeyev <[email-masked]>
Date: Sat, 07 May 2011 07:22:53 +0300

> Well, you know, the "Not a Standard But Might Be One Later" really are 
Why invent something new?  The Experimental category covers what you 
want to name "Not a Standard But Might Be One Later"; Experimental docs. 
are fine for permanent record of some protocol to encourage its 
experimental implementations to produce the Standards Track document of 
really high level (since it's based on received experience with this 
protocol).  Some recent RFCs are good examples; see eg. 
http://tools.ietf.org/html/rfc6137#section-1.5

The other problem (if it is) is that implementators really don't care 
whether the Standards Track document is Proposed, Draft or Full; as 
mentioned before in this thread, everything named RFC is considered to 
be stable and OK for implementation (maybe, except Historic documents 
:-).  And this can't be changed; this is established; this is 
implementator's mentality.  Considering it, many WGs/folks who once 
wrote Proposed Standard see no sense to move it forward to Draft and 
Full.  Considering this, even though it was planned by RFC 2026 in the 
other way, Proposed Standards are actually worth that scrutiny they are 
currently given (even though I personally can hardly agree with this 
statement).

Mykyta Yevstifeyev