Subject: Re: Last Call: <draft-housley-two-maturity-levels-06.txt> (Reducing the Standards Track to Two Maturity Levels) to BCP

From: Keith Moore <[email-masked]>
Date: Thu, 5 May 2011 12:49:53 -0400

I am opposed to approving this document as BCP.  It is trying to solve the wrong problem; or to put it another way, it is trying to solve a relatively minor problem in a way that distracts attention away from more important problems.  Approval of this document will exacerbate an already bad situation, and further dilute the value of IETF standardization.

1. I strongly disagree with the stated goal to publish Proposed Standards as soon as rough consensus is achieved.   This goal is consistent with the original intended meaning of "proposed standard".  But the reality is that the public has long associated "proposed standard" with "suitable for widespread deployment", and is likely to continue to do so no matter how IETF changes its process.   Rough consensus is not, and never will be, sufficient to indicate that a standard is suitable for widespread deployment.

Instead, I would suggest introducing one or more stages to the process, with clear and unambiguous labels, prior to advancement to Proposed Standard.  e.g. "recommended for prototype implementations" or "recommended for isolated interoperability testing" or "recommended for limited deployment only" or "candidate for Proposed Standard".   These stages shouldn't require full IESG review or publication as RFCs - merely naming the Internet-Draft (with version number) should be sufficient.    There should be clear and simple criteria for these labels, and WG consensus + approval of the responsible AD should be sufficient.  There should also be defined mechanisms for reporting defects in these documents, and for responding to defect reports.

2. It is arguable that there is little value added by the current transition from Draft Standard to Full Standard, and that the process is too cumbersome in relation to the value provided to the Internet community.  But there is still a need to  provide incentives to update standards in light of interoperability problems that are observed in the field, and/or changing conditions on the network.   There is still a need to inform the Internet community about the current applicability of old standards.   Rather than simply drop the last state transition from the current process, we should be examining how to better tailor the process to meet actual community needs.

3. Six months is not sufficient time between approval of Proposed Standard and approval for the final standards maturity level.

4. The increased requirements that have been imposed in practice for Proposed Standard status exist mostly for valid reasons that benefit the community.  It is possible that these are indications of flaws in the original 2026 process, but merely relaxing the requirements without addressing the need for those requirements in other ways, will do great harm to the value of IETF documents and to IETF's ability to benefit the Internet community.  I strongly disagree with that provision.

5. I strongly object to the idea that IETF should endorse widespread deployment and use of a Proposed Standard prior to any reports of interoperability testing.

6. I support the removal of requirement for annual review of Proposed and Draft Standards, but I think that other mechanisms are needed to ensure that the community is informed about current document applicability.  For example, a document's label as standard or its applicability label could automatically expire if not updated within two years' time.

7. I agree that downward references are a significant cause of delay in document approval, but I disagree with the provision to permit downward references to Proposed Standards if the requirements for Proposed Standards are relaxed from what they currently are in practice.  I'd feel better about downward references if the current set of requirements were largely kept intact and more formally codified.

8. I think STD numbers have not been beneficial to the community and should probably be abandoned.

Keith

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 5 May 2011 12:54:00 -0400

On Thu, May 05, 2011 at 09:13:06AM -0700, The IESG wrote:
> 
> The IESG has received a request from an individual submitter to consider
> the following document:
> - 'Reducing the Standards Track to Two Maturity Levels'
>   <draft-housley-two-maturity-levels-06.txt> as a BCP
> 
> The IESG plans to make a decision in the next few weeks, and solicits
> final comments on this action.

Dear colleagues,

I have read the document.  I have some comments.  Some of these I have
said before, but since there's a last call I thought I should state my
view for the record.

First, I don't oppose publishing the draft as an RFC.  Neither do I
support it.  I am indifferent, because I don't believe it will make a
great deal of difference.

Second, the document appears to claim in section 2.1 that the removal
of three tiers is intened to remove the impediments to publication of
RFCs at PS.  I think this result would be a good thing, but I see no
reason to believe that it will happen.  It seems to me to be at least
as likely that IESG members will over time regard PS as _de facto_
standards, because the draft requires wide deployment of such RFCs in
order to move a protocol to Internet Standard.  This could as easily
reinforce the current situation as to relieve it.

Third, the draft appears to be arguing that advancement is somewhat
more likely under the revised definitions.  While it would be
interesting if this happened, my experience as co-chair of DNSEXT
makes me dubious.  In that role, I have observed that things don't get
moved along the standards track because there is no incentive whatever
to do so.  Things are widely deployed already as PS.  Once they have
been widely deployed, there is great resistance to change.  So in
effect, any advancement is a housekeeping operation in which a
document we've all known as "RFC NNNN" comes to be called "RFC MMMM",
(and sometimes, minor changes are introduced in the text that turn
into great points of contention because of the potential for
incompatibility).  Nothing about this draft changes that state of
affairs; instead, it is reinforced by the requirement of wide
deployment.  

Fourth, the decision to permit downward normative references from IS
to PS strikes me as a little strange.  It means that in order really
to evaluate the maturity of a given RFC, you will need also to
evaluate the maturity of all the RFCs it refers to.

That said, the draft has the noble qualities that it is clear and to
the point.  I like the solution to the problem of the obsolete
definition DS laid out in section 5.

Best regards,

Andrew

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Ted Hardie <[email-masked]>
Date: Thu, 5 May 2011 10:49:20 -0700

The document currently says:

   Downward normative references to Informational documents continue to
   be allowed using the procedure specified in RFC 3967 [2].

   Downward normative references to Historic documents, Experimental
   documents, and Internet-Draft documents continue to be prohibited.

I believe these two statements are in conflict and the conflict must be
resolved.  My reading of 3967 is that if the community has approved the use
of an Experimental document or internet-draft as a downref in a specific
last call, that downref is permitted and may even become institutionalized
(see the text on an AD waiving further last calls).  One of the reasons
given in RFC 3967 is:

o A migration or co-existence document may need to define a standards track
mechanism for migration from, and/or co-existence with, an historic
protocol, a proprietary protocol, or possibly a non-standards track
protocol.

Unless a downref is permitted here, in other words, no coexistence or
migration mechanism can ever advance to Internet Standard.  This seems to me
contrary to the aim of the document.  I recommend that the Last Call for
advancement re-iterate any downrefs as a specific part of the call and that
these calls be lengthened to 4 weeks.  This seems to me to follow the
current BCPs best, but other resolutions are, of course, possible.

I strongly object to this text in Section 5:

2) At any time after two years from the approval of this document as
       a BCP, the IESG may choose to reclassify any Draft Standard
       document as Proposed Standard.

Section 1 already provides a method for any interested party to request that
a document advance from Draft to Internet Standard.  Creating another, less
stringent method to be employed only after 2 years is, at best, odd.  At
worst, it seems like a signal that the IESG desires to do a mass update in
the future, but does not wish to deal with the political fall-out of saying
so at the same time as the publication of the document.  I draw this
inference from the fact that the current section 2 does not require any
IETF-wide Last Call.  If this is meant to say that after 2 years, any IESG
member may put forward a Last Call for any document to be advanced, then I
believe that the IESG member is simply taking the role of community member
set forward in Section 1, and no further section is needed.  IESG review,
Last Call, and approval would still go forward according section 1.

In case this is not clear, I do not think a mass update is valuable or
desirable.  Many of the critical RFCs are pre-Track and no one much seems to
mind.  At most, I think saying that Internet Standards are permitted to
reference Draft Standards is needed.  (This can be inferred now from the
fact that Proposed Standards may be referenced, but it wouldn't hurt to
update that in section 4).

I also strongly believe that this proposal will not have the intended effect
without concerted efforts by the IESG and WG Chairs to adhere to the new
"Proposed Standard" vision.  As a new WG Chair, I plan to push that vision
for my own group, and I hope that the IESG will support that effort as this
document intends.

regards,

Ted Hardie

---

From: Russ Housley <[email-masked]>
Date: Thu, 5 May 2011 14:21:09 -0400

Ted:

> I believe these two statements are in conflict and the conflict must be resolved.  My reading of 3967 is that if the community has approved the use of an Experimental document or internet-draft as a downref in a specific last call, that downref is permitted and may even become institutionalized (see the text on an AD waiving further last calls).  One of the reasons given in RFC 3967 is:
> 
> o A migration or co-existence document may need to define a standards track mechanism for migration from, and/or co-existence with, an historic protocol, a proprietary protocol, or possibly a non-standards track protocol.
> 
> 
> Unless a downref is permitted here, in other words, no coexistence or migration mechanism can ever advance to Internet Standard.  This seems to me contrary to the aim of the document.  I recommend that the Last Call for advancement re-iterate any downrefs as a specific part of the call and that these calls be lengthened to 4 weeks.  This seems to me to follow the current BCPs best, but other resolutions are, of course, possible.

Good catch.  I think we should revise this to allow normative references to Informational, Experimental, and Historic RFCs using the procedures in RFC 3967.

> Section 1 already provides a method for any interested party to request that a document advance from Draft to Internet Standard.  Creating another, less stringent method to be employed only after 2 years is, at best, odd.  At worst, it seems like a signal that the IESG desires to do a mass update in the future, but does not wish to deal with the political fall-out of saying so at the same time as the publication of the document.  I draw this inference from the fact that the current section 2 does not require any IETF-wide Last Call.  If this is meant to say that after 2 years, any IESG member may put forward a Last Call for any document to be advanced, then I believe that the IESG member is simply taking the role of community member set forward in Section 1, and no further section is needed.  IESG review, Last Call, and approval would still go forward according section 1.
> 
> In case this is not clear, I do not think a mass update is valuable or desirable.  Many of the critical RFCs are pre-Track and no one much seems to mind.  At most, I think saying that Internet Standards are permitted to reference Draft Standards is needed.  (This can be inferred now from the fact that Proposed Standards may be referenced, but it wouldn't hurt to update that in section 4).
> 
> I also strongly believe that this proposal will not have the intended effect without concerted efforts by the IESG and WG Chairs to adhere to the new "Proposed Standard" vision.  As a new WG Chair, I plan to push that vision for my own group, and I hope that the IESG will support that effort as this document intends.

I am lost here.  Based on a previous version of the Internet-Draft, the question was raised by Brian Carpenter about documents that were stuck in a state Draft Standard after that label is abandoned.  There was discussion on the list, and some people felt that it was neither confusing nor harmful, while others thought that it would lead to confusion.  This proposal is intended to let the IESG decide,  If it is not considered a problem, they can do nothing.  If it is considered a problem, they can move the Draft Standard DOWN to Proposed Standard.  Your use of "be advanced" makes me wonder if you got a different interpretation.

Russ

---

From: [email-masked] (Scott O. Bradner)
Date: Thu, 5 May 2011 14:33:51 -0400 (EDT)

As I have stated before, I do not think that this proposal will achieve
anything useful since it will not change anything related to the
underlying causes of few Proposed Standards advancing on the standards
track.  I see it as window dressing and, thus, a diversion from the
technical work the IETF should focus on.

If it were up to me, I would not approve this ID for publication as a
RFC (of any type) 

Scott

---

From: Ted Hardie <[email-masked]>
Date: Thu, 5 May 2011 12:08:07 -0700

On Thu, May 5, 2011 at 11:21 AM, Russ Housley <[email-masked]> wrote:

> Section 1 already provides a method for any interested party to request
> that a document advance from Draft to Internet Standard.  Creating another,
> less stringent method to be employed only after 2 years is, at best, odd.
>  At worst, it seems like a signal that the IESG desires to do a mass update
> in the future, but does not wish to deal with the political fall-out of
> saying so at the same time as the publication of the document.  I draw this
> I also strongly believe that this proposal will not have the intended
> effect without concerted efforts by the IESG and WG Chairs to adhere to the
> new "Proposed Standard" vision.  As a new WG Chair, I plan to push that
> vision for my own group, and I hope that the IESG will support that effort
> as this document intends.
>
>
> I am lost here.  Based on a previous version of the Internet-Draft, the
> question was raised by Brian Carpenter about documents that were stuck in a
> state Draft Standard after that label is abandoned.  There was discussion on
> the list, and some people felt that it was neither confusing nor harmful,
> while others thought that it would lead to confusion.  This proposal is
> intended to let the IESG decide,  If it is not considered a problem, they
> can do nothing.  If it is considered a problem, they can move the Draft
> Standard DOWN to Proposed Standard.  Your use of "be advanced" makes me
> wonder if you got a different interpretation.
>
>
I did misread this, sorry.  I still don't think it is a good idea, but I
will drop from "strongly object" to "object".

I think we should acknowledge that we had an era in which Draft Standards
existed and leave it at that.  Updates from Draft to Internet Standard make
some sense, because in at least some cases they would have been updated to
Standard.  But the only way under our current process to force something
from Draft to Proposed is to re-cycle it, and I don't think that should
change.  I think a mass update (in any direction) would be a mistake.  At
the core of that belief is a conviction that any change that didn't result
from a re-cycle should require a Last Call to the community.  Up, Down,
Sidewise--these should be community decisions that the IESG approves, not
IESG decisions without community input.

Thanks for your quick reply,

Ted

Russ

---

From: Bernard Aboba <[email-masked]>
Date: Thu, 5 May 2011 12:46:42 -0700

Speaking for myself only, I believe that this proposal does attempt to address some 
issues relating to advancement, so that it is not entirely  "window dressing".  
For example, I believe that the changes with respect to "down references" (Section 4) 
and "annual review" (Section 3) are constructive, and long overdue. 

Implementation reports are a more difficult topic since they constitute both an obstacle
to advancement as well as an important step on the road to development of interoperable
standards.   In particular, the development of implementation reports, while cumbersome,
provides objective evidence of progress. 

It is difficult to simultaneously lower the barriers to advancement while keeping most
of the value (and objectivity) that implementation reports provide.   

I am not sure that the document currently has this balance quite right.   Section 2.2 states:

   Note that the distinct requirement from RFC 2026 [1] for reports of
   interoperability testing among two or more independent
   implementations is intentionally subsumed by the requirement for
   actual deployment and use of independent and interoperable
   implementations.  The Last Call is intended to identify unused
   portions of the specification that greatly increase implementation
   complexity without burdensome implementation testing and
   documentation.
Today it is quite common within WGs to see conflicting claims about protocol implementations and
interoperability.   IMHO one of the critical purposes served by implementation reports is to require proponents
to "produce the evidence" backing their claims.   The above paragraph left me wondering what the
"burden of proof" would be in practice.   For example, I would not want to see the IESG put in the
position of adjudicating "he said, she said" arguments made during Last Call.  

As a result, I cannot endorse the approval of this ID as it exists today, but could see it being changed to address these concerns.

> To: [email-masked]
> Subject: Re: Last Call: <draft-housley-two-maturity-levels-06.txt> (Reducing	the Standards Track to Two Maturity Levels) to BCP
> Date: Thu, 5 May 2011 14:33:51 -0400
> From: [email-masked]
> CC: [email-masked]
>

---

From: Donald Eastlake <[email-masked]>
Date: Thu, 5 May 2011 15:52:55 -0400

I think this draft may do a little good, but mostly based on the
attention it brings to the issue.

If it is actually desired to make it easier to become a Proposed
Standard, it would be quite easy and straightforward to take real
steps that would make a real different. For example, to *prohibit* the
requirement of multiple interoperable implementations, a requirement
sometimes applied in an inconsistent and haphazard manner to
candidates for Proposed Standard.

On STD numbers, they were an interesting experiment but I believe, as
currently implemented, they have been proven to add only confusion and
bureaucracy. It would be quite easy and straightforward to have a
different document sequence for Standards. For success in this, it
would be essential to assure that they do *not* have RFC numbers.
History shows that, regardless of other labels, if a document has an
RFC #, most references to it will be via that number.

Thanks,
Donald
=============================
 Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
 155 Beaver Street
 Milford, MA 01757 USA
 [email-masked]

On Thu, May 5, 2011 at 2:33 PM, Scott O. Bradner <[email-masked]> wrote:
> _______________________________________________
> Ietf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf
>

---

From: John Leslie <[email-masked]>
Date: Thu, 5 May 2011 17:24:47 -0400

> candidates for Proposed Standard.

   +1

   I suppose where such "requirements" are dropped, a warning-label
such as "Not considered safe for widespread deployment" deserves to
be attached; but IMHO we'd be much better off with explicit warning
labels than with implicit expectiations which are poorly documented.

   Proposed Standard _used_to_ imply "May not be safe for widespread
deployment; but I'm afraid that whole mindset has disappeared over the
years.

   I would suggest a serious effort to list mission-creep that has
found its way into "requirements" for Proposed Standard; and to work
out what sort of warning labels we could use instead.

   Otherwise, I see escalating mission-creep, regardless of the number
of "maturity levels".

--
John Leslie <[email-masked]>

---

From: Dave CROCKER <[email-masked]>
Date: Thu, 05 May 2011 15:47:50 -0700

Folks,

On 5/5/2011 11:33 AM, Scott O. Bradner wrote:
> track.

We currently have a standards process that largely ignores two of its stages.

At the least, our community should be embarrassed about this cruft and should 
want to streamline things and make them more likely to be fully exercised.

I happen to be skeptical abut the one motivation offered in the document, that 
that the proposed changes will affect how working groups or the IESG handle 
criteria for achieving Proposed (modulo the downref change.)  There are folks 
who have some faith that this change will make it easier to get to Proposed.  I 
don't understand their logic, but what the heck, it's a substantial constituency 
that believes it.

I lobbied with Russ to expand the list of benefits so that the evaluation of the 
proposed change would not rely /only/ on that one issue.

I think there is an array of benefits that plausibly can accrue from this change 
and that we need not rely on only one:

    1. The criteria for the second stage are significantly clarified and rely 
exclusively on actual adoption and use (again, modulo some important nuances.) 
Whatever happens with the practice of getting to Proposed, this should make it 
more predictable and easier to get to (Full) Internet Standard, based solely on 
actual success of the specification.

    2. The model is cleaned up, in my opinion significantly.  This improves 
transparency of the process a bit.  Also, I think Draft made sense when our 
whole endeavor was less mature and we needed to help the community understand 
what is needed for achieving interoperable deployments.  Today we need the 
/practice/ of interop efforts, but we do not need it in the formal process, as 
demonstrated by how few specs get to Draft in spite of becoming entirely 
successful community services.[1]

    3. The changes are likely to remove bits of community confusion about the 
IETF standards labels.  Not entirely of course, because people are creative. 
But the proposed changes make a very clean distinction between the initial 
technical work and the later adoption/deployment/use work.

We should be careful not to latch onto a non-normative detail in the doc as the 
basis for accepting or rejecting it.  What should matter is whether the changes 
make sense.

d/

[1] There is an argument that doing the work of getting to Draft makes for 
better specs.  I have no trouble believing that.  Were it widely practiced, we'd 
probably have a better Internet.  But the fact that there is a majority practice 
of staying at Proposed renders the minority practice of diligently qualifying 
for Draft irrelevant, IMO.

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: RJ Atkinson <[email-masked]>
Date: Fri, 6 May 2011 04:26:24 -0400

> final comments on this action. 

Yes, please, and as soon as possible.
This is very long overdue.

Many thanks to Russ H for being patient and persistent on this topic.

Yours,

Ran Atkinson

---

From: Dave Cridland <[email-masked]>
Date: Fri, 06 May 2011 10:29:00 +0100

> We currently have a standards process that largely ignores two of  
That's part of the story. It's not a complete observation, though.

We have a standards process that is four stage, not three, and a  
first step should be to document what we're doing. That's how we work  
- we document running code.

> At the least, our community should be embarrassed about this cruft  
> and should want to streamline things and make them more likely to  
> be fully exercised.
> 
> 
No. The minimum is that we should be embarrassed about the cruft.

It is a trivial observation that we do not follow RFC 2026, and we  
should ensure that we have a standards process we actually follow.

>    1. The criteria for the second stage are significantly clarified  
> and rely exclusively on actual adoption and use (again, modulo some  
> important nuances.) Whatever happens with the practice of getting  
> to Proposed, this should make it more predictable and easier to get  
> to (Full) Internet Standard, based solely on actual success of the  
> specification.
> 
> 
More predictable is good. Easier? I'm not sure.

But the real problem I have is the phrase hidden away in the above -  
"Whatever happens with the practise of getting to Proposed". You're  
not seriously intending to put forward another document which  
"defines" our standards process yet not expecting anyone to follow  
it, are you?

>    2. The model is cleaned up, in my opinion significantly.  This  
> improves transparency of the process a bit.  Also, I think Draft  
> made sense when our whole endeavor was less mature and we needed to  
> help the community understand what is needed for achieving  
> interoperable deployments.  Today we need the /practice/ of interop  
> efforts, but we do not need it in the formal process, as  
> demonstrated by how few specs get to Draft in spite of becoming  
> entirely successful community services.[1]
> 
> 
I'll go along with this one.

>    3. The changes are likely to remove bits of community confusion  
> about the IETF standards labels.  Not entirely of course, because  
> people are creative. But the proposed changes make a very clean  
> distinction between the initial technical work and the later  
> adoption/deployment/use work.
> 
> 
This one is hysterically funny.

To quote from draft-bradner-ietf-stds-trk-00 (paraphrasing newtrk).

   4/ there seems to be a reinforcing feedback loop involved: vendors
      implement and deploy PS documents so the IESG tries to make the  
PS
      documents better

This is the core issue, which far from addressing, the proposal tries  
to discard the feedback loop, stick its fingers in its ears, and sing  
la-la-la-I'm-not-listening.

The fact remains that vendors treat PS maturity RFCs as "standards".  
By reverting to the letter of RFC 2026, this will undoubtedly  
increase confusion - indeed, it's apparent that much of the deviation  
from RFC 2026 has been related to this very confusion.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: "t.petch" <[email-masked]>
Date: Fri, 6 May 2011 11:23:59 +0200

I oppose publication of this as an RFC.

It is about politics, not about technical matters, and politics is the art of
the possible.  Even if this
proposal succeeds in persuading (most of) the IETF to rethink the meaning of
'Proposed Standard',
its impact on the rest of the world will be nil.  The rest of the world will see
more 'Proposed Standard's and will assume that they have been produced with the
same care, consideration and review as in the past decade.  Which will be a
mistake.

If you must have a new lightweight, 'back-to-the-2026' offering, then the name
must make it clear that that is what it is and any use of the word 'Standard'
for it would be wrong, regardless of the original intention of RFC2026.  Rather
it should be along the lines of 'Prototype Specification' or Experimental or
Preliminary or ...

Tom Petch

---

From: John Leslie <[email-masked]>
Date: Fri, 6 May 2011 06:44:48 -0400

Dave Cridland <[email-masked]> wrote:
>      PS documents better

   Please excuse the hyperbole -- Dave's just trying to get our attention.

> from RFC 2026 has been related to this very confusion.

   Nothing we put in a rfc2026-bis will change this. Nothing we put in
a rfc2026-bis _CAN_ change this.

   If we want to change this, we need to start putting warning-labels
in the _individual_ RFCs that don't meet a "ready for widespread
deployment" criterion.

   (I am speaking neither for nor against two-maturity-levels here:
warning-labels need to happen if we expect to change implementors'
expectations of PS RFCs.)

---

From: Jari Arkko <[email-masked]>
Date: Fri, 06 May 2011 12:52:40 +0200

As the sponsoring AD for Russ' draft, I'm very interested in hearing 
what everyone thinks about this. Please keep those comments coming! The 
last call was started as it was felt that discussion may have converged 
enough so that we could perhaps move forward with this proposal, or at 
least agree that its harmless and does not stand in the way of other 
improvements that some people may wish in addition. We'll see what the 
consensus will be. I can also see that some specific changes have 
already been discussed. Thanks for all those suggestions.

But I also wanted to provide my own opinion. I would like to see the 
draft adopted. I personally do not believe that it will make a very 
significant change by itself; to a large extent the two (or even one) 
level model is already the running code. However, it does make the 
process a bit easier and it does allow us to get rid of at least some of 
the cruft from the IETF process. I think it is very important to 
simplify the process and align process documents with actual practice, 
as these will make it easier to describe, use, and evolve the IETF 
process. There are plenty of IETF process problems to address, even if 
this draft moves forward.

Jari

---

From: Eliot Lear <[email-masked]>
Date: Fri, 06 May 2011 13:05:15 +0200

Jari, and others,

I support this draft with the caveat that we can establish a set of
significant metrics that provide us an understanding as to the impact of
the change.

Eliot

---

From: Dave Cridland <[email-masked]>
Date: Fri, 06 May 2011 12:04:40 +0100

>    Please excuse the hyperbole -- Dave's just trying to get our  
I concede that the draft neither has fingers nor sings; the point  
remains valid however.

> a rfc2026-bis _CAN_ change this.
I'm in total agreement with this, which is why I'm so against a  
proposal which exacerbates the issue.

> in the _individual_ RFCs that don't meet a "ready for widespread

I do not believe this will work, actually.

In general, I think boilerplate warning messages get ignored - people  
quickly learn to expect and ignore them as routine - and I don't  
think we're likely to be able to construct unique and varying warning  
messages for every RFC we publish.

Dave.

---

From: John Leslie <[email-masked]>
Date: Fri, 6 May 2011 07:35:03 -0400

Dave Cridland <[email-masked]> wrote:
>> If we want to change this, we need to start putting  
>> warning-labels in the _individual_ RFCs that don't meet
>> a "ready for widespread deployment" criterion.
> 

   It is at least a step which _might_ work...

> In general, I think boilerplate warning messages get ignored -
> people quickly learn to expect and ignore them as routine -

   It's not fair to compare this to government-warnings on
cigarette packs.

   However, I agree that if warning-labels look like boilerplate,
folks will ignore them.

> and I don't think we're likely to be able to construct unique
> and varying warning messages for every RFC we publish.

   I offer as evidence the quite-limited warning-labels that the
IESG may put on RFCs that are not IETF series RFCs. These happen
routinely and seem to be accomplishing their intent.

   And, if I may speculate, we might consider warning-labels
that refer readers to status pages maintained by area teams to
show progress on issues not (yet) resolved at the time of
publication.

   There _are_ things worthy of trying here.

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 07 May 2011 13:02:34 +1200

Ship it. We are way past the point of diminishing returns in
polishing this.

Regards
   Brian Carpenter

---

From: Mykyta Yevstifeyev <[email-masked]>
Date: Sat, 07 May 2011 08:14:43 +0300

Hello again,

Now as for the document itself.  I really appreciate its purpose; 
encouraging Proposed-Standards-writers to advance their document 
replacing 3-tier system to 2-tier one is a good idea.  However, in fact, 
this proposal in its current view really can't work effectively because of:

>     Any protocol or service that is currently at the abandoned Draft
>     Standard maturity level will retain that classification, absent
>     explicit actions.
In this case, as Brian Carpenter said during discussions of previous 
versions of this document, "unless we have a firm sunset date, the job 
will never be finished and in fifty years there will still be DS 
documents." 
(http://www.ietf.org/mail-archive/web/ietf/current/msg65870.html)  
Considered this, removal of the sunset period is probably harmful; as 
nobody cared about progressing their DSs (see my previous message), they 
will continue to do this.
> Two possible actions are available:
>
>     (1) A Draft Standard may be reclassified as an Internet Standard as
>         soon as the criteria inSection 2.2  <http://tools.ietf.org/html/draft-housley-two-maturity-levels-06#section-2.2>  are satisfied.  This
> [ . . . ]
>
>     (2) At any time after two years from the approval of this document as
A reasonable question: whether such action won't be available after 2 
years?  Anyway, leaving the situation with DSs "as is" won't result in 
something useful.  As proposed before, 2-year sunset period is OK; if 
nobody undertakes any steps to advance their DS to FS during these 2 
years, such DS is reclassified to PS.

Another issue is removing the requirement for implementation reports.  
This must really result in low quality of FSs.  As Bernold Adoba 
mentioned before,

> interoperability.   IMHO one of the critical purposes served by 
> implementation reports is to require proponents
I fully agree with the last sentence; implementation reports serve as 
some documentary justification of existing of implementations that suit 
all requirements of the spec.

The following requirement for FSs seems unclear:

> * If patented or otherwise controlled technology is required for
>          implementation, the separate implementations must also have
>          resulted from separate exercise of the licensing process.
Maybe I'm misunderstanding smth., but this is not a requirement but 
rather just statement.  This introduces new difficulties to advancing 
PSs to FSs; this hard-to-understand demand will mislead almost 
everybody, I think.

Also, the minimum period for existing on Standards Track of 6 month is 
not really sufficient to fulfill the following requirement:

> * There are a significant number of implementations with
>          successful operational experience.
Some words on STD numbers as well.  Their initial purpose was to provide 
clear reference point to FSs; RFC 1796 encouraged their use to 
distinguish Standards from other RFCs.  However, the experience showed 
that they doesn't work (or work partly).  The well-known problem is what 
to do with STD number when FSs is recycled to PS.  Some recent 
proposals, such as written by John Klensin 
(http://tools.ietf.org/html/draft-klensin-std-numbers-01) proposed 
assignment of STD numbers to all Standards Track RFCs, including PSs.  
This, even solving aforementioned problem, decreases the usefulness of 
STD numbers; but now we are not speaking about that proposal.  Russ' 
document leaves this question open; I think it shouldn't.  I really 
think STD numbers should be abandoned; they just make a mess into the 
Standards Track.  Another proposal should be to put some identifier of 
maturity level in the STD number, assigning one for all documents moving 
through the Standards Track system, ie. STD P-xx, STD D-xx and STD xx 
(for FSs), where xx is assigned to PSs only while all its successors 
retain it; but I don't think such proposal will ever be adopted.

Removal of requirement for annual reviews of PSs is a good deed, since 
nobody really suited to it after NEWTRK WG effort, except rare cases.

There are also some minor defects in this draft, concerning Section 4 
mostly; I don't want to mention them now.

So, taking everything into account, I wouldn't like to see this document 
approved as BCP.  Having a good idea, its realization isn't as good.

Mykyta Yevstifeyev

05.05.2011 19:13, The IESG wrote:
> final comments on this action. Please send substantive comments to the
> [email-masked]  mailing lists by 2011-06-02. Exceptionally, comments may be
> sent [email-masked]  instead. In either case, please retain the
> beginning of the Subject line to allow automated sorting.
>
> The file can be obtained via
> http://datatracker.ietf.org/doc/draft-housley-two-maturity-levels/
>
> IESG discussion can be tracked via
> http://datatracker.ietf.org/doc/draft-housley-two-maturity-levels/
>
>
> No IPR declarations have been submitted directly on this I-D.
>
>
> IETF-Announce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf-announce
>

---

From: SM <[email-masked]>
Date: Sat, 07 May 2011 10:18:01 -0700

>[email-masked] mailing lists by 2011-06-02. Exceptionally, comments may be

 From Section 1:

   "This document proposes several changes to the Internet Standards
    Process defined in RFC 2026 [1].  In recent years, the Internet
    Engineering Task Force (IETF) has witnessed difficulty in advancing
    documents through the maturity levels: Proposed Standard, Draft
    Standard, and finally Standard.  These changes are designed to
    simplify the IETF Standards Process and reduce impediments to
    standards progression while preserving the most important benefits of
    the IETF engineering approach."

There is a documented case where the difficulty occurred at the IESG 
level.  I could not find anything in 
draft-housley-two-maturity-levels-06 to reduce that impediment.

   "In addition, IETF working groups and IESG members provide much
    more scrutiny than is called for by RFC 2026 [1] prior to
    publication as Proposed Standard."

As much as I would like to use the IESG as a scapegoat, the reality 
is that IETF working groups also work briskly to on 
impediments.  Section 4 mentions that "the rules that prohibit 
references to documents  at lower maturity levels are a major cause 
of stagnation in the advancement of documents".  I beg to 
disagree.  Quoting RFC 4897:

   "With appropriate community review, the IESG may establish procedures
    for when normative downward references should delay a document and
    when downward references should be noted."

There is an IESG statement [1] about that.  I'll highlight the 
following sentence:

   "Normative references specify documents that must be read to
    understand or implement the technology in the new RFC, or
    whose technology must be present for the technology in the
    new RFC to work."

Quoting RFC 2026:

   "Standards track specifications normally must not depend on
    other standards track specifications which are at a lower maturity
    level or on non standards track specifications other than referenced
    specifications from other standards bodies."

Let's take a document moving to Draft Standard as an example.  When 
we talk about "down-ref", it is the maturity that is the issue.  What 
it means, in my opinion, is that the  referenced (normative) Proposed 
Standard can be changed in ways which affect the stability of the 
"Draft Standard" document.  An implementation that is compliant with 
the Draft Standard may end up being incompliant overnight as the 
group that worked on the referenced Proposed Standard found some good 
reason for adding some requirements.   Having down-refs on the "No 
Fly" list can be an impediment.  By explicitly calling out the 
down-ref during a Last Call, the IETF offers a means to evaluate 
whether the document can live with the down-ref.

I commented a week ago on the down-refs in RFC 5953 which is being 
advanced to Draft Standard.  One of the down-refs could be fixed 
easily.  Another one could be addressed with some 
rewording.  Sometimes, such a change is not possible.  In a distant 
future, the IETF community might come to terms with the notion that 
down-refs are not evil.

 From Section 2:

   "Note that the distinct requirement from RFC 2026 [1] for reports of
    interoperability testing among two or more independent
    implementations is intentionally subsumed by the requirement for
    actual deployment and use of independent and interoperable
    implementations."

This draft conflates interoperability and deployment.  Having a 
two-track level sounds reasonable at first glance.  My garden variety 
protocol might qualify for "Internet Standard" now that wide-spread 
deployment has been watered down to actual deployment.

 From the same section:

   "The Last Call is intended to identify unused portions of the
    specification that greatly increase implementation complexity
    without burdensome implementation testing and documentation."

That sounds like a good idea.  I would strongly support the removal 
of all the key words that are literally sprinkled in specifications 
nowadays.  It is always a good idea to kill the pet peeve that you 
strong-armed the editor into adding through "consensus by 
exhaustion".  But identifying the unused portions of the 
specification tends to reopen old wounds.

Going back to Section 1:

   "One desired outcome is to provide an environment where the
    IETF community is able to publish Proposed Standards as soon as rough
    consensus is achieved."

That can mean reviewing how IESG evaluation is carried out.  If you 
want to provide such an environment, one possible step is to remove 
DISCUSSes at Proposed Standard level and have the IESG only use rough 
consensus as a measure of whether a document can be published or not.

Consensus is a double-edged sword.  Let's assume that a group of 
well-known vendors implement a specification and people who do not 
have an existing implementation oppose the proposal.  Should the IETF 
ignore the latter and allow publication?  That is a question which I 
would like not to answer because it puts fairness into question.  RFC 
5657 offers a path out of the quagmire by highlighting the need for 
interoperability.

Section 3 mentions that:

   "Lack of this review has not revealed any ill effects on the
    Internet Standards Process."

I strongly object to that statement.  If there aren't any ill 
effects, why is there a Last Call for this proposal?  If the IESG did 
not perform more scrutiny than intended, would we have seen more 
proposals at Draft Standard?

Instead of removing the annual review, it would be better to set a 
sell-by date for documents at Proposed Standard.  The IETF can then 
leave it to  one or more advocates to make a convincing argument that 
the specification meets the multiple implementation and feature 
support requirements of the IETF Standards Process.  The 
implementation report is a measure of whether specifications are 
actually being implemented.

A RFC misses its objective if only the working group participants 
understand how to implement the specification.  The text in a RFC is 
only clear if someone out there can read the document and turn it 
into running code which can interoperate.

 From Section 6:

   "In several situations, an RFC that has reached the full Standard
    maturity level has been obsoleted by a RFC at Proposed Standard
    maturity level, causing great confusion about which specification
    ought to be implemented."

Some people still refer to STD 10, well, the RFC number.  Most people 
refer to its revision even though that version has been obsoleted ten 
years ago.

A Draft Standard can be viewed as a contract.  It specifies that the 
specification is stable.  If the IETF working group decides to do 
some gross rewriting of the document, they are gently reminded that 
the document will be slapped with a Proposed Standard label instead 
of qualifying for Internet Standard.  The final level is a test of 
time.  That's where the IETF says that the widespread deployment of 
the technology did not bring down the Internet.

In Section 1:

   "A specification shall remain at the Proposed Standard maturity level
    for at least six (6) months before consideration for advancement to
    the Internet Standard maturity level.  The six months begins when the
    RFC is published."

As a specification is obviously a RFC, it is redundant to specify 
when the clock begins.

Is six months enough to assess the maturity of a specification?  If 
it is a move from Proposed Standard to Draft Standard, that may be a 
workable minimum.  But that is way too short for widespread 
deployment experience.

The draft-housley-two-maturity-levels-06 effort is a good thing.  I 
commend the authors for that.  I unfortunately have to give it a "Do 
Not Publish" rating as it misses the mark.

Regards,
-sm

1. http://www.ietf.org/iesg/statement/normative-informative.html

---

From: Russ Housley <[email-masked]>
Date: Mon, 9 May 2011 10:34:15 -0400

SM:

> s much as I would like to use the IESG as a scapegoat, the reality is that IETF working groups also work briskly to on impediments.  Section 4 mentions that "the rules that prohibit references to documents  at lower maturity levels are a major cause of stagnation in the advancement of documents".  I beg to disagree.  Quoting RFC 4897:
> 
> There is an IESG statement [1] about that.  I'll highlight the following sentence:
> 
> Let's take a document moving to Draft Standard as an example.  When we talk about "down-ref", it is the maturity that is the issue.  What it means, in my opinion, is that the  referenced (normative) Proposed Standard can be changed in ways which affect the stability of the "Draft Standard" document.  An implementation that is compliant with the Draft Standard may end up being incompliant overnight as the group that worked on the referenced Proposed Standard found some good reason for adding some requirements.   Having down-refs on the "No Fly" list can be an impediment.  By explicitly calling out the down-ref during a Last Call, the IETF offers a means to evaluate whether the document can live with the down-ref.
> 
> I commented a week ago on the down-refs in RFC 5953 which is being advanced to Draft Standard.  One of the down-refs could be fixed easily.  Another one could be addressed with some rewording.  Sometimes, such a change is not possible.  In a distant future, the IETF community might come to terms with the notion that down-refs are not evil.

My person experience with advancing documents is that downrefs are a significant hindrance.  As you point out, procedures have been adopted to permit downrefs, but they are not sufficient.  We often see Last Call repeated just to resolve a downref that was caught very late in the process.  These intoduce delay, and they almost never produce a single comment from the community.

Russ

---

From: Julian Reschke <[email-masked]>
Date: Mon, 09 May 2011 18:08:58 +0200

On 09.05.2011 16:34, Russ Housley wrote:
> ...
> ...

Maybe we should find out why we actually ever start a LC with unexpected 
downrefs... I think we have at least two tools to check for those.

Best regards, Julian

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 09 May 2011 17:10:37 +0100

Hi Julian,

Julian Reschke wrote:

>> My person experience with advancing documents is that downrefs are a 
>> significant hindrance.  As you point out, procedures have been 
>> adopted to permit downrefs, but they are not sufficient.  We often 
>> see Last Call repeated just to resolve a downref that was caught very 
>> late in the process.  These intoduce delay, and they almost never 
>> produce a single comment from the community.
>> ...
>
> Maybe we should find out why we actually ever start a LC with 
> unexpected downrefs... I think we have at least two tools to check for 
> those.

Sometimes references get reclassified during IESG review and this causes 
downrefs. I've certainly caused this several times.

---

From: Julian Reschke <[email-masked]>
Date: Mon, 09 May 2011 18:12:31 +0200

On 09.05.2011 18:10, Alexey Melnikov wrote:
>>> significant hindrance. As you point out, procedures have been adopted
>>> to permit downrefs, but they are not sufficient. We often see Last
>>> Call repeated just to resolve a downref that was caught very late in
>>> the process. These intoduce delay, and they almost never produce a
>>> single comment from the community.
>>> ...
>>

Oh, "informative" -> "normative". Good point.

---

From: SM <[email-masked]>
Date: Mon, 09 May 2011 10:07:38 -0700

Hi Russ,
At 07:34 09-05-2011, Russ Housley wrote:
>significant

Thanks for sharing that.

>  hindrance.  As you point out, procedures have been adopted to 
> permit downrefs, but they are not sufficient.  We often see Last 
> Call repeated just to resolve a downref that was caught very late 
> in the process.  These intoduce delay, and they almost never 

This is an extract from the output of Id-nits for draft-ietf-yam-rfc1652bis-03:

  "Checking references for intended status: Proposed Standard

      (See RFCs 3967 and 4897 for information about using normative references
      to lower-maturity documents in RFCs)

      No issues found here."

For what it is worth, the draft was intended for publication as an 
Internet Standard (STD 71).  As I see it, the problem here is that 
"Intended status: Standards Track" is assumed to be "Proposed 
Standard".  As the Document Shepherd runs a draft through Id-nits, he 
or she will not catch the above issue.  It's unlikely that the IETF 
Secretariat will catch the issue.

If down-refs are a process burden (Last Call has to be repeated) and 
the community does not see any value in having that "restriction", 
the IETF could do any with it.  I don't think that would be a good 
idea as it wipes out the notion of maturity levels.

There are a lot of things that do not produce a single comment from 
the community.  They are done for a reason.  For example, there was a 
message about "Draft Secretariat SOW for Community Comment".  There 
has been only one comment on that.  There is a cost to adhering to a 
standard of goodness.  Once you do away with that, it is not as easy 
to get it back.

Regards,
-sm

---

From: Julian Reschke <[email-masked]>
Date: Tue, 10 May 2011 07:12:19 +0200

On 09.05.2011 19:07, SM wrote:
> "Intended status: Standards Track" is assumed to be "Proposed Standard".
> As the Document Shepherd runs a draft through Id-nits, he or she will
> not catch the above issue. It's unlikely that the IETF Secretariat will
> catch the issue.
> ...

rfc2629.xslt allows specifying the intended maturity in the XML source...:

<http://greenbytes.de/tech/webdav/rfc2629xslt/rfc2629xslt.html#rfc.section.12.1.p.2>

...but it's only metadata in the XML source. Maybe we should add it to 
the ID boilerplate in the future?

Best regards, Julian

---

From: SM <[email-masked]>
Date: Mon, 09 May 2011 23:24:27 -0700

Hi Julian,
At 22:12 09-05-2011, Julian Reschke wrote:
>...but it's only metadata in the XML source. Maybe we should add it 
>to the ID boilerplate in the future?

I'll ignore details such as authors running an ASCII version of their 
draft through Id-nits.  Quoting Alexey:

  "Sometimes references get reclassified during IESG review and this 
causes downrefs."

The issue can occur at the IESG evaluation stage.  With the new RFC 
Editor Model, it's unlikely to occur during AUTH48.

If intended maturity is viewed as a mechanical issues and what you 
suggested fixes 80% of the problem, it may be worth a try.  One could 
also argue that the IESG might see a value in having a reference 
reclassified (things you need to read to implement this specification).

Instead of trying to capture the intended standards level, you could 
simply approve publication as Experimental.  The author gets a RFC 
number.  The IESG does not have to repeat the Last Call.  Obviously, 
authors will lobby against that. :-)

If you would like a glimpse of the outside world, read the thread at 
http://lists.cluenet.de/pipermail/ipv6-ops/2011-May/005514.html

Regards,
-sm

---

From: Pete Resnick <[email-masked]>
Date: Mon, 30 May 2011 18:20:25 -0500

On 5/5/11 11:13 AM, The IESG wrote:
> sent to [email-masked] instead. In either case, please retain the

Man without hat: Speaking as an individual contributor, not an AD.

Summary: I am in favor of most of the changes in this document. The 
primary change I am opposed to is the one identified in the title and 
therefore object the adoption of this document as a BCP. However, I 
believe there is a single (though significant) change that would make 
the document acceptable to me.

Details:

The document says in section 1 that the motivation for the changes is:

    In the current environment, many documents are published as Proposed
    Standard and never advance to a higher maturity level.  In addition,
    IETF working groups and IESG members provide much more scrutiny than
    is called for by RFC 2026 [1] prior to publication as Proposed
    Standard.

I certainly agree with addressing the above concerns. Section 2 lists 
one additional motivation for the changes:

    The benefit associated with a third maturity level has proven
    insufficient to justify the effort associated with document
    progression.

I'll address this below.

Here is a summary of the process changes that occur in the document, by 
section:

Section 2
(a) Generally, go to two maturity levels, Proposed Standard and Internet 
Standard
(b) Only review the changes, not the entire document, when recycled at 
Proposed

Section 2.1
(a) Go back to 2026 level of (i.e., less) scrutiny for Proposed Standard

Section 2.2
(a) Specifically, merge Draft Standard into Internet Standard
(b) Combine criteria from Draft Standard and Standard
     (i) Significant number of implementations with successful 
operational experience
     (ii) No unresolved errata causing interoperational problems
     (iii) No unused features, except allow unused features that do not 
greatly increase implementation complexity
     (iv) Independent patent/licensing for implementations
(c) Remove overt requirement for documentation of interoperability testing

Section 3
(a) No more annual review of Proposed Standards for movement to Historic

Section 4
(a) Allow normative references from Internet Standards to Proposed Standards

(Editorial note: Please move 2(b) into 2.1. That change is a change to 
the handling of Proposed Standards and therefore belongs in 2.1. That 
makes the rest of Section 2 just introductory text and not making a 
specific procedural change.)

My analysis by section:

2(a) - I am opposed to this change and will discuss it below in section 
2.2(a).

2(b) - I like this change and support it.

2.1(a) - I wholeheartedly agree with this change (or, more to the point, 
reversion to the documented way of doing things). However:
- This document gives no mechanism to facilitate this change. I would 
like to hear how this will be accomplished.
- This change, along with 2(b) has the potential to greatly increase the 
number of RFCs published. The document does not discuss the impact of that.

2.2(a) - I see no motivation for this change other than the one sentence 
in section 2. This change does not address the difficulty of going from 
Proposed to Draft, and it doesn't address the heightened scrutiny for 
going to Proposed. Therefore, if the only reason for changing from three 
levels to two is that getting through the three levels is too hard, and 
most of the procedural changes in this document are to make it easier to 
get through the first two levels, I see no justification for removing 
the third (and in 2026, easiest to get through) level. It does not solve 
any identified problem.

2.2(b)(i) - This is the current requirement for Internet Standard. I'm 
fine with that remaining, but object to the removal of any notion of 
interoperability. If this were changed to "A significant number of 
interoperable implementations...", I would have no objection.

2.2(b)(ii) - I have no objection to this new requirement, but have no 
strong feeling about it.

2.2(b)(iii) - I would prefer that this be amended to "All unused 'MUST' 
requirements will be changed to 'SHOULD' requirements." If deployment is 
interoperable and a feature is unused, it means that the feature was not 
actually REQUIRED for interoperability. I object to this as it stands.

2.2(b)(iv) - This is the current requirement for Draft Standard. I'm 
fine with that remaining.

2.2(c) - If this is actually "subsumed" by 2.2(b)(i) (see my change 
above), then I am fine with this change.

3(a) - I have no objection to this change, but hope that the review can 
be reinstated once the rest of the process settles down.

4(a) - I have no objection to this change as it applies to Draft Standards.

So, to summarize, here is how I break down the changes this document 
proposes and my position on them:

A. For Proposed Standard:
- Go back to 2026 review levels for approval
- Only review changes, not the entire document, for recycling at Proposed.

I am in favor of these changes.

B. For Draft Standard:
- Remove requirement for formal documentation of interoperability tests.
- Require only a significant number of [interoperable] implementations 
with successful operational experience.
- Allow no unresolved interoperability errata.
- Allow for unused [SHOULD level] requirements if they don't increase 
implementation complexity.
- Allow for downrefs to Proposed Standard documents.

I am in favor of these changes with the square bracketed insertions I 
have made, except for the last one which I can live with.

C. Eliminate annual review of Proposed Standards.

I would prefer not to make this change, but since it does not reflect 
reality, I am OK with it.

D. For Internet Standards:
- Combine Draft Standards into this category.

This is the only change that I strongly object to since I don't see any 
problem that is solved by doing this and therefore oppose adoption of 
the present document as BCP.

So, here is my a proposed alternative:

1. Make the changes in (A). We still need to say how to make that 
happen, and how to deal with the increased number of RFCs.

2. Make the changes in (B).

3. Make the changes in (C).

With regard to the STD numbers, I think that it would be reasonable to:

4. Assign STD numbers to everything at Draft Standard.

And if people really think that the current *titles* of the levels are a 
problem:

5. Change the name of "Internet Standard" to "Mature Standard" (or 
"Deployed Standard" or whatever label we like).

6. Change the name of "Draft Standard" to "Internet Standard".

I am happy to write an I-D if there is support for this.

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

---

From: SM <[email-masked]>
Date: Mon, 30 May 2011 17:39:23 -0700

>happen, and how to deal with the increased number of RFCs.

The annual review provides an alternative to deal with the increased 
number of (non-historic) RFCs.  A "no substantive objection" clause 
might enable the removal of "drive-by" RFCs.

The IETF can then focus on non-historic RFCs, i.e. specifications 
that are relevant to current discussions.  Or else, there can be an 
automatic reclassification to Historic if an implementation report 
has not been produced within a year of the publication date.

What's been missing during the discussion of this draft is a 
practical way to deal with known issues.

>2.2(b)(iii) - I would prefer that this be amended to "All unused 
>'MUST' requirements will be changed to 'SHOULD' requirements." If 
>deployment is interoperable and a feature is unused, it means that 
>the feature was not actually REQUIRED for interoperability. I object 
>to this as it stands.

That's one way to deal with RFC 2119 creep.  I'll go one step 
further.  If there is a significant number of implementations that do 
not implement a SHOULD, the feature can be removed.  The resulting 
specification might be easier to implement once the amount of 
requirements are reduced.

Regards,
-sm

---

From: Pete Resnick <[email-masked]>
Date: Tue, 31 May 2011 08:27:08 -0500

> might enable the removal of "drive-by" RFCs.

My concern was not the absolute number of RFCs. It is that, if we go 
back to something like 2026 criteria for Proposed, we should expect a 
bunch more revisions of RFCs (since we will find bugs that need to be 
fixed), and that may put an awful lot of pressure on the RFC Editor. 
Because the changes are likely to only be specific bug fixes and not 
total rewrites, perhaps the RFC Editor might be OK with only checking 
the new parts and not worrying about the old ones. But this is not 
addressed at all in the current document and needs to be.

> requirements are reduced.

This to me is wordsmithing. The current text says that you remove the 
ones that "greatly increase implementation complexity." You're 
suggesting removing ones that "might" make it harder to implement. I 
think there's probably a happy median in there someplace.

pr

---

From: John C Klensin <[email-masked]>
Date: Tue, 31 May 2011 12:23:48 -0400

--On Thursday, May 05, 2011 09:13 -0700 The IESG
<[email-masked]> wrote:

> 
> The IESG has received a request from an individual submitter
> to consider the following document:
> The IESG plans to make a decision in the next few weeks, and
> solicits final comments on this action. Please send
> substantive comments to the [email-masked] mailing lists by
> 2011-06-02. 
>...

Hi.

I made several comments about earlier versions of this document
then stopped because, while the incremental versions have all
included small improvements, they have addressed few of the
basic issues and the document seemed to be riding something of
a juggernaut.  Because it is now in Last Call, I want to
summarize and update those comments, in large measure because I
don't want my near-silence since late January to be taken as
agreement.

This proposal is problematic in several ways.  The first is
obviously the most important, but the others suggest changes
that might be useful whether or not that main provision is
adopted.

(1) Excessively-high requirements for Proposed Standard and
dropping the three-level model.

	The document asserts, I believe correctly, that one of the
	problems with the IETF Standards Process is that the
	relatively low bar established for Proposed Standards in
	RFC 2026 has evolved into a very high bar and that we
	should return to that lower requirement level.  Yet it
	effectively proposed to solve that problem by reducing the
	number of steps in the standards process to two.  No
	evidence has been offered that whether or not the standards
	process is two levels or three has anything to do with
	difficulties completing Proposed Standards or getting them
	to a satisfactory level of completeness and correctness.
	The logic on that subject in the document appears to me to
	quite similar to a physician saying "We agree there is a
	problem with your hand.  We don't know how to treat it
	either surgically or medically, but we should do something,
	so let's amputate your foot".  That really makes no sense
	and is probably bad for the patient (only probably... one
	really never knows).

	In addition, draft-housley-two-maturity-levels-06 does not
	discuss any mechanism for getting from the current
	as-practiced requirements for Proposed Standard back to the
	requirements of RFC 2026, a schedule for doing so, nor
	whether it is necessary to warn readers about the level of
	scrutiny applied (it may not be, but there has been little
	or no discussion of that subject in the IETF and there is
	no such discussion in the document).  If this were a
	technical specification, unless there were an expectation
	that a transition will occur immediately and by magic, that
	omission would be a "known defect".

	Suggestion: The level of completeness and quality required
	for a Proposed Standard, at least insofar as those
	requirements exceed the requirements in RFC 2026, are
	entirely in the hands of the IESG, relevant WGs, and the
	Last Call process.  Whether those requirements can be
	reduced to the level required by 2026 has nothing to do
	with this document because there is no change to the 2026
	requirements.  If we are serious about making that change,
	I recommend that the IESG:

        (i) Put this document aside temporarily
        (ii) Announce to the community (presumably via an IESG
            statement) that, as a general matter, documents
			submitted for Proposed Standard will not be
			required to meet conditions beyond those imposed
			by 2026.  Note that 2026 allows imposition of
			additional requirements, especially requirements
			for implementation experience.  Whether a
			particular WG should strive for a higher quality
			standard for its documents should be treated as a
			management issue (e.g., it would rarely make sense
			to downgrade documents that were nearly finished
			at a higher level), but reviews on Last Call or
			from within the IESG that suggest a higher level
			of perfection should probably be ignored unless
			they provide reasoning for the greater requirement.
        (iii) The above implies that different areas, and
			perhaps even different WGs within an area, will end
			up with different practical requirements based, at
			least in part, on their perception of the risks
			implied by a less comprehensive document.  I see
			that as an advantage and recognition of reality,
			not a problem. 
		(iv) After a couple of years -- probably the minimum
		    needed to achieve a steady state given documents
		    already under development and targeting a greater
			level of requirements than 2026 anticipated -- the
			IESG should review the situation and make a
			recommendation to the community as to whether
			further changes are needed to the standards process
			based on experience with this revised way of
			dealing with proposed standards. 

	Note that the model described above is different from the
	one in draft-bradner-restore-proposed-00, especially with
	regard to schedule.  draft-bradner-restore-proposed-00
	suggests a fixed transition schedule; these notes propose
	to make transition a management matter (and, indirectly, a
	test of the IESG's seriousness about restoring the intended
	Proposed Standard definition); as noted above,
	draft-housley-two-maturity-levels-06 does not address this
	issue.  After a few months of consideration, I now believe
	that "management matter" approach provides both a better
	transition plan and long-term framework than trying to
	force documents into a single level of completeness and
	scrutiny.  The other comments and suggestions in
	draft-bradner-restore-proposed-00 could be considered
	useful supplements to the suggestions above. 

(2) Elimination of the "downref" requirement for
standards-track documents.

    The original motivation for the "no normative references to
	documents at lower maturity levels" rule was to be sure
	that the documents referenced were of adequate quality to
	be used, effectively, as part of the document under
	consideration.  If one thinks about it, having a document
	that the IETF certifies as representing an implemented,
	deployed, and useful specification dependent on something
	that is no more that an untested probably-good idea makes
	little or no sense.  Referencing Proposed Standards that
	have survived the level of scrutiny that has prevailed in
	the last several years should not be nearly as
	problematic, but removing the downreferencing requirement
	at precisely the same time that we intend to get Proposed
	Standards back to the 2026 level creates exactly the risk
	that the rule was intended to prevent.

	Suggestion: The community had this almost correct in RFC
	4897 in that the community should be able to review
	proposed downward references.  RFC 4897 more or less
	assumes that the review should require explicit community
	review and signoff.  That might be excessive; a more or
	less default review might be sufficient.  But we should
	preserve the community's ability to conclude that a
	downreferenced document is inadequate for the purposes of
	the referencing document, if only through a Last Call
	response.  The change made in
	draft-housley-two-maturity-levels-06 appears to eliminate
	"the referenced specification is of inadequate quality to
	be used as part of this specification in a normative
	reference".  It seems to me that is a bad idea and probably
	even an unintended side-effect.

	Note that this suggestion should be as effective at
	eliminating downward references as a "major cause of
	stagnation in the advancement of documents" as the proposal
	in draft-housley-two-maturity-levels-06 if, in fact, that
	is a major issue.

(3) Maturity level, document editorial quality, and RFC Editor
issues 

	The rather cavalier assumption that a three-stage standards
	process somehow makes it less possible to apply the
	Proposed Standard criterion of RFC 2026 omits another
	important reason why the bar for Proposed Standards has
	become so high.  It would be a reasonable (and arguably
	historically accurate) statement of the traditional
	editorial quality requirement for Proposed Standard
	documents is that they needed to be good enough to be used
	for implementation among cooperating parties, i.e., clear
	enough that an implementer with access to WG mailing lists
	(or authors) could implement them and evaluate document
	quality for implementation.  Perfect English writing style
	was not a requirement, nor was sufficient clarity that one
	was assured of being able to do an interoperable
	implementation in a "clean-room" environment relative to
	the authors and discussions about development of the spec.
	Viewed from the editorial quality perspective, the Draft
	Standard level was the one at which document quality was
	expected to be improved to meet those much higher standards
	of editorial quality and technical completeness and
	clarity.  RFC 2026 requires "no known omissions" for
	Proposed Standard, not "no omissions" or strong assertions
	of completeness.

	One of the significant obstructions to getting Proposed
	Standard documents published quickly is that the IESG has
	insisted on a rather high level of technical quality,
	usually doing the editing itself rather than trusting the
	RFC Editor staff to get that right.  WGs and the community
	have responded to the expectation of IESG insisting on
	documents of very high editorial quality (and sometimes the
	expectations that documents will never advance beyond
	Proposed Standard) by nit-picking editorial quality
	internally and on IETF Last Call, creating additional
	delays.  If we are going to get Proposed Standards
	published more quickly and under 2026 criteria, the IESG
	will need to do some self-examination about these editorial
	document quality issues and perhaps either settle for lower
	quality at Proposed, shift more of the editorial burden to
	the RFC Editor, or both.  In any case, if documents are to
	be less than editorially perfect at the Proposed Standard
	level, there needs to be provision, or at least an
	understanding, for assuring that documents meet a higher
	standard at subsequent maturity levels.  Those provisions
	are absent from draft-housley-two-maturity-levels-06, as is
	even a statement of the possibility that they might be
	relevant...  another "known omission".

	Note that accepting a less formal, and less
	formally-correct, editorial style is consistent with
	publishing "Proposed Standards as soon as rough consensus
	is achieved".  Spending weeks or months sorting out
	editorial issues is not.  

	In addition to the above, Long-duration MISSREF entries in
	the RFC Editor queue are a major source of the claim that
	downward referencing requirements are a significant
	impediment to advancement of documents.  However, if those
	entries are examined, a significant fraction of them are
	actually either missing references pointing to I-Ds (for
	which the present proposal provides no help at all) or a
	mechanism for identifying a set of documents that have to
	be published together to make sense.  As long as we
	periodically approve and publish a single logical standard
	as a collection of documents that may go through a WG or
	the IESG at different rates, there will be a need to hold
	the first-approved of those documents until the
	last-approved one is ready.
	draft-housley-two-maturity-levels-06 does not make
	provision for whatever additional RFC Editor states are
	needed in the IETF Stream to cover that problem.  While not
	major, it is another sign that, if the present proposal
	were being evaluated as a technical specification, it would
	be found to contain known omissions.

(4) Discussion of the STD document series in Section 6.

	It has been pointed out several times, in recent months to
	suppress discussion of proposals that might be alternates
	to some or all of this one, that there are many issues with
	how the IETF develops, approves, documents, and manages
	standards that are not addressed in this specification and
	that the community cannot reasonable expect to address all
	at the same time.  The question of what should be done
	about the STD numbers is just one entry on this very long
	list.  Examination of other IETF procedural RFCs indicates
	that we rarely, if ever, include a discussion of a single
	option that we have decided to _not_ pursue (even if only
	in the near term).  The only apparent justification for
	including Section 6 in this draft is that there was a
	specific proposal in an earlier version, a proposal that,
	as the draft indicates, did not get community consensus.
	The section should be dropped from this document before
	publication.  If the community wants to have a series
	discussion of STD numbers, I suggest that
	draft-klensin-std-numbers might be one contribution to that
	discussion.

(5) Patents

    The specification says (last bullet in Section 2): 

    > * If patented or otherwise controlled technology is
	>   required for implementation, the separate
	>   implementations must also have resulted from separate
	>   exercise of the licensing process.  

	While this text is copied from 2026, things have changed
	and the requirement has been interpreted very loosely,
	particularly to leave interpretation of "required" in the
	hands of the implementers.

	If it is repeated in a spec that is approved now and that
	claims to reset and clarify Draft Standard requirements, it
	can easily be read as a requirement that the community has
	to somehow evaluate whether claimed controlled technology
	is actually "required for implementation".  That may be
	obvious in some cases; it may be highly controversial and
	dependent of validity of patent claims (something that can
	normally be resolved only by the courts) in others.  The
	IETF has been repeatedly advised, both by legal counsel and
	by various incarnations of IPR WGs, to avoid straying into
	the territory in which it takes a position on whether an
	IPR claim is valid.  Consider a scenario in which some
	company, say ChuckCo, decides that an IETF Internet
	Standard for Alice's proposal would put them at a
	competitive disadvantage.  It is merely necessary for
	ChuckCo to file an IPR claim against the technology in
	Alice's spec, even a obviously spurious one, assert a
	completely irrational and punitive licensing model that
	cannot be exercised in a reasonable way, and then sit back
	and watch the IETF paralyze itself over this provision as
	written.

	Suggestion: I believe that advice of Counsel should be
	sought about this language and, if feasible, alternate
	language produced that either turns the requirement into an
	"if feasible" request or that places the responsibility for
	evaluating whether the claims are applicable on the
	implementers and not on the IETF or IESG.

Conclusion: 

	This document is not ready for approval.  It is justified
	in terms of one issue but offers no evidence that it will
	help at all with that issue and instead proposes a solution
	that is associated with another set of issue that may not
	be critical-path.  For that primary issue (speeding
	documents into Proposed Stanard), it fails to address known
	issues and, more important, is not required at all.  While
	"no known omissions" is not formally a requirement for IETF
	procedural documents to be published in the BCP series, it
	is nonetheless a useful guideline and this document does
	contain such omissions in several areas.  It removes an
	important criterion for evaluating document readiness for
	publication when the document depends on other
	specifications that are unclear or worse.  It makes no
	provision for transition.  It contains spurious and
	almost-irrelevant text.  And it potentially opens an IPR
	can of worms that the IETF has been warned against.

thanks for listening
   john

---

From: John C Klensin <[email-masked]>
Date: Tue, 31 May 2011 12:37:17 -0400

--On Tuesday, May 31, 2011 08:27 -0500 Pete Resnick
<[email-masked]> wrote:

>>> 1. Make the changes in (A). We still need to say how to make
>>> that  happen, and how to deal with the increased number of
>>> RFCs.
>> 
>> The annual review provides an alternative to deal with the
>> increased  number of (non-historic) RFCs.  A "no substantive
>> objection" clause  might enable the removal of "drive-by"
>> RFCs.
> 
> My concern was not the absolute number of RFCs. It is that, if
> we go back to something like 2026 criteria for Proposed, we
> should expect a bunch more revisions of RFCs (since we will
> find bugs that need to be fixed), and that may put an awful
> lot of pressure on the RFC Editor. Because the changes are
> likely to only be specific bug fixes and not total rewrites,
> perhaps the RFC Editor might be OK with only checking the new
> parts and not worrying about the old ones. But this is not
> addressed at all in the current document and needs to be

Concur.

I do believe that, for Proposed Standards only, adopting a rule
that only changes are examined when recycling in grade could
reasonably be applied to the RFC Editor as well as IETF review.  

However, I think it is workable only if:

-- There is some sort of exception procedure that can be applied
when good sense requires it.    For example, while our normal
practice is that the editor of a spec is the editor of a
revision of that spec, there are exceptions.  The exceptions can
involve sufficiently large changes in style that not editing the
whole document could produce a result that is very hard to read
and understand.

-- Any sort of "tolerate editorially-poor documents" strategy,
whether involving "edit only changes" as you suggest above,
"accept less-than-ideal writing style as long as the protocol
intent is clear" as suggested in my comments, or something else
needs to have a clear point at which we apply a different set of
expectations.  I believe that ought to be the second level in
the standards process (whatever that is called).  However the
line is drawn, I don't think we can have an expectation of
high-quality finished documents, fast approval and publishing of
Proposed Standards, and little editing work on documents going
to the second level

    john