Subject: Re: Conclusion of the last call on draft-housley-two-maturity-levels

From: SM <[email-masked]>
Date: Fri, 02 Sep 2011 12:12:06 -0700

At 13:17 30-08-2011, Jari Arkko wrote:
>There were a number of smaller details raised in the discussion. But 
>I did not see an overwhelming consensus on any specific issue to 
>make changes. But I will ask Russ to take a look at the issue raised 
>by Scott, whether he wants to add an informative reference to RFC 5657.

I read draft-housley-two-maturity-levels-09.  I read the messages 
which might be interpreted as statements of support.  Mr Burger 
offered that we are moving a baby step forward.   Mr Resnick asked "A 
baby step toward what exactly" to which Mr Saint-Andre pointed out 
that "we are more closely aligning our documentation with our 
organizational running code".  The organizational running code 
actually sets a higher bar than what is documented in RFC 2026 for 
the publication of a Proposed Standard.  The draft does not even 
discuss about that.

Mr Carpenter believes that "the present situation is confusing both 
to IETF newcomers (who may falsely believe that the IETF actually 
follows the 3 stage process) and, worse, confusing to users of IETF 
standards (who may falsely believe that a document isn't useful until 
it's advanced). We, and those users, gain by reducing the 
confusion".  In terms of document clarity, RFC 2026 taken together 
with draft-housley-two-maturity-levels-09 only reinforces the 
confusion for anyone who takes the time to read BCP 9.

Mr Atkinson pointed out that a change in perception alone is 
sufficient to increase [his] own motivation.  Mr Burger confirmed 
that the intent of the proposal is to change the perception.

Mr Halpern mentioned that the draft tries to align what we document 
with what we do.  In a response, Mr Klensin mentioned that the draft 
addresses one provision of our processes in which documentation and 
practice don't align, a provision about which there is no subtlety or 
confusion within the community at all (even though new participants 
may be confused)".

Mr Housley in response to one of my comments mentioned that the 
argument I raised was for the status quo and added that "We have 
decades of experience with that not working.  That is essentially an 
argument for a single maturity level; that is how the process is 
really working today".  As a note to the reader, I may have quoted Mr 
Housley out of context.  I presume that members of the IESG have 
followed the discussions surrounding this draft to understand the context.

The Sponsoring Area Director mentioned that the opposing opinions 
were more about a desire to do something else than specific 
objections about this proposal.  An Area Director generally sponsors 
documents that he or she believes in.  I would like to point out that 
even if a desire to do something else was tabled as a proposal, my 
perception is that it would be difficult to have such a proposal 
sponsored by the relevant Area Director.

Mr Crocker and Mr Housley are listed as authors of STD 71 and STD 70 
respectively.  It would be informative if they could comment on the 
impediments they came across in advancing their documents to Full 
Standard.  Mr Gellens and Mr Klensin might also be able to comment on 
the impediments given that they are listed as the authors of a soon 
to be published STD.

Regards,
-sm

---

From: John C Klensin <[email-masked]>
Date: Fri, 02 Sep 2011 16:20:19 -0400

--On Tuesday, August 30, 2011 23:17 +0300 Jari Arkko
<[email-masked]> wrote:

> I have reviewed the discussion from the last call on this
> document.
> 
> My conclusion as the sponsoring AD is that we have consensus
> to move forward. There was clearly a constituency who believed
> this is a good (albeit small) step forward. A number of other
> people did not care so much; did not believe there was either
> harm or benefit. I also saw a couple of opposing opinions,
> though some of them were more about a desire to do something
> else than specific objections about this proposal. I will be
> recommending that the IESG approve the draft.

Jari,

Like Scott, I wonder if there is some misunderstanding here.
Part of the problem is the way that this draft was developed and
the discussion has been handled, despite your heroic efforts.
For example:

(1) If someone says "we should be looking in this different
direction instead", the response has been "irrelevant to
consideration to this proposal, so it should go forward".  The
irrelevancy is debatable, but that may be another issue.

(2) If someone says "the proposal claims to solve problem X, but
there is no evidence for that", the assertion about what
problems are being solved is removed, but there is no
substantive change to the draft.

(3) If someone says "this solves no problem", the response has
been that things have been broken for years and therefore this
proposal should be approved.   (The difficulty with that logic
should be clear.)  Sometimes that has been accompanied by a
claim that it is the only proposal on the table and should
therefore be adopted (even though that statement isn't true and,
true or not, would never even be considered if we were
considering a protocol specification).  One of the co-authors
has recently argued for a very high standard of compelling
necessity to make changes to important processes or related
documents, but that criterion obviously does not apply to this
document.

(4) There have been a few arguments made that making this sort
of change --without compelling justification and without
evidence that it would accomplish anything-- would actually be
harmful.  There has been no substantive response to those
arguments.   In normal Last Calls, such comments are known as
"unresolved issues" and the sponsoring AD does not move the
document forward until they are addressed (or even dismissed) in
some substantive way.

(5) This is probably off-topic unless someone decides to appeal,
but, to a certain extent, the processing of this document points
out a far more significant problem with the handling of process
change suggestions in the IETF.  The IESG holds a discussion.
An IESG member prepares a draft.  An IESG member (the same one
or a different one - it makes a difference, but not much)
decides what other process change proposals can be considered
(either at the same time or otherwise).  While the IESG would
normally decide that anything that has produced this much
controversy needs a Working Group to consider alternatives and
get a real consensus determination, the IESG decides that no WG
will be considered for this work (the claim that previous WGs
addressed to process issues have been a problem --one with which
I personally agree-- may not be relevant unless the IESG is
ready to consider other review and mechanisms).   The IESG gets
to pick and choose which arguments for and against the proposal
"count" -- normal, but see (4) above and the many "solves no
problem" comments.  And then the IESG decides to advance the
document.

(6) Unfortunately, although the document has improved
significantly since -00 --by removing material for which there
was little or no support and some question about relevance and
by removing unsupportable claims-- the basic pattern outlined in
(5) has been perceived as inevitable, i.e., that this
AD-produced draft, produces in response to a discussion and
conclusion already reached by the IESG, was going to go through
because the IESG had prejudged the ultimate outcome before the
draft was written.  Whether that perception is correct or not,
it leads all but the most persistent members of the community to
tune out and stop making comments, either early on or after
several rounds.   I am not going to take a position about
consensus among some silent majority because I simply don't know
how those who are not speaking up feel, but I think the
community should exercise caution about the possibility of
consensus-by-exhaustion in any discussion that has dragged on as
long as this document and its predecessors have been debated on
the IETF list.

regards,
    john

---

From: Keith Moore <[email-masked]>
Date: Fri, 2 Sep 2011 16:37:05 -0400

Honestly, the thing that is the most broken about this draft is the idea that there's something wrong with our process because few drafts make it to full standard, so the solution is to short-circuit the process.  The transition from Draft to Full Standard is the least of the problems with our process.  And the sooner we stop trying to fix irrelevancies, the better.

Keith

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 2 Sep 2011 22:48:42 +0200

On 2 September 2011 22:20, John C Klensin wrote:

> I simply don't know how those who are not speaking up feel

https://profiles.google.com/103449397114700758824/posts/1KALM7oLKzi

---

From: Ross Callon <[email-masked]>
Date: Fri, 2 Sep 2011 17:29:31 -0400

In looking through this discussion, I see:

 - People saying that moving from 3 steps to 2 steps is a small step in the right direction, lets do it. Many people who have said this (including I) have been silent for a while quite possibly because they have gotten frustrated with the endless discussion. 

 - People saying that there are other more important problems that we should be focusing on. Therefore, rather than either making this simple change or discussing other possible improvements in the process, instead let's debate this simple step forever and never get anything done. 

 - People saying that this step won't do anything. 

Two things that I don't seem to have picked up on: (i) Any consensus that a 3 step process is better than a 2 step process; (ii) Any hint of moving towards an agreement on other things that we might do to improve the process. 

I think that we should go to a "two maturity level" process, be done with this little step, and also enthusiastically encourage people to write drafts that propose *other* changes to the process. Then at least we can be debating something different 6 months from now than we were debating last year. 

Ross

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of John C Klensin
Sent: Friday, September 02, 2011 4:20 PM
To: Jari Arkko
Cc: [email-masked]
Subject: Re: Conclusion of the last call on draft-housley-two-maturity-levels

Jari,

regards,
    john

_______________________________________________
Ietf mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Keith Moore <[email-masked]>
Date: Fri, 2 Sep 2011 17:34:53 -0400

> Two things that I don't seem to have picked up on: (i) Any consensus that a 3 step process is better than a 2 step process; (ii) Any hint of moving towards an agreement on other things that we migh…

(iii) Any consensus that a 2 step process is better than a 3 step process.

---

From: [email-masked]
Date: Fri, 02 Sep 2011 14:36:42 -0700 (PDT)

>  - People saying that moving from 3 steps to 2 steps is a small step in the
> right direction, lets do it. Many people who have said this (including I) have
> been silent for a while quite possibly because they have gotten frustrated with
> the endless discussion.

Ross, I'm right there with you. I fully support this document at worse a small
incremental step that clears away the some brush (at best it may actually turn
out to be quite valuable) and I'm completely frustrated that this discussion is
continuing.

This really needs to stop now. And yes, some people aren't happy with the
outcome. Thems the breaks.

>  - People saying that there are other more important problems that we should
> be focusing on. Therefore, rather than either making this simple change or
> discussing other possible improvements in the process, instead let's debate
> this simple step forever and never get anything done.

At least part of the problem is lack of agreement on what the issues are. Even
if this step is a waste of time - I think that's unlikely but let's suppose - 
at least it will make it clear where the problems *aren't*. 

> Two things that I don't seem to have picked up on: (i) Any consensus that a 3
> step process is better than a 2 step process; (ii) Any hint of moving towards
> an agreement on other things that we might do to improve the process.

Well, that's the real problem, isn't it? Even if you believe this is a
distraction and even actively harmful, it's not like we've been able to move
past it either. The "running code" result here seems pretty clear, and it does
not argue in favor of another round of discussion.

> I think that we should go to a "two maturity level" process, be done with
> this little step, and also enthusiastically encourage people to write drafts
> that propose *other* changes to the process. Then at least we can be debating
> something different 6 months from now than we were debating last year.

+100

				Ned

---

From: "James M. Polk" <[email-masked]>
Date: Fri, 02 Sep 2011 16:57:37 -0500

At 04:29 PM 9/2/2011, Ross Callon wrote:
>  - People saying that moving from 3 steps to 2 steps is a small 
> step in the right direction, lets do it. Many people who have said 
> this (including I) have been silent for a while quite possibly 
> because they have gotten frustrated with the endless discussion.

I think there are many that have voiced their frustrations that this 
draft isn't addressing the more important issue (or issues) in their 
minds. I don't see a consensus on what that 1 issue is, but many 
(including I) have said it's the problem of such a high hurdle to get 
a draft to PS. Because this draft isn't addressing that problem, I'm 
frustrated with this draft - because - I don't know that if this 
draft were to RFC that the high hurdle for PSs is the next thing tackled.

OTOH, if the high hurdle for PSs were what we worked on initially, 
and solve it, then I'd probably be much more comfortable with this 
draft progressing (then having started to appreciate what this means 
as a second step where getting a draft to PS is the first step).

just my opinion

James

>  - People saying that there are other more important problems that 
> we should be focusing on. Therefore, rather than either making this 
> simple change or discussing other possible improvements in the 
> process, instead let's debate this simple step forever and never 
> get anything done.
>
>Two things that I don't seem to have picked up on: (i) Any consensus 
>that a 3 step process is better than a 2 step process; (ii) Any hint 
>of moving towards an agreement on other things that we might do to 
>improve the process.
>
>I think that we should go to a "two maturity level" process, be done 
>with this little step, and also enthusiastically encourage people to 
>write drafts that propose *other* changes to the process. Then at 
>least we can be debating something different 6 months from now than 
>we were debating last year.
>
>Ross
>

---

From: Jari Arkko <[email-masked]>
Date: Sat, 03 Sep 2011 01:05:58 +0300

SM,

> The Sponsoring Area Director mentioned that the opposing opinions were more about a desire to do something else than specific objections about this proposal.  An Area Director generally sponsors documents that he or she believes in.  I would like to point out that even if a desire to do something else was tabled as a proposal, my perception is that it would be difficult to have such a proposal sponsored by the relevant Area Director.

I already responded to Scott, clarifying what I had said about the objections.

But what I really wanted to say here was a response to your concern about those proposals to do something else. Let me just state this clearly. I know I would be VERY happy to sponsor many different kinds of improvement proposals. In sequence or in parallel with the current proposal. I'm already sponsoring another one (though it is destined to an IESG statement and somewhat related to this one, but it could have been something else too). I think I'm speaking also for the rest of the IESG in this. Please make the proposals.

Of course, this does not mean that I would sponsor anything -- I do have to believe in it, I do have to think it has a reasonable chance of success. And I'm not too eager to take draft-grand-unified-theory-of-ietf-process-00.txt while everything else has to sit still. But if you have been sending e-mail to the list that we should not waste our time with Russ' draft and that we should do something else instead -- here's an idea. Send that something to us, and we'll get it done! Simple, no?

Jari

---

From: Keith Moore <[email-masked]>
Date: Fri, 2 Sep 2011 18:12:52 -0400

> outcome. Thems the breaks.

As far as our process is concerned, the question is, do we have rough consensus to accept it?  I think it's dubious that we have such consensus, and apparently so do others.   

Personally I think this proposal is Mostly Harmless, so I'm willing to hold my nose about it.   But I'm very concerned about the argument that the default assumption should be that we change our process even in the absence of consensus to do so.   

Regarding the proposal, I get the impression that people are mostly in three camps:

1) Even if this is a baby step, it's a step in the right direction.  Or even if it's not a step in the right direction, taking some step will at least make it possible to make some changes in our process.  Maybe we'll not like the results of taking this step, but at least then we'll have learned something, and if the result is clearly worse we'll be motivated to change it.  
(I call this "change for the sake of change")

2) Fixing the wrong problem doesn't do anything useful, and will/may serve as a distraction from doing anything useful.  
(I call this "rearranging the deck chairs")

3) People should stop arguing about this and just hold their noses about it, because the arguing will make it harder to do anything else in this space.  
(I call this "Oceana has always been at war with Eurasia".  Ok, that's probably too harsh, but it's what immediately comes to mind.)

All of these are defensible theories.    As it happens, I don't believe #1 applies in this space, I do believe #2, and I have to admit that #3 does happen.  

The arguments that people are giving in favor of approving this bother me more than the proposal itself does.  (I'm a firm believer that good decisions are highly unlikely to result from flawed assumptions, and flawed assumptions often affect many decisions.  So challenging a widely-held flawed assumption is often more important than challenging any single decision.)

The core problem, I suspect, is that we don't really have any consensus on what IETF's role is.   Is it to help ensure that the Internet works well?  Or is it to enable vendors to ship products that use new/updated protocols as quickly as possible?  These two aren't diametrically opposed, but there's a fair amount of tension between them.

Keith

---

From: Jari Arkko <[email-masked]>
Date: Sat, 03 Sep 2011 01:31:01 +0300

First, I'm in full agreement with Ross.

Second, for the record and as a response to Keith, my read of the discussion on the last call was the biggest group of responses said that we should move forward with the draft. There were two smaller groups, those with a clear objection and those with roughly a "no-objection" or "it does not cause harm" opinion (and a group who seemed to discuss orthogonal issues and not respond to the question). I could of course have made mistakes in this determination, but I thought it was rough (perhaps very rough) consensus.

Of course, it gets more interesting if you start thinking about the reasons why people wanted to move forward. Keith's latest e-mail has interesting theories about those. I don't think anyone thinks this is the priority #1 process fix for the IETF. For me, cleaning cruft from the IETF process RFCs is a big reason for supporting this work. And I must admit that we seem to be in a place where its very, very hard to make _any_ process RFC changes. Getting one done, even if its a small change would by itself be useful, IMO. Finally, I think two levels are enough.

Jari

---

From: Keith Moore <[email-masked]>
Date: Fri, 2 Sep 2011 18:38:03 -0400

My main conclusion for the moment is that Last Call comments should indicate first of all a definite Support or Oppose for the decision at hand if they are to be counted for or against consensus.

I'm not going to state a position, which means that you should not count me as either for or against the proposal.

Keith

---

From: Barry Leiba <[email-masked]>
Date: Fri, 2 Sep 2011 18:41:34 -0400

I'm in the group that likes this document, thinks it will help us move
forward, and thinks we should stop babbling and just do it.

That said, I have a issue with what Ross says:

> Two things that I don't seem to have picked up on: (i) Any consensus that a 3 step
> process is better than a 2 step process; (ii) Any hint of moving towards an agreement
> on other things that we might do to improve the process.

The issue is that we aren't looking for consensus that what we have is
better than what's being proposed.  What we're looking for is
consensus that what's being proposed is better than what we have.
That is, a proposal for change needs to establish consensus FOR THE
CHANGE, not the other way around.

It's up to Jari and the rest of the IESG to determine whether we have
that rough consensus.  I'm not sure.  In numbers, we seem to -- it's
mostly a vocal few who are urging us not to do it.  The thing is that,
as we know, rough consensus isn't just a numbers game, and one or two
reasonable minority arguments can derail things.

Barry

---

From: John C Klensin <[email-masked]>
Date: Fri, 02 Sep 2011 19:15:38 -0400

--On Friday, September 02, 2011 14:36 -0700 Ned Freed
<[email-masked]> wrote:

>...
> Well, that's the real problem, isn't it? Even if you believe
> this is a distraction and even actively harmful, it's not like
> we've been able to move past it either. The "running code"
> result here seems pretty clear, and it does not argue in favor
> of another round of discussion.

To make one thing clear that I may not have been clear enough
about: I felt a need to respond to Jari's summary.  I did not
want to set off another round of discussion.  I do not think
another round of discussion would be in the least useful: as far
as I can tell, no one who is expressing opinions about whether
or not we should adopt this change now has changed opinions on
that subject (sometimes the reasons stated have changed).  That
doesn't suggest that more discussion would change any minds.

>From that point of view, I believe the _only_ question at this
point, with this process, is whether there is sufficient
consensus that this proposal meets the community's norms for
making process changes.  I've heard all of the following argued
to be those norms for this type of proposal:

(1) No worse than what we have (i.e., no proof that three levels
are better than two)

(2) Likely useless, but the burden is on those who don't like it
to prove that rather than on those who advocate it to
demonstrate its utility.

(3) Possibly useful and harmless at worst.  

(4) Possibly useful but possibly harmful.

(5) Definitely useful and a compelling argument established for
this change.

For the first, I think that consensus probably exists, but I'm
not on the IESG and may not have a correct perspective.  For (2)
- (4), I really don't have a good estimate of how the community
feels: figuring that out is why we pay the IESG the big bucks.
And I don't think anyone has really made a compelling case for
the 5th.  YMMV.

And, fwiw, I think it would be a pity to prolong this discussion
by debating those categories, either as to whether they are the
correct categories or as to which one we should be using.  To
paraphrase a recent note from Russ, if someone feels a need to
do that, at least start another thread.

    john

---

From: [email-masked]
Date: Fri, 02 Sep 2011 16:07:51 -0700 (PDT)

> As far as our process is concerned, the question is, do we have rough
> consensus to accept it?  I think it's dubious that we have such consensus, and
> apparently so do others.

Simply put, I've watched the responses to this fairly closely, and I completely
disagree with your assessment.

> Personally I think this proposal is Mostly Harmless, so I'm willing to hold
> my nose about it.   But I'm very concerned about the argument that the default
> assumption should be that we change our process even in the absence of
> consensus to do so.

> Regarding the proposal, I get the impression that people are mostly in three

Well, none of these describe my own position, which is that eliminating the
three step process will at a minimum act as an incentive to move more documents
along. (You, and most others engaging in this debate, routinely neglect the
psychological factors involved.)

I can easily name a dozen RFCs, all currently at proposed, that I for one will
be strongly incented to work to advance if this step is taken. And there isn't
a chance in hell that I'll bother with any of them if this step doesn't happen,
especially after the recent debacle surrounding the attempt to move 4409bis to
full standard, and more generally given how the entire YAM experiment played
out. I'm sorry, but passing down the advancement gauntlet is plenty hard enough
to do once. Requiring it be done twice? Been there, done that, not remotely
interested in doing it again.

Additionally, by simplifying the process, we will gain essential insight into
where other problems lie. Without such simplification I see no chance at all at
making progress on any of these issues.

> 1) Even if this is a baby step, it's a step in the right direction.  Or even
> if it's not a step in the right direction, taking some step will at least
> make it possible to make some changes in our process.  Maybe we'll not like
> the results of taking this step, but at least then we'll have learned
> something, and if the result is clearly worse we'll be motivated to change it.

That last substantially and obviously mischaracterizes this position. In fact
I strongly recommend that you stop trying to summarize complex position with
cute - and utterly wrong - phrases like this. This is annoying and
quite unhelpful.

> 2) Fixing the wrong problem doesn't do anything useful, and will/may serve
> as a distraction from doing anything useful.

> 3) People should stop arguing about this and just hold their noses about it,
> because the arguing will make it harder to do anything else in this space.
> (I call this "Oceana has always been at war with Eurasia".  Ok, that's
> probably too harsh, but it's what immediately comes to mind.)

Actually, I think there are a substantial numer of people who believe exactly
the opposite of this.

> All of these are defensible theories.    As it happens, I don't believe #1
> applies in this space, I do believe #2, and I have to admit that #3 does
> happen.

> The arguments that people are giving in favor of approving this bother me
> more than the proposal itself does.  (I'm a firm believer that good decisions
> are highly unlikely to result from flawed assumptions, and flawed assumptions
> often affect many decisions.  So challenging a widely-held flawed assumption is
> often more important than challenging any single decision.)

Well, the main argument I'm giving is based on my own perception of the effect
this will have on myself and similarly minded people as a contributor. If you
think that assessment is incorrect, then I'm sorry, but I think you're being
extraordinarily foolish.

				ned

---

From: [email-masked]
Date: Fri, 02 Sep 2011 16:27:20 -0700 (PDT)

> Second, for the record and as a response to Keith, my read of the discussion
> on the last call was the biggest group of responses said that we should move
> forward with the draft. There were two smaller groups, those with a clear
> objection and those with roughly a "no-objection" or "it does not cause harm"
> opinion (and a group who seemed to discuss orthogonal issues and not respond to
> the question). I could of course have made mistakes in this determination, but
> I thought it was rough (perhaps very rough) consensus.

FWIW, this matches my own assessment almost exactly.

> Of course, it gets more interesting if you start thinking about the reasons
> why people wanted to move forward. Keith's latest e-mail has interesting
> theories about those. I don't think anyone thinks this is the priority #1
> process fix for the IETF.

Agreed.

> For me, cleaning cruft from the IETF process RFCs is a big reason for
> supporting this work. And I must admit that we seem to be in a place where its
> very, very hard to make _any_ process RFC changes. Getting one done, even if
> its a small change would by itself be useful, IMO. Finally, I think two levels
> are enough.

Cruft elimination is also a Good Thing.

				Ned

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 03 Sep 2011 11:42:12 +1200

On 2011-09-03 09:29, Ross Callon wrote:
...
> I think that we should go to a "two maturity level" process, be done with this little step, and also enthusiastically encourage people to write drafts that propose *other* changes to the process. T…

+1. Let's please not hear all over again the arguments that were made during
the *concluded* last call. The consensus is rough; time to move on.

   Brian

---

From: Keith Moore <[email-masked]>
Date: Fri, 2 Sep 2011 20:12:01 -0400

> disagree with your assessment.

ok.

> be strongly incented to work to advance if this step is taken.

At the risk of playing devil's advocate, how will that help?  Will the specifications significantly improve in quality and interoperability improve as a result?   Will the blessing of these documents as Internet Standards result in wider implementation and thus greater benefit to users?   (not knowing which RFCs you're talking about, I can't even guess)

From my perspective there's little problem with implementing and deploying at Proposed and having documents stay at Proposed indefinitely, provided we can ensure that the specifications are of high quality by the time they get to Proposed.  And given that people do tend to implement and deploy at Proposed, there's only marginal benefit to promoting them to anything else - except on those occasions where this serves as a carrot to fix bugs in the original spec that people might otherwise live with.  And it's not clear to me that the proposed change increases the incentive to do either.

> making progress on any of these issues.

Okay, I can see that as a possibility.  Sometimes when undertaking a great task, it doesn't matter what subtask you pick to do next, as long as you do something.   Momentum is often more important than doing things in order of importance.   My question is then, how many people think that we need to undertake a great task where our process is concerned, and how many of those think that given current political conditions, if we undertake such a task, we're likely to end up with something substantially better than we have now?  (I'm open to the idea but skeptical)

> cute - and utterly wrong - phrases like this. This is annoying and

There are definitely cases where "a journey of a thousand miles begins with a single step", I'm just skeptical that that argument applies in this specific case. 

> Actually, I think there are a substantial numer of people who believe exactly

I'm not sure I understand what you mean here.   Are you saying that there are a substantial number of people who wish to make it harder to do anything at all in this space, so they keep arguing about it?  Or something else?

> think that assessment is incorrect, then I'm sorry, but I think you're being

I think you're in an excellent position to understand how approval or disapproval of this document will affect your interest in doing work on the documents you mentioned, and I'm sure you're not the only one who would be encouraged by such a change to our process.

Keith

---

From: [email-masked]
Date: Fri, 02 Sep 2011 17:24:42 -0700 (PDT)

> At the risk of playing devil's advocate, how will that help?  Will the
> specifications significantly improve in quality and interoperability improve
> as a result?

First of all, since when is improving specifications the sole purpose of the
IETF standards advancement process? As I understand it, the goal behind having
different document statuses is for the labels to tell implementors how well
things are likely to actually work.

In other words, I categorically reject your implied assumption here that the
IETF advancement process only makes sense when specifications are improved
along the way. Plenty of specifications currently at proposed are of sufficient
quality to be full standards right now. Our problem is right now the process is
so onerous few people bother, so the presence of the higher label is useful but
the absence of it is not.

But even if I were to accept your tacit assumptions about the purpose of the
process - which I most assuredly do not - the answer is yes, some and perhaps
all of these specifications will be improved along the way, and at least a
couple are likely to be significantly improved. Features that in hindsight
turned out to be problematic will be removed, prose will be checked and
tightend, etc. etc.

If you think specification advancement doesn't receive this level of scrutiny
and resulting improvement currently, you haven't been paying attention.

> Will the blessing of these documents as Internet Standards result in wider
> implementation and thus greater benefit to users?   (not knowing which RFCs
> you're talking about, I can't even guess)

Keith, you know as well as I do that there can be no guarantess that the
Internet will care about anything we do, let alone see value in such
advancements. But given past experience it seems very likely that at least some
benefits along these lines will be accrued.

> From my perspective there's little problem with implementing and deploying at
> Proposed and having documents stay at Proposed indefinitely, provided we can
> ensure that the specifications are of high quality by the time they get to
> Proposed.  And given that people do tend to implement and deploy at Proposed,
> there's only marginal benefit to promoting them to anything else - except on
> those occasions where this serves as a carrot to fix bugs in the original spec
> that people might otherwise live with.  And it's not clear to me that the
> proposed change increases the incentive to do either.

And what happens in this world of only proposed standards when, as is often the
case despite all the absurd added scrutiny we now apply to proposed standards,
something problematic slips past us all? The simple fact that we almost always
make changes, and often significant ones, to documents that advance, shows we
don't always get it right. And when this happens what's the alternative? Once
something is approved it can be very difficult to get consensus to move it
historic.

You appear to be assuming that if we stare at stuff long enough and review it
hard enough we'll find all the issues in one step. Sorry, it never worked this
way in the past, and as the complexity of specifications has increased, it
works even less well now than ever before. Specifications have to be tried, and
tried at scale, to tell if they are any good.

Indeed, past experience indicates that the sorts of additional review you have
repeatedly advocoted for in these discussions usually backfires: It has an
uncanny way of turning the process into a beauty contest were important work
gets stifled just because it ran contrary to some higher-up's beliefs about
something. Once again you're failing to take psychological factors into
account, and they *matter*.

> Okay, I can see that as a possibility.  Sometimes when undertaking a great
> task, it doesn't matter what subtask you pick to do next, as long as you do
> something.   Momentum is often more important than doing things in order of
> importance.   My question is then, how many people think that we need to
> undertake a great task where our process is concerned, and how many of those
> think that given current political conditions, if we undertake such a task,
> we're likely to end up with something substantially better than we have now? 
> (I'm open to the idea but skeptical)

My answer is I have no idea. Our current process is so complex - and
unnecessarily so - that every time we even try and discuss substantial changes
the discussion goes off in about fifty different directions. When it comes to 
the really big stuff we can't even agree on where to start, let alone get
consensus on what to try and fix.

So let's please take the small step of simplifying the process a little first,
so perhaps we can get some perspective on the big stuff. Or not - it may  well
be that this small step isn't sufficient to gain any sort of perspective, but
I've already given my reasons for believing it's useful in it's own right even
if that does not happen.

> There are definitely cases where "a journey of a thousand miles begins with a
> single step", I'm just skeptical that that argument applies in this specific
> case.

> I'm not sure I understand what you mean here.   Are you saying that there are
> a substantial number of people who wish to make it harder to do anything at all
> in this space, so they keep arguing about it?  Or something else?

I think a substantial number of people believe there is nonegligable benefit
in doing this. No nose-holding required.

> I think you're in an excellent position to understand how approval or
> disapproval of this document will affect your interest in doing work on the
> documents you mentioned, and I'm sure you're not the only one who would be
> encouraged by such a change to our process.

And IMO that's a good thing.

				Ned

---

From: SM <[email-masked]>
Date: Sat, 03 Sep 2011 03:45:01 -0700

Hi Jari,
At 15:05 02-09-2011, Jari Arkko wrote:
>But what I really wanted to say here was a response to your concern 
>about those proposals to do something else. Let me just state this 
>clearly. I know I would be VERY happy to sponsor many different 
>kinds of improvement proposals. In sequence or in parallel with the 
>current proposal. I'm already sponsoring another one (though it is 
>destined to an IESG statement and somewhat related to this one, but 
>it could have been something else too). I think I'm speaking also 
>for the rest of the IESG in this. Please make the proposals.

I sent you a message off-list.

If I had to reach a conclusion, I'd say: it's one of this situations 
where you can be as fair as it gets but you cannot please 
everybody.  You have put in more that enough effort into this.

Regards,
-sm

---

From: Russ Housley <[email-masked]>
Date: Sat, 3 Sep 2011 12:13:13 -0400

Keith:

The current IETF Standards Process has become essentially a one-step process.  The goal, as I believe is stated in the document, is gather some benefit from implementation and deployment experience.  We are not getting that today.  When we do get it, the document recycles at the same maturity level due to problems that are not addressed in this proposal, must commonly dependencies on documents that are not ready to advance.  So, the reader cannot tell when a document has the benefit of implementation and deployment experience.

My reading of the comments on earlier proposals was to focus on one problem in the document, so the handling of the dependency problem and others was removed.  I'm pleased to work on them in the future if this document gains rough consensus.

Russ

---

From: Hector Santos <[email-masked]>
Date: Fri, 02 Sep 2011 17:17:08 -0400

I can't help but see the conflicts of three/four QA ideas in action:

    "Who moved my Cheese?"
    "Getting it right... the first time!"
    "Customers are always right!"  including the follow up
    "Customers are always right .... sometimes."

Improvement come by taking risk and don't settle with just old way of 
doing things.  We try to do work with maximum quality, minimum cost 
and strive to avoid future problems or revisiting old one so that all 
parties are winners. Customers are satisfied, yet, there is a point 
where you can't satisfy all.

Its a balance. IMV, as long as the the new two maturity level process 
does not change the IETF "QA process" negatively, I don't see a 
problem with it but it does sound it will necessitate a higher, more 
rigorous document reviews.

SM wrote:
>> There were a number of smaller details raised in the discussion. But I 
>> did not see an overwhelming consensus on any specific issue to make 
>> changes. But I will ask Russ to take a look at the issue raised by 
>> Scott, whether he wants to add an informative reference to RFC 5657.
> 
> I read draft-housley-two-maturity-levels-09.  I read the messages which 
> might be interpreted as statements of support.  Mr Burger offered that 
> we are moving a baby step forward.   Mr Resnick asked "A baby step 
> toward what exactly" to which Mr Saint-Andre pointed out that "we are 
> more closely aligning our documentation with our organizational running 
> code".  The organizational running code actually sets a higher bar than 
> what is documented in RFC 2026 for the publication of a Proposed 
> Standard.  The draft does not even discuss about that.
> 
> Mr Carpenter believes that "the present situation is confusing both to 
> IETF newcomers (who may falsely believe that the IETF actually follows 
> the 3 stage process) and, worse, confusing to users of IETF standards 
> (who may falsely believe that a document isn't useful until it's 
> advanced). We, and those users, gain by reducing the confusion".  In 
> terms of document clarity, RFC 2026 taken together with 
> draft-housley-two-maturity-levels-09 only reinforces the confusion for 
> anyone who takes the time to read BCP 9.
> 
> Mr Atkinson pointed out that a change in perception alone is sufficient 
> to increase [his] own motivation.  Mr Burger confirmed that the intent 
> of the proposal is to change the perception.
> 
> Mr Halpern mentioned that the draft tries to align what we document with 
> what we do.  In a response, Mr Klensin mentioned that the draft 
> confusion within the community at all (even though new participants may 
> be confused)".
> 
> Mr Housley in response to one of my comments mentioned that the argument 
> I raised was for the status quo and added that "We have decades of 
> experience with that not working.  That is essentially an argument for a 
> single maturity level; that is how the process is really working 
> today".  As a note to the reader, I may have quoted Mr Housley out of 
> context.  I presume that members of the IESG have followed the 
> discussions surrounding this draft to understand the context.
> 
> The Sponsoring Area Director mentioned that the opposing opinions were 
> more about a desire to do something else than specific objections about 
> this proposal.  An Area Director generally sponsors documents that he or 
> she believes in.  I would like to point out that even if a desire to do 
> something else was tabled as a proposal, my perception is that it would 
> be difficult to have such a proposal sponsored by the relevant Area 
> Director.
> 
> the impediments given that they are listed as the authors of a soon to 
> be published STD.
> 
> Regards,
> -sm

-- 
Sincerely

Hector Santos
http://www.santronics.com

---

From: Ross Callon <[email-masked]>
Date: Tue, 6 Sep 2011 11:38:14 -0400

> rigorous document reviews.

I haven't heard anyone currently on the IESG say that the two step process would require "higher more rigorous document reviews". 

Ross

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 6 Sep 2011 12:11:09 -0400

> I haven't heard anyone currently on the IESG say that the two step process would require "higher more rigorous document reviews". 

That particular refusal to recognize part of reality is the thing that
annoys me about this draft and about the discussions leading to its
modification of the official process.

The causal claim asserted early in the I-D's life was that, since many
RFCs effectively live forever today at step 1 of the standards track,
IESG members feel a responsibility to make sure that an I-D is "right"
before publication as PS even though that requirement is much higher
than the RFC 2026 process requires.

As a result, proponents argued, the process would be made less onerous
by moving to a two-step process in which initial publication at step 1
is the same as RFC 2026's step 1, except that it is even easier to go
from that and get the honorific "Internet Standard" than it is today.

I find it impossible to believe that this will not result in even more
hard-line positions on the part of some IESG members when something
with which they disagree is a candidate for PS.  I see no way in which
the draft solves this problem, which remains one of its implicit
goals.  I said before, I don't care if it is published, because I
think it will have little effect.  But I think we'd better be prepared
for some IESG members to insist on the same high bar for PS that we
have under RFC 2026, regardless of what the RFC says.

A

-- 
Andrew Sullivan
[email-masked]

---

From: Keith Moore <[email-masked]>
Date: Tue, 6 Sep 2011 12:43:03 -0400

> have under RFC 2026, regardless of what the RFC says.

+1

Best statement of the problem with this document that I've seen so far.

Keith

---

From: [email-masked]
Date: Tue, 06 Sep 2011 11:26:55 -0700 (PDT)

> Best statement of the problem with this document that I've seen so far.

Except for one small problem: It's nonsensical.

Why is it nonsensical? Because you're comparing the status quo with a possible
future course of action. The one thing that's we can be certain of is things
won't remain the same. They never do. So in order to make a reasonable
comparison you have to project what's likely to happen if this document isn't
approved, then compare that with what might happen if it is.

And the future where this isn't approved is fairly easy to predict: As more and
more documents become proposed standards and then fail to progress along the
standards track - and the trend lines for this could not be clearer - we
effectively move more and more to a one-step process. The IESG has only one
rational response to that: Continue to raise the bar on the initial step to
proposed.

Will the imposition of a two step process change this? It certainly won't do so
immediately, so the likely outcome is that yes indeed, the bar will continue to
go up, at least initially, irrespective of whether or not this document is
approved. But if more documents start advancing - and yes, that's an if - that
will lessen the pressure on the initial step, and perhaps break the cycle we're
currently stuck in.

And please don't try trotting out a bunch of additional what ifs about how if
this proposal fails we can then get past this distraction (or however you would
characterize it) and address whatever it is you think the actual problems are.
Given the time that has gone into trying to make this one simple and fairly
obvious change, if it fails, the odds of someone attempting even more
fundamental changes are pretty darned low.

				Ned

---

From: Ted Hardie <[email-masked]>
Date: Tue, 6 Sep 2011 14:35:10 -0700

I actually have a lot of sympathy with Andrew's formulation, largely because
the document wants you to infer something rather than making it explicit.
 Take this text:

2.1. The First Maturity Level: Proposed Standard

  The stated requirements for Proposed Standard are not changed; they
  remain exactly as specified in RFC 2026 [1].  No new requirements are
  introduced; no existing published requirements are relaxed.

The document doesn't actually say out loud there that the requirements for
Proposed Standard have been considerably increased by IESG practice over the
years, nor does it charge subsequent IESGs to return to a faithful reading
of the actual text.  You can infer it from the refrain ("stated
requirements" and "published requirements"), but I don't think you could
fairly call it explicit.  You can certainly get that from Russ in bar or on
a mailing list, but we normally try to write our documents such that you
don't have to have shared a bar with the author to get their real intent.

If the IESG does not choose to follow-up that inference with action, we have
effectively moved from a one step standards process pretending to be a
three-step standards process to a one step standards process pretending to
have two.  That's hardly worth the electrons which have been spent in this
argument.

My personal opinion for some time has been that we ought to recognize that
the previous PS moved into "WG draft" years ago and that anything named an
RFC should be recognized as something that market will consider a standard.
(My own somewhat-tongue-in-cheek proposal was to make advancement completely
automatic, barring a stated objection, but that went nowhere.)  If we are
going to try to move the goalposts for PS back and retain more than one
maturity level with the label RFC, I think we'd have a better chance of
success if  we fess up that they were moved *and put into the document that
the community wants the IESG to use those and only those for PS*.  Without
that, the real requirements for PS remain up to the IESG of the moment--a
matter of lore rather than documented practice.

To add one final comment, I think what little consensus we have on this is
was well characterized as "by exhaustion" in John's earlier note.  It may be
no less real for that, but let's not pretend to ourselves that this is
anything but a much-needed removal of the requirement for annual review,
along with a fond hope that the future may be different from the past.

regards,

Ted

---

From: John C Klensin <[email-masked]>
Date: Tue, 06 Sep 2011 17:56:11 -0400

While I disagree with the "WG Draft" part, partially because I
think we do still derive significant value from cross-area
review, I agree with the rest of this.

FWIW, I am actually quite sympathetic to Ned's argument that the
presence of two levels past proposed and the amount of nonsense
that seems necessary with each is demoralizing and acts as an
obstruction for some people to get to the second level.  It
hasn't persuaded me that this proposal is helpful only because,
if the experience with Proposed is extended from a three levels
to two, we will see the nonsense level at the second level
increase on the basis of "last shot, so we better get it right"
reasoning (which I believe is one of the things that has driven
the threshold for Proposed to its current level).  If we could
somehow take the combination of a quest for perfection and the
notion that, if the IESG has to review something, ADs have to
find something to complain about or some way to impose their
stamp on the document out of the system, then I'd completely
believe Ned's argument... except that, with the nonsense removed
from the 2nd and 3rd levels, that incentive to remove the third
might be much reduced.

As I indicated in my note last week, the way out of these
problems seems to me to be is much less symbolic tuning of the
standards process and much more having the community express
clear support for the IESG changing the way things are done
--within the existing rules or even closer to their intent-- and
having the IESG do it.  

best,
   john

--On Tuesday, September 06, 2011 14:35 -0700 Ted Hardie
<[email-masked]> wrote:

> I actually have a lot of sympathy with Andrew's formulation,
> largely because the document wants you to infer something
> rather than making it explicit.  Take this text:
> 
>   The stated requirements for Proposed Standard are not
> changed; they   remain exactly as specified in RFC 2026 [1].
> No new requirements are   introduced; no existing published
> requirements are relaxed.
> 
> The document doesn't actually say out loud there that the
> requirements for Proposed Standard have been considerably
> increased by IESG practice over the years, nor does it charge
> subsequent IESGs to return to a faithful reading of the actual
> text.  You can infer it from the refrain ("stated
> requirements" and "published requirements"), but I don't think
> you could fairly call it explicit.  You can certainly get that
> from Russ in bar or on a mailing list, but we normally try to
> write our documents such that you don't have to have shared a
> bar with the author to get their real intent.
> 
> If the IESG does not choose to follow-up that inference with
> action, we have effectively moved from a one step standards
> process pretending to be a three-step standards process to a
> one step standards process pretending to have two.  That's
> hardly worth the electrons which have been spent in this
> argument.
>...

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 07 Sep 2011 10:01:12 +1200

On 2011-09-07 09:35, Ted Hardie wrote:
...
> RFC should be recognized as something that market will consider a standard.

And who raised the bar? It wasn't the IESG, it was the market, and more
specifically the product managers and IT managers who adopted RFC conformance
as their criterion.

I'm a bit fed up with the IESG being blamed for this, rather than being
congratulated on adapting to it.

    Brian

---

From: Keith Moore <[email-masked]>
Date: Tue, 6 Sep 2011 18:08:59 -0400

On Sep 6, 2011, at 2:26 PM, Ned Freed wrote:

> effectively move more and more to a one-step process.

Face it, we've effectively had a one-step process pretty much ever since 2026 was approved.   For the most part, the documents that have advanced have been those that were buggy enough to need to be fixed, but not so buggy that they had to recycle at Proposed.  We've been using "advancement" as a proxy for "maintenance" for about as long as I've been in IETF.    (Which is why what I think we need is to restructure our processes so that they actually are designed to _maintain_ our specifications rather than pretending that there's ever a situation when those specifications are "mature" in this constantly changing world.)

> The IESG has only one rational response to that: Continue to raise the bar on the initial step to proposed.

And that was indeed a perfectly rational, and reasonable, response.  Nor is there anything that changing our process can really do about that, so long as our first publications intended for public consumption are labeled "______ Standard" and as RFCs (as people think the latter means "standard" no matter how much we protest to the contrary).   Changing what happens _after_ Proposed Standard and/or RFC publication will never address this problem.

> will lessen the pressure on the initial step, and perhaps break the cycle we're

You might turn out be right, but I don't see things happening that way.  The reason is that I don't think that either implementors or the consumers of hardware and software that implement these protocols care about whether we label something as a Proposed Standard or an Internet Standard.   Proposed Standards are still going to get implemented and widely deployed.  And when they break, it's still going to be a big mess.  IESG is still going to feel a responsibility to try to do something about it.  As they should.

> characterize it) and address whatever it is you think the actual problems are.

The actual problem is that people think that deploying products based on Proposed Standards is a good idea, and our process doesn't consistently produce documents of sufficient quality to warrant that.    There are two ways to fix that problem.  One is to stop labeling our initially published specifications (intended for prototyping and testing) as either Proposed Standards or RFCs.  The other is to impose more engineering rigor on the process that leads to the creation of Proposed Standards.

> fundamental changes are pretty darned low.

Except that, for some reason, the "obvious" change is obviously wrong to many of us.

Keith

---

From: Julian Reschke <[email-masked]>
Date: Wed, 07 Sep 2011 00:12:01 +0200

> congratulated on adapting to it.

Well, if that's really what happened, then 
draft-housley-two-maturity-levels seems to solve the wrong problem.

Best regards, Julian

---

From: Keith Moore <[email-masked]>
Date: Tue, 6 Sep 2011 18:12:40 -0400

> congratulated on adapting to it.

+1.

IESG has been doing its job.

More broadly, no system of rules governing decision-making can work well in the absence of (a) some flexibility in the interpretation in the rules; and (b) competent, conscientious, well-intended people who will interpret those rules in such a way as to get appropriate and useful results.

---

From: Dave Cridland <[email-masked]>
Date: Tue, 06 Sep 2011 23:13:01 +0100

On Tue Sep  6 23:01:12 2011, Brian E Carpenter wrote:
> And who raised the bar?

I couldn't give a damn. I'm far more concerned with who's lowering it.

To rephrase - where the bar is is significant, not how it got there.  
Moving it further up would not have real harm on the end result -  
although it does incur us more pain in meeting the bar - but lowering  
it does have problematic knock-on effects.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Keith Moore <[email-masked]>
Date: Tue, 6 Sep 2011 18:13:48 -0400

On Sep 6, 2011, at 5:35 PM, Ted Hardie wrote:

> The document doesn't actually say out loud there that the requirements for Proposed Standard have been considerably increased by IESG practice over the years, nor does it charge subsequent IESGs to return to a faithful reading of the actual text. 

Is IESG really misreading "no known technical omissions with respect to the requirements placed on it"?   

If the bar has been raised since the publication of 2026, might this actually be reasonable given that the Internet is much larger, more diverse, and more hostile than it used to be?

Keith

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 07 Sep 2011 10:59:43 +1200

> draft-housley-two-maturity-levels seems to solve the wrong problem.

The problem it is claimed to solve is an easier one: making it
more likely that people will do the work to progress *beyond* PS.

I don't see it as having the slightest impact on the quality or quantity
of PS documents. That is a different, and harder, problem. And that's
why I changed the Subject header; I like it when messages have a subject
header that describes the content, not the content of another thread
that finsihed a while back.

     Brian

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 6 Sep 2011 17:01:29 -0600

>> RFC should be recognized as something that market will consider a standard.

If you want my opinion on who raised the bar, it was the participants of the IETF that wrote many BCPs that they expected future RFCs to be compliant with. For example, take internationalization or security - 2026 does not have a lot to say about these but various other IETF documents do suggest certain things need to be done.  Though there are some BCP I might like to ignore or deprecate, overall, I think the type of things that are considered in an RFC today are much better than the situation was when 2026 was written.

I don't believe the IESG raised the bar, I think the community raised it in a series of IETF Last Calls. And I think this is good - if this document were lowing the PS bar from what it is today, I'd be strongly objecting to it. The problem in my mind is getting work done quickly, not figuring out how to lower the quality of our work. 

Cullen

and on the irrelevant side note category 

1) for just about all people I work with using or deploying things on the internet, RFC == standard regardless of type of RFC 

2) though I don't see this draft helping a lot, I applaud those that are trying to make things better and perhaps when this draft is "running code", I will see that it helps more than I expected. If it does not harm, it's OK with me. The reason I don't' see this changing what happens is that I don't see how this draft changes the incentives for progression beyond PS. 

3) we do have a three tracks in widespread use - the first track is "looks like dog food we can ship and see if the dogs will eat it" (PS), "this is awful dog food and I would not feed it to my children but someone thinks they can make a buck on it so lets ship it" (Info), and "many experts consider this dog food either toxic or against their religion but to stop the whining we are feeding it to you any way - sort of like the Canadian red cross tainted blood" (Experimental). I find all these tracks, and the ISE, very useful in allowing the IETF to get important work done. 

> It wasn't the IESG, it was the market, and more

---

From: Ted Hardie <[email-masked]>
Date: Tue, 6 Sep 2011 16:33:40 -0700

On Tue, Sep 6, 2011 at 3:13 PM, Keith Moore <[email-masked]>wrote:

> of the actual text.
Read further to:

   However, since the content of Proposed Standards may be changed if
   problems are found or better solutions are identified, deploying
   implementations of such standards into a disruption-sensitive
   environment is not recommended.

The IESG has been working to the assumption that Proposed Standards will be
widely deployed into all environments for a long time.  That may well be an
appropriate response to the deployment practice (heck, if "the internet runs
on internet drafts" we're lucky that we don't have an IESG review step
before i-d publication).  But if the result of this exercise is that the bar
for PS stays as-is and the bar for the second stage merges, we will retain
what is a functionally a one-stage standards process.  We can certainly live
with that (we live with it now), but it means we are changing out a standard
that doesn't accurately reflect what we do now for one that doesn't
accurately reflect what we will do.

regards,

Ted

> If the bar has been raised since the publication of 2026, might this
> actually be reasonable given that the Internet is much larger, more diverse,
> and more hostile than it used to be?
>
> Keith
>
>

---

From: Ted Hardie <[email-masked]>
Date: Tue, 6 Sep 2011 16:38:26 -0700

On Tue, Sep 6, 2011 at 3:12 PM, Julian Reschke <[email-masked]>wrote:

> Well, if that's really what happened, then draft-housley-two-maturity-**levels
> seems to solve the wrong problem.
>
> Best regards, Julian
>

In at least one reading, it could be said that this draft is trying to push
against that market perception by re-lowering the bar for PS, contrary to
the market reality.  I occasionally find a little windmill tilting
refreshing, but I'm confused, Brian, as to why you both want the IESG
congratulated for adapting to that reality and simultaneously wants them to
adopt this.

But possibly a hobgoblin is preying on my little mind, despite my being
neither statesman, nor philosopher, nor divine.

Ted

---

From: Fred Baker <[email-masked]>
Date: Tue, 6 Sep 2011 16:48:20 -0700

On Sep 6, 2011, at 4:33 PM, Ted Hardie wrote:

> The IESG has been working to the assumption that Proposed Standards will be widely deployed into all environments for a long time.  That may well be an appropriate response to the deployment practice (heck, if "the internet runs on internet drafts" we're lucky that we don't have an IESG review step before i-d publication).  But if the result of this exercise is that the bar for PS stays as-is and the bar for the second stage merges, we will retain what is a functionally a one-stage standards process.  We can certainly live with that (we live with it now), but it means we are changing out a standard that doesn't accurately reflect what we do now for one that doesn't accurately reflect what we will do.  

Agreed. That has been my primary puzzlement with this entire discussion.

The problem that "cycling at Proposed" solves is "what if I need to change the technology in some way". "Foo updates bar" does the same. Generally speaking, we don't get a rewrite until A has been updated by B1, B2, and B3, B2 has been obsoleted by C, and C has been updated by D. What Draft Standard was supposed to fix was a raft of testing coupled with that rewrite of the specification that also removed cruft that wasn't used.

Frankly, the only thing I ever figured out that "Full Standard" was useful for was "obsolete" (he ducks).

I wonder if we would be better off discarding the concept of layers of standards, call PS "Standards track", and instead specify a way to report interoperability tests. If we have a document A that has been updated by B and someone has tested several implementations of A+B, could they say "I tested A+B in this configuration, which used features A.1, A.2, A.3, A.5, and B.1, with these results", without forcing the rewrite or the major conniption fits that DS involves.

Folks in fact do interoperability tests with some regularity. They do them for equipment they want to buy, they do bake-offs, and they do other things.

---

From: Keith Moore <[email-masked]>
Date: Tue, 6 Sep 2011 19:55:19 -0400

On Sep 6, 2011, at 7:33 PM, Ted Hardie wrote:

> but it means we are changing out a standard that doesn't accurately reflect what we do now for one that doesn't accurately reflect what we will do.  

Yup.   And IMO we're better off with the old incorrect statement than a new one.  At least with the current situation a great many people know how things work.

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 6 Sep 2011 19:58:28 -0400

On Sep 6, 2011, at 7:48 PM, Fred Baker wrote:

> I wonder if we would be better off discarding the concept of layers of standards, call PS "Standards track", and instead specify a way to report interoperability tests.

+1.  Perhaps along with periodically updated applicability statements.

---

From: John Leslie <[email-masked]>
Date: Tue, 6 Sep 2011 20:08:55 -0400

Keith Moore <[email-masked]> wrote:
>> I wonder if we would be better off discarding the concept of layers
>> of standards, call PS "Standards track", and instead specify a way
>> to report interoperability tests.
> 

   This is a serious proposal...

   It has been a serious proposal ever since newtrk.

   Details differ, but really not significantly: some folks want to
update 2026 to do this; others want to do it first and decide later
whether to update 2026; others want to do it and let the IESG decide
whether to use it as input to deciding about advancing levels.

   Many distinctions; no real difference...

--
John Leslie <[email-masked]>

---

From: "t.petch" <[email-masked]>
Date: Wed, 7 Sep 2011 10:27:04 +0200

----- Original Message -----
From: "Keith Moore" <[email-masked]>
To: "Ted Hardie" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Wednesday, September 07, 2011 1:55 AM

On Sep 6, 2011, at 7:33 PM, Ted Hardie wrote:

> but it means we are changing out a standard that doesn't accurately reflect
what we do now for one that doesn't accurately reflect what we will do.

Yup.   And IMO we're better off with the old incorrect statement than a new one.
At least with the current situation a great many people know how things work.

<tp>
Yes, even.  The purchasers, RFPers and like that I have worked with just about
understand the difference between an I-D and an RFC, but have no concept of the
classifications used inside the IETF (PS, DS, IS) so I cannot see any
comprehension of this new scheme making it into the world at large for a long
time.  We may not be following RFC2026 but at least there is some understanding
of what we are doing.

Tom Petch
</tp>

Keith

--------------------------------------------------------------------------------

---

From: [email-masked]
Date: Wed, 07 Sep 2011 07:17:40 -0700 (PDT)

> Face it, we've effectively had a one-step process pretty much ever since 2026
> was approved. For the most part, the documents that have advanced have been
> those that were buggy enough to need to be fixed, but not so buggy that they
> had to recycle at Proposed.

Just one small problem here - every document advancement exercise I've seen in
the past two decades - and I've seen a bunch - directly contradicts your
assertions here.

In essentially every case advancement occurred when some individual or some
subgroup believed doing so was important and pushed the issue. The most common
reason for believing that is probably that the document in question replaces
some other document that's already at draft or full, and IETF rules require
advancement before the original document can be marked historic. The SNMP and
MSGFMT/SMTP specifications are both good examples of this.

In other cases advancement was the result of a strong sense that it was needed
to legitimize the specification. This is why MIME was advanced so quickly. And
please note that in the case of MIME we advanced it with minimal changes first
(RFCs 1521/1522 aren't that far from 1341/1342), and only then revised it
extensively (RFCs 2045-2049) without further advancement. So much for the
notion of using advancment as an excuse to fix stuff.

Another common reason to advance a specification has been the simple belief
that following out processes is the right thing to do. I assure you that was my
only real motivation for working to advance the SMTP extensions framework, the
SMTP size extension, and the PIPELINING extension.

And finally, there are cases where the motivations for pushing for advancment
were frankly mysterious to me. The FAX WG experience comes to mind here. To
this day I have no idea why advancment of those specifications was so
important, especially when many similar groups clearly didn't give a crap about
following the process.

Now, I have no doubt that if you looked hard enough you can find a case or two
where the primary motivation was to pry something open and fix it. However, the
opposite sentiment is far more common: "Opening that up would be a real can of
worms so please don't try and advance it". Nevertheless, the assertion that
this is the only reason for advancment is demonstrably false.

> We've been using "advancement" as a proxy for "maintenance" for about as long
> as I've been in IETF.

Wow, you really think that? I'm frankly amazed at the degree of disconnect
here.

> (Which is why what I think we need is to restructure our processes so that they
> actually are designed to _maintain_ our specifications rather than pretending
> that there's ever a situation when those specifications are "mature" in this
> constantly changing world.)

Well, now you're shifting to talking about a fundamental change of
philosophies. Tell you what - let's see if even a small change like this one is
possible first, because if it isn't a shift like this isn't even worth wasting
the electrons to discuss.

> You might turn out be right, but I don't see things happening that way.  The
> reason is that I don't think that either implementors or the consumers of
> hardware and software that implement these protocols care about whether we
> label something as a Proposed Standard or an Internet Standard.   Proposed
> Standards are still going to get implemented and widely deployed.  And when
> they break, it's still going to be a big mess.  IESG is still going to feel a
> responsibility to try to do something about it.  As they should.

There are things we have control over and things we don't. We have no control
over this. The best we can do is to make our labels meaningful - and they
aren't currently. So perhaps we should fix that, you know?

> The actual problem is that people think that deploying products based on
> Proposed Standards is a good idea, and our process doesn't consistently produce
> documents of sufficient quality to warrant that.    There are two ways to fix
> that problem.  One is to stop labeling our initially published specifications
> (intended for prototyping and testing) as either Proposed Standards or RFCs. 
> The other is to impose more engineering rigor on the process that leads to the
> creation of Proposed Standards.

That presuppoes we have the ability to actually perform such analysis without
actually trying things at some sort of scale. I'm sorry, but I've seen no 
evidence that the necessary skills for this actually exists.

				Ned

---

From: Keith Moore <[email-masked]>
Date: Wed, 7 Sep 2011 11:32:49 -0400

> MSGFMT/SMTP specifications are both good examples of this.

I am aware of those, but I consider those the rare exceptions to the general rule.  I also think that the email community has a couple of influential individuals who really believe in following the process all the way through, but that belief is not typical of IETF as a whole.

> Wow, you really think that? I'm frankly amazed at the degree of disconnect

Yes, I really think that holds as a general rule.  Again, to me the email advancement efforts look like the exceptional cases.   (I applaud those individuals for their diligence!)

> the electrons to discuss.

If it were generally clear that this change was a Good Idea, I might buy that argument.  But what you seem to be saying is that if people don't back this change even though it appears to many people to be useless at best (and harmful to some), they'll never back any change to our process even if it appears to be more useful.

> aren't currently. So perhaps we should fix that, you know?

FIne.  Let's rename "Proposed Standard" to something that doesn't contain the word "standard" (call them "frobs" for the sake of argument).  And let's not publish them as RFCs, but instead leave them as Internet-Drafts, and amend the rules for Internet-Drafts to allow "frobs" (once approved by IESG) to expire in two years rather than six months.  

> evidence that the necessary skills for this actually exists.

I agree at least somewhat with that.   I do think that our processes in general need more use of engineering discipline, but I also think that there will always be things that slip through the cracks.  Which is why we need a process that recognizes the need for specifications to be maintained in light of experience.

Keith

---

From: Hector Santos <[email-masked]>
Date: Tue, 06 Sep 2011 19:40:40 -0400

Cullen Jennings wrote:

> If you want my opinion on who raised the bar, it was the 
> participants of the IETF that wrote many BCPs that they 
> expected future RFCs to be compliant with. 

+1.

Another good example is when an unvetted informational "helper" RFC 
was fast tracked, becomes a central piece of the total integration 
with the main proposed standard RFC along with other documents and 
integration engineering conflicts fall through the "review" cracks. 
When its finally caught, goals are now less realistic and the proposed 
standard now becomes stagnant.

-- 
Sincerely

Hector Santos

---

From: JP Vasseur <[email-masked]>
Date: Wed, 7 Sep 2011 08:47:06 +0200

> congratulated on adapting to it.

Completely agree !

JP.

---

From: JP Vasseur <[email-masked]>
Date: Wed, 7 Sep 2011 08:49:14 +0200

Completely agree !

JP.

---

From: SM <[email-masked]>
Date: Wed, 07 Sep 2011 09:19:36 -0700

At 16:01 06-09-2011, Cullen Jennings wrote:
>I don't believe the IESG raised the bar, I think the community 
>raised it in a series of IETF Last Calls. And I think this is good - 
>if this document were lowing the PS bar from what it is today, I'd 
>be strongly objecting to it. The problem in my mind is getting work 
>done quickly, not figuring out how to lower the quality of our work.

draft-cheshire-dnsext-multicastdns-00 was published in 2001 and 
version 14 in February 2011.  There is an IPR disclosure.  If the 
proposal were to advance along the Standards Track, the following 
might be applicable:

   "If patented or otherwise controlled technology is required for
    implementation, the separate implementations must also have resulted
    from separate exercise of the licensing process."

The draft went through several Last Calls.  If I recall correctly, 
there were some objections raised during the Last Call.  I doubt 
whether the community is aware of what changes were agreed upon given 
that this is an individual submission which finally got processed 
after four months.

This is a draft which is going to be evaluated shortly.  It got the 
following comment:

   "Downref: Normative reference to an Informational RFC: RFC 2818"

I could nit-pick and say that the write-up mentioned that and the RFC 
is listed in the Downref registry.  Anyway, could someone please 
explain to me what is the rationale for flagging downrefs if the it's 
a one-step process in practice?

Is it as Sam Hartman said, a nice game of "publish that doc" and 
people have to get out their copy of the IETF rules, the IETF rules 
errata and the IETF player's magazine articles with rules commentary 
and figure out what is going on?

draft-hixie-thewebsocketprotocol-76 was submitted in May 2010.  There 
were at least two implementations.  Would it qualify for Proposed Standard?

draft-ietf-avt-srtp-not-mandatory-07 is the output of a working 
group.  One of the DISCUSSes mention that:

  "This draft seems to say that BCP 61 does not apply to RTP.  BCP 61
   describes the "Danvers Doctrine", which says that the IETF should
   standardize on the use of the best security available, regardless of
   national policies."

It's been over a year since the above point has been raised.  The 
draft has not been published yet.

draft-ietf-mboned-ssmping-08 has five DISCUSSes, one of which says:

  "The mboned charter says:
    This is not meant to be a protocol development Working Group.

   Is this not a protocol? In which other WGs has it been reviewed?"

I don't know who raised the bar.  It doesn't really matter unless 
people confuse finger pointing with problem solving.  The 
requirements frustrate implementers, which is understandable, when 
there is no relation between quality and known technical defects.

Regards,
-sm

---

From: Thomas Narten <[email-masked]>
Date: Fri, 09 Sep 2011 21:33:15 -0400

I am surely going to regret posting, because I have largely tuned out
of this discussion due to the endless repetition, etc. I am not
supportive of the current document, because I don't think it solves
anything. To me, it smack a bit of "change for changes sake".

One of the key problems that isn't being addressed is that mixing
"advancement" of a spec with "revising" a spec are fundmentally at
odds with each other. 

Advancing a spec is done for marketing, political, process and other
reasons. E.g., to give a spec more legitimacy. Or to more clear
replace an older one. Nothing wrong with that.

But the real reason that the IETF *should* be revising specs is to fix
bugs and improve protocol quality.

By definition, you cannot revise a spec (in a real, meaningful way)
and advance at the same time. The spirit (if not letter) of
advancement says you advance a spec, when there are implementations
*based on the spec being advanced*. That means you can't revise a spec
and at the same time have implementations derived from the revised
spec.  (You can have implementations based on mailing list
discussions, but that is NOT the same thing.)

The IETF is about making the Internet work better. That means revising
specs (from a technical perpective) when they need to be revised.

If we want to fix what's broken, we should focus on getting documents
revised (without simultaneously advancing them).
But once you do that, one quickly finds out that there are real and
sometimes  complicated
reasons why revising documents is hard.

In many cases, widely deployed protocols really need to have a revised
spec developed (and the authors will readily admit that). But that
just doesn't happen, not because of process, but because of other much
more fundamental problems. E.g., Not enough energy from the relevant
experts. key people who know a spec have moved on to other newer
technologies or other higher priority things. Fixing specs can also be
painful because some vendors won't or can't change their deployed
implementations, so don't really want an updated spec that invalidates
their implementation. etc., etc. It can be very hard for a WG to walk
the line between "we need to fix this" and "can we tweak the spec
without invalidating various deployed implementations".

IMO, these sorts of issues are the real reasons documents don't
advance more. It's not just about process.

Thomas

---

From: Mykyta Yevstifeyev <[email-masked]>
Date: Sat, 10 Sep 2011 07:47:24 +0300

Taking into account the controversy we all are able to observe on the 
mailing list, I'd like to point out several points.

1) Did the IESG consider processing this as RFC 3933 process 
experiment?  (I actually don't know whether such approach has already 
been proposed during the discussions, and whether there has been some 
outcome, so, if this has already been proposed, just re-consider.)  I 
personally see no "consensus" or "rough consensus" for this document 
being approved as BCP, i. e., on the permanent basis, but as some people 
claimed that this might be useful, processing the document as 
Experimental process RFC will allow to make the final judgment based on 
the actual experience, not the assumptions.  As soon as we find out that 
two-tier maturity levels system works, the BCP will be simple to be 
written and published; if we reach the agreement that "this is a bad 
idea", then the proposed experimental change will be rescinded, and the 
maturity system will be returned to the 2026 model.

2) How do we make the consensus judgment (in this particular case)?  As 
IETF is based on "rough consensus" model, rough consensus may only be 
claimed if the consensus-qualifier (a WG chair or Sponsoring AD, in case 
of Individual Submission) reaches the inner conviction that the idea 
proposed in the document satisfies the community, or at least its 
predominant part.  Since IETF has no formal membership, the "community" 
size may not be determined precisely, and we take into account those 
folks who participate in the discussion (who, correspondingly, found 
themselves interested in the discussed topic and are assumed to be 
knowledgeable in it) in the WG, or elsewhere.  So now, when many of the 
most experienced and most knowledgeable members of our community claim 
that the proposed change is not a good idea, or is a bad idea, or there 
is no actual problem, or there is a problem but its proposed solution 
isn't fine and has some omissions, or there are a number of other 
problems which are also to be fixed, or something else, I actually have 
no idea how the consensus-qualifier (in our case, Sponsoring AD - Jari 
Arkko) may claim "consensus" or "rough consensus" for, at least, 
adopting it as BCP.  (I'm not following the thread closely, but this is 
what I see from those messages I eventually read.)

3) Do we actually need to make cosmetic changes to our process 
documentation?  RFC 2026 was published in 1996, and precisely 15 years 
have passed.  RFC 2026 is really morally obsolete, and, in presence of 
RFC 4844, that defines RFC submission streams, was to be revised closely 
after it was published.  I see a number of drafts proposing revisions of 
RFC 2026 at 
<https://datatracker.ietf.org/doc/search/?name=Internet+Standards+Process&rfcs=on&activeDrafts=on&oldDrafts=on>, 
but none of them were processed.

BTW, RFC 1310 was published in 1992, RFC 1602 - in 1994, and RFC 2026 - 
in 1996.  I don't believe something happened in 1996 which made the 
procedures unnecessary to be aligned with the current practice.  The 
only changes made were IPR documents, PS->DS reports reqs, and IESG 
procedures for review of Independent Submissions and IRTF stream 
submissions.

More precisely, don't we need to revise RFC 2026 rather than make 
separate changes to it?

Mykyta Yevstifeyev

30.08.2011 23:17, Jari Arkko wrote:
> I have reviewed the discussion from the last call on this document.
>
> My conclusion as the sponsoring AD is that we have consensus to move 
> forward. There was clearly a constituency who believed this is a good 
> (albeit small) step forward. A number of other people did not care so 
> much; did not believe there was either harm or benefit. I also saw a 
> couple of opposing opinions, though some of them were more about a 
> desire to do something else than specific objections about this 
> proposal. I will be recommending that the IESG approve the draft.
>
> Another issue that I wanted to highlight is the question of what kinds 
> of discusses are desirable/acceptable for documents that move from PS 
> to IS. It is outside the scope of Russ' document, but generated 
> nevertheless some interest. The IESG has discussed this matter and 
> drafted some suggested guidelines. Look for a different thread on this 
> list, under "Discuss criteria for documents that advance on the 
> standards track". Feedback on the guidelines would be appreciated.
>
> Jari
>

---

From: Barry Leiba <[email-masked]>
Date: Sat, 10 Sep 2011 09:56:12 -0400

> 1) Did the IESG consider processing this as RFC 3933 process experiment?

How on Earth could that possibly work?

First, simply the fact of the experiment will almost certainly prompt
people to participate, resulting in a number of specs upgrading from
PS to IS during the experiment... regardless of whether that pattern
would continue afterward.  The experiment would be entirely tainted by
its own existence, and would appear to succeed regardless of whether
the change will actually help in the long term or not.

Second, should we decide that the experiment failed and we do not want
to continue the process change, what happens to all the IS documents
that advanced from PS during the experiment?  Are they rolled back to
DS?  Do they stay at IS?  In the former case, what does that do to
assumptions that had been made on the basis of IS status?  If the
latter case, how is that fair to documents that didn't have the
opportunity to "skip" a rung in the ladder?  And wouldn't that be even
*more* of an incentive for people to push their docs from PS to IS
during the experiment, exacerbating the first effect?

Doing this as a process experiment doesn't make any sense; we either
make this change, or we don't.  And we need to stop wasting time
arguing about it.

Barry

---

From: Russ Housley <[email-masked]>
Date: Sat, 10 Sep 2011 10:44:09 -0400

Mykyta:

> Taking into account the controversy we all are able to observe on the mailing list, I'd like to point out several points.
> 
> 1) Did the IESG consider processing this as RFC 3933 process experiment?  (I actually don't know whether such approach has already been proposed during the discussions, and whether there has been some outcome, so, if this has already been proposed, just re-consider.)  I personally see no "consensus" or "rough consensus" for this document being approved as BCP, i. e., on the permanent basis, but as some people claimed that this might be useful, processing the document as Experimental process RFC will allow to make the final judgment based on the actual experience, not the assumptions.  As soon as we find out that two-tier maturity levels system works, the BCP will be simple to be written and published; if we reach the agreement that "this is a bad idea", then the proposed experimental change will be rescinded, and the maturity system will be returned to the 2026 model.

I do not recall the whole IESG discussing this idea, but I did consider it.  I did not consider it a good fit, and until now, no one else seemed to even consider it.

> 2) How do we make the consensus judgment (in this particular case)?  As IETF is based on "rough consensus" model, rough consensus may only be claimed if the consensus-qualifier (a WG chair or Sponsoring AD, in case of Individual Submission) reaches the inner conviction that the idea proposed in the document satisfies the community, or at least its predominant part.  Since IETF has no formal membership, the "community" size may not be determined precisely, and we take into account those folks who participate in the discussion (who, correspondingly, found themselves interested in the discussed topic and are assumed to be knowledgeable in it) in the WG, or elsewhere.  So now, when many of the most experienced and most knowledgeable members of our community claim that the proposed change is not a good idea, or is a bad idea, or there is no actual problem, or there is a problem but its proposed solution isn't fine and has some omissions, or there are a number of other problems which are also to be fixed, or something else, I actually have no idea how the consensus-qualifier (in our case, Sponsoring AD - Jari Arkko) may claim "consensus" or "rough consensus" for, at least, adopting it as BCP.  (I'm not following the thread closely, but this is what I see from those messages I eventually read.)

I think that Jari explained his thought process, at least twice.

> 3) Do we actually need to make cosmetic changes to our process documentation?  RFC 2026 was published in 1996, and precisely 15 years have passed.  RFC 2026 is really morally obsolete, and, in presence of RFC 4844, that defines RFC submission streams, was to be revised closely after it was published.  I see a number of drafts proposing revisions of RFC 2026 at <https://datatracker.ietf.org/doc/search/?name=Internet+Standards+Process&rfcs=on&activeDrafts=on&oldDrafts=on>, but none of them were processed.
> 
> BTW, RFC 1310 was published in 1992, RFC 1602 - in 1994, and RFC 2026 - in 1996.  I don't believe something happened in 1996 which made the procedures unnecessary to be aligned with the current practice.  The only changes made were IPR documents, PS->DS reports reqs, and IESG procedures for review of Independent Submissions and IRTF stream submissions.
> 
> More precisely, don't we need to revise RFC 2026 rather than make separate changes to it?

Please take a look at the minutes of the PUFI BOF held at IETF 71.  The IETF community seems to think that process changes fall into one of two piles.  Either they are too insignificant to be worth the trouble or they are too big and onerous to consider.  The experience with this draft does not disprove those results.

That said, at the plenary at IETF 81, Harald suggested that we have waited so long that incremental improvements may not be the right approach, rather it was time for 2026bis.  I agree that it is needed, but I am not sure the IETF community is willing to tackle a project of that size and complexity.

Russ

---

From: Keith Moore <[email-masked]>
Date: Sat, 10 Sep 2011 11:14:00 -0400

> That said, at the plenary at IETF 81, Harald suggested that we have waited so long that incremental improvements may not be the right approach, rather it was time for 2026bis.  I agree that it is n…

I suspect that there are two underlying problems, both significant.

1. Many people would see an attempt to change the process as a "threat" of one kind or another, feeling like the result of such change under current circumstances would likely be worse than the status quo.

2. There seems to be wide perceptual variation among IETF participants as to the organization's purpose and/or what's wrong with the current process.

I think there's a need for consensus-building around these two topics (and perhaps others) before even considering whether and how to revise 2026.  In the first case, we need to find out what people see as threats and use that to bound the charter of the rewrite effort so that people won't feel like they have to be in damage control mode.  But the second topic is even more important.   

Keith

---

From: Mykyta Yevstifeyev <[email-masked]>
Date: Sat, 10 Sep 2011 19:25:56 +0300

> How on Earth could that possibly work?

Those who are interested in something working will certainly find the 
way to make it work.

> during the experiment, exacerbating the first effect?

So, for the purpose of the experiment, there will be a "2nd level", 
which is Draft Standard now and would be Internet Standard.  People 
wanting to participate in the experiment will be warned that a "2nd 
level" of Internet Standard might be of temporary nature and therefore, 
in the case of experiment failure, will be Draft Standard.  
Correspondingly, RFCs advanced to "2nd level" of IS, in the case of 
permanent adaptation, will be Internet Standard, and in case of 
returning to 2026 model, will be Draft Standard.  The term "experiment" 
implies that something will go in some way which is unusual; this is 
that unusual way.

Anyway, do we need to care what maturity level has some spec reached?  
Eg., YAM has undertaken an effort to advance RFC 4409, mail submission 
protocol spec, to Full Standard (now 4409bis is approved as FS), but, 
I'm sure, this won't affect the many implementations which use and will 
continue to use mail submission protocol independent on its current 
maturity level.  SMTP spec is now on Draft Standard, and I don't think 
it is now more widespread that when its spec was on Proposed Standard.  
Does that matter?

> make this change, or we don't.  And we need to stop wasting time

"We either make change, or we don't" - unless there is a stable 
consensus on the change, the second variant is going to be the right one.

BTW, stopping arguing is indeed a good idea; starting discussing is 
indeed a better idea (if we haven't yet).  So what we're doing - 
discussing or arguing?

Mykyta Yevstifeyev

---

From: Mykyta Yevstifeyev <[email-masked]>
Date: Sat, 10 Sep 2011 19:44:22 +0300

> I do not recall the whole IESG discussing this idea, but I did consider it.  I did not consider it a good fit, and until now, no one else seemed to even consider it.

See my previous message.

> I think that Jari explained his thought process, at least twice.

Well, what Jari explained is that (how I understand), there were 
thoughts like "good minor change", and other thoughts that don't support 
the change but, due to lack of "reasonable reasons" for people opposed 
to the change to think so, such thoughts may be overlooked.  Whereas 
there might have been such positions, I don't think such approach is 
good to claim "community consensus".

"Status of This Memo" boilerplate for BCPs include the following statement:

> It represents the consensus of the IETF community.

Does that mean "IETF community consensus"?

>
>> 3) Do we actually need to make cosmetic changes to our process documentation?  RFC 2026 was published in 1996, and precisely 15 years have passed.  RFC 2026 is really morally obsolete, and, in presence of RFC 4844, that defines RFC submission streams, was to be revised closely after it was published.  I see a number of drafts proposing revisions of RFC 2026 at<https://datatracker.ietf.org/doc/search/?name=Internet+Standards+Process&rfcs=on&activeDrafts=on&oldDrafts=on>, but none of them were processed.
>>

I concur here entirely.  (Were we revising process documentation on the 
regular basis, this wouldn't be a problem, though.)

Mykyta Yevstifeyev

---

From: Sam Hartman <[email-masked]>
Date: Sat, 10 Sep 2011 16:11:05 -0400

Hi.  I feel it's reasonable for me to speak up since I have not done so
in over a year on this document so my opinion probably has not been
counted.

1) I support moving to a two level process.

2) I've generally supported versions of this document I have read. I
have not read this version in detail.

In regard to more global issues.

I do not think the following types of comments should be considered as
objections when judging this sort of consensus:

1) You are not solving the most important problem

2) This will not do any good

Statements of those forms can be combined with objections. "This will
not do any good and might do harm so I don't support it," clearly is an
objection. Objections can be as simple and non-specific as "I don't like
it," or more actionable. More thought out objections carry more weight
in some senses. I certainly would hate to see us block on someone simply
saying "I don't like it." Either enough people say that that we fail to
have rough consensus or not enough people say that and we move on. More
detailed objections may be worth blocking on for a time to try and
resolve; we seem to be past point of diminishing returns here.

---

From: Hector <[email-masked]>
Date: Sat, 10 Sep 2011 17:14:29 -0400

The question I have is who are the beneficiaries?

My input as a implementator.

For 30 or so years, I chose to followed the IETF output as an 
commercial implementator based on sound engineering trust and faith on 
follow peers, no reason to suspect or otherwise feel I need to appeal 
anything.  For the most part, it was all good positive and successful 
adventure. No naiveness in separating what was for common good against 
those what were isolated vendor specific, proprietary in nature - yet, 
I do have the GM/WORLD conservative mindset - "Whats good for IETF, is 
good for the World!"  I don't wish to buck the system.

The wind has changed that for sure, and I wonder what is the end 
result. I do know one thing: I don't like the idea that "IETF Appeal" 
is something new (to me) I have to consider these days and when that 
seems like an insurmountable mountain to climb when it comes to even 
security conflicts and back down because of it, it becomes even more 
vexing to whats going on.  I am not use to this and the older one 
gets, you seem to accept more even when you know (or feel) there are 
problems. Only the young have the energy to fight for a cause and even 
then less these days accepting things once considered problematic.

Just I hope the IETF leaders here make the right decision.  From an 
engineering standpoint, when you couple this two-step with the 
RFC2119bis efforts, it will most likely;

   a) make people think more about implementing what they
      had trouble with implementing in the first place,

   b) raise the barriers to adoption so you have less
      implementators.

With my marketing hat on, rest assured this is leveraging material.

--
HLS

---

From: hector <[email-masked]>
Date: Sat, 10 Sep 2011 17:47:44 -0400

Whether they were planned as part of revamp plan or not, I don't see 
the two-step and the RFC2119bis efforts and recent debates here as a 
mere coincidence.

Here is how Pareto Optimality design decisions are made such that 
no-one could be made better off without making someone else worse off. 
  I only point this out that we sometimes we to accept change where 
one might not be better off.

    http://en.wikipedia.org/wiki/Pareto_efficiency#Pareto_frontier

    The Pareto frontier is particularly useful in engineering:
    by restricting attention to the set of choices that are
    Pareto-efficient, a designer can make tradeoffs within this
    set, rather than considering the full range of every parameter.

One we view we are an optimal state right now where we don't wish to 
benefit one without making someone else worst and I see the two-step 
and RFC2119bis as attempts to making Pareto Frontier Efficient 
engineering decisions.

The question is always "how much" change we want without ignoring 
those that can be made worst off.

--

Keith Moore wrote:
> ------------------------------------------------------------------------
>

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 11 Sep 2011 10:49:20 +1200

On 2011-09-11 08:11, Sam Hartman wrote:

<snip>

> 2) This will not do any good

Exactly. A very large part of the discussion has not been relevant
to the Last Call of this particular version of this particular
draft. In deciding the question, the IESG only needs to look
at comments that are relevant.

There are many other issues in 2026 that probably need attention,
but as we've learnt over the last N years, trying to tackle them
simultaneously is manifestly impossible.

    Brian

---

From: "Scott O. Bradner" <[email-masked]>
Date: Sat, 10 Sep 2011 19:30:35 -0400

>> 2) This will not do any good

now lets see, this argument seems to be that the fact that a particular process change is useless should not 
stop the IETF from adopting the change

this argument would be nonsense if applied to a proposal for a technical standard - i.e. the 
IETF should adopt a technical standard that is known to be useless -- it is no less nonsense when
applied in this case - changes for the sake of publishing new bits should not be what the IETF
spends its time on

Scott

---

From: Keith Moore <[email-masked]>
Date: Sat, 10 Sep 2011 19:34:51 -0400

> 1) You are not solving the most important problem

I don't think that was anybody's objection.  Rather, the objection were against change for the sake of change without any convincing argument that the change would improve things; and/or that the changes would actually make it more difficult to improve things.

> 2) This will not do any good

IMO, that is a valid objection.   Stability in our process is desirable; therefore change merely for the sake of change is undesirable.

Keith

---

From: Hector <[email-masked]>
Date: Sat, 10 Sep 2011 19:39:11 -0400

> simultaneously is manifestly impossible.

Please trust there is no disrespect in stating this but this *sounds* 
like there is a realization that the reasons for problematic stagnant 
proposals are now going to be view as non-valid reasons.

I don't have a particular concern if that is used for progressing and 
"labeling" proposals with a new status, but I am weary if that idea is 
used to circumvent the initial concerns.

--
HLS

---

From: John C Klensin <[email-masked]>
Date: Sat, 10 Sep 2011 20:14:43 -0400

--On Saturday, September 10, 2011 16:11 -0400 Sam Hartman
<[email-masked]> wrote:

> I do not think the following types of comments should be
> considered as objections when judging this sort of consensus:
> 

Sure.  As long as we differentiate between than and "this will
not solve the problem it claims to solve", "the problem this
addresses is not worth solving", and "this may make things
worse".   While I agree with many of the meta-comments that have
been made about this discussion, I think the some of the
supporters of the proposal have tended to lump comments along
the lines of those three together with the one you cite and its
close relatives, "you should solve some other problem instead"
and "you aren't solving all of the problem".   They are
different, at least IMO.

> 2) This will not do any good

Again, I agree, but I suggest that is different from the
argument Dave Crocker makes frequently (and better than I can)
which, if I may paraphrase, is that any process change has
significant coats and some risks and therefore it should be a
requirement that anyone advocating such a change show a
compelling need for it.  I can accept "this will not do any
good" as a category of position that should not be considered an
objection iff "no compelling requirement and/or advantage for
this has been demonstrated" still counts as legitimate objection.

If that were not the case, I can think of any number of protocol
specifications  that could be decorated with a sufficient number
of optional-and-useless features to make the most option- and
profile-happy of SDOs dance with joy.  Such decorations would be
of particular interest in the security area where we have
traditionally felt that options that are useless and likely to
be badly-implemented and little-used often increase risk by
providing unnecessary attack vectors. :-(

   john

    john

---

From: Eric Burger <[email-masked]>
Date: Sat, 10 Sep 2011 21:26:02 -0400

So should we move to a one-step process?

---

From: Hector <[email-masked]>
Date: Sat, 10 Sep 2011 22:23:49 -0400

> So should we move to a one-step process?

I asked the great visionary sayer - google, the question:

             What is the IETF Purpose?

and got this top result link:

      http://www.alvestrand.no/ietf/iesg/purpose.php

Alvestrand's conclusion states it as a QA problem which I personally 
agree with:

    A problem in the first 2 bullets is that we have a quality control
    problem - if the WG proposals or the WG consensus documents are not
    high quality, speed will drop like a stone - and SHOULD. We need
    metrics to capture this, somehow.

The search also provided a link to RFC3844,"IETF Problem Resolution 
Process"

Reading RFC3844 now for the first time, seems to have a down a darn 
good job of laying out all the issues in 2002 and thats to be still 
true today, in 2011.

Says in its intro:

    1. Introduction

    ...
    This document begins with an outline of what are currently thought to
    be the purpose and core values of the IETF, and it offers a reminder
    of the good things about the IETF that we don't want to lose in the
    process of solving our problems.

and says pretty profoundly in section 2:

    2. IETF Purpose and Core Values

    As we consider how to address the problems with the IETF processes
    and organizational structure, it is important to keep in mind the
    things about the IETF that we don't want to change -- our sense of
    purpose, and the core values that give the IETF its unique identity.

And it list the items that were concluded in 2002 as non-core values 
that could be considered for change in section 2.1:

   - The division into WGs and Areas
   - The three-step standards process
   - The ASCII format for RFCs and I-Ds
   - The format of IETF meetings
   - The structure of WG mailing lists
   - The powers of the IESG and IAB

and says:

   These things were designed to help us achieve our goals in a way that
   is consistent with our core values.  If they are no longer effective,
   we can and should change them.

I disagree that any of these has shown not the be effective, in 
particular the one on topic with "The Three-step standards process" 
which can only be viewed as a way to bypass the results of these tools 
in keeping core-values.

In section 3, it highlights there ideas that is has put the IETF in a 
bind:

    - Economics,  Commercialization and Convergence.

I personally agree these are the realities, but don't think it should 
alter the essential IETF core values.

--
HLS

---

From: John C Klensin <[email-masked]>
Date: Sat, 10 Sep 2011 22:26:52 -0400

Eric,

Thomas may well have a different answer but, speaking
personally, if we have a choice between a nominal three-step
process that is actually one-step with a few exceptions and a
nominal two-step process that is actually one-step with a few
exceptions, I think we would be much better off with a one-step
process.   Ideally, we should be able to annotate that one-step
process with how mature we think a spec is, but the "facing
reality" situation is that, unless we can change how we and the
marketplace do things, we have a one-step process today and
trying to cut things from unused-three to unused-two
accomplishes nothing other than giving us an extra opportunity
to confirm our failure to be able to use a multi-step process.

I'd find a change to one-step a lot easier to support than a
change to two-step, if only because moving to one-step is not
only closer to present reality but also would give us a starting
part for new work to express maturity (if we still care).
Two-step neither gets us to present reality nor gets us away
from the idea that the multistep model actually expresses
maturity and other useful information.

    john

--On Saturday, September 10, 2011 21:26 -0400 Eric Burger
<[email-masked]> wrote:

>> Advancing a spec is done for marketing, political, process
>> and other reasons. E.g., to give a spec more legitimacy. Or
>> to more clear replace an older one. Nothing wrong with that.

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 11 Sep 2011 14:49:49 +1200

> So should we move to a one-step process?

There is a detailed proposal for that at
http://tools.ietf.org/id/draft-loughney-newtrk-one-size-fits-all-01.txt

I don't think the arguments have changed much since 2006.

   Brian

---

From: John C Klensin <[email-masked]>
Date: Sat, 10 Sep 2011 23:01:23 -0400

> I don't think the arguments have changed much since 2006.

Well, yes and no.

To some extent, that proposal evolved into
draft-ietf-newtrk-repurposing-isd which, without John Loughney's
direct involvement, evolved into draft-klensin-isdbis.  Both of
those can be construed as "one level and explanation" proposals,
rather than "multiple levels and trying to make the situation
fit the categories" ones.

   john

---

From: hector <[email-masked]>
Date: Sat, 10 Sep 2011 23:02:57 -0400

Amazing, this all seems to be a rehash of the RFC3844 WG (2002 - 2004) 
debates and discussions:

    http://www.alvestrand.no/pipermail/problem-statement

The June 2003 quarter seems to be an interesting period of discussions 
touching based with many of the central issues.

--

---

From: Mykyta Yevstifeyev <[email-masked]>
Date: Sun, 11 Sep 2011 07:03:49 +0300

> http://tools.ietf.org/id/draft-loughney-newtrk-one-size-fits-all-01.txt

This draft is what actually matches reality.  I'd better consider 
publishing it rather than draft-housley-two-maturity-levels, unless the 
latter or other similar proposal can demonstrate that it will find the 
way to affect the understanding of people whom RFCs are aimed to (which 
is "a Standards Track RFC is a Standard; maturity level doesn't 
matter"), predominantly implementors, and the real-life reality (which 
is one-step process) effectively (if the don't want to fit themselves to 
this understanding and this reality).

Mykyta Yevstifeyev

---

From: Jari Arkko <[email-masked]>
Date: Sun, 11 Sep 2011 14:50:35 +0300

Thomas,

I am in full agreement that document revision and bug fixing is the more important activity for IETF. Not just in my opinion, but I think we can also see it from the numbers of bis documents versus numbers of advancing documents.

But I think some amount of bug fixing and revision is compatible with advancing a document. Clarifications. Removing cruft that no one ever implemented. Documenting security or other concerns better. I don't think it is quite as black and white as you wrote ("you cannot revise a spec in a real meaningful way"). Its just a question of how big surgery the specification needs.

That being said, I do agree with you that the reasons behind lack of work are complex and often have little to do with IETF processes. It is indeed often the case that people with real capability to do something about a specification are either not interested in too minor changes or have vested interests in not causing their implementations to become non-compliant due to bigger changes. There is little that we can do in the IETF process about this, except maybe make the overall RFC publication easier. The industry will come to the IETF when they have a common incentive for a standard or an update thereof *and* they believe they can actually get the work done.

But bringing ourselves back to the topic of process changes, I agree that Russ' draft is not solving our biggest problems. (I personally view it as a tiny effort to remove an unused feature from our process. It could have removed two levels instead of one, but would that have made it easier or harder to find consensus for the change?) But more importantly, I know that you have driven several process and administration related efforts. Do you have a suggestion on what kind of useful thing we could do to address some of the bigger issues? Or better, a draft in your desk drawer that I could take forward?

Jari

---

From: Russ Housley <[email-masked]>
Date: Sun, 11 Sep 2011 11:57:13 -0400

I think you will see that this question was discussed at least once.  We asked about moving to a one-level maturity model instead.  The conclusion was that it was possible to go from a two-level to a one-level in the future if that is appropriate.  However, if we go straight to a one-level now, and then learn that a two-level would have been better, we would be stuck.

Russ

On Sep 11, 2011, at 7:50 AM, Jari Arkko wrote:

> I personally view it as a tiny effort to remove an unused feature from our process. It could have removed two levels instead of one, but would that have made it easier or harder to find consensus for the change?

---

From: John C Klensin <[email-masked]>
Date: Sun, 11 Sep 2011 12:35:47 -0400

--On Sunday, September 11, 2011 11:57 -0400 Russ Housley
<[email-masked]> wrote:

> I think you will see that this question was discussed at least
> once.  We asked about moving to a one-level maturity model
> instead.  The conclusion was that it was possible to go from a
> two-level to a one-level in the future if that is appropriate.
> However, if we go straight to a one-level now, and then learn
> that a two-level would have been better, we would be stuck.

But, if we go from a three-level to a two-level now, without
compelling evidence that it would make an improvement, and then
learn that a three-level would have been better, we would also
be stuck.   I'm not sure I see the difference between the two
cases.

But I also don't see any advantage in prolonging the discussion.
If I correctly understand Jari's note, the IESG has decided that
there is adequate consensus for this move.  Either people will
appeal that decision after it is formally announced or they
won't.  I would hope that, even if there is an appeal, it would
not reopen the discussions we have been having over and over
again.  If no one does, or if any appeal that is ultimately
filed is ultimately rejected, my hope is that we can all pull
together to try to make this work.

     john

---

From: Hector <[email-masked]>
Date: Sun, 11 Sep 2011 18:01:04 -0400

> together to try to make this work.

Do you have list of documents that might immediately benefit or would 
be top candidates to be moved to IS?  Top ten?

FWIW, I was wondering which BIS documents with no RFC publication 
dates would be candidates.  93 total.

---------------------------------------------------------------------------------------------
DRAFT                                              INITDATE    DAYS 
REVS CSTATE
---------------------------------------------------------------------------------------------
draft-ietf-appsawg-rfc3536bis                      2011-05-06  128   4 
    RFC Ed Queue
draft-ietf-yam-rfc4409bis                          2011-05-06  128   0 
    Publication Requested
draft-ietf-appsawg-rfc3536bis                      2011-05-02  132   0 
    RFC Ed Queue
draft-iesg-rfc1150bis                              2011-04-22  142   0 
    RFC Ed Queue
draft-iesg-rfc1150bis                              2011-04-22  142   0 
    RFC Ed Queue
draft-faibish-nfsv4-pnfs-block-disk-protection     2011-03-07  188   0 
    ID Exists
draft-hoffman-rfc3536bis                           2011-03-07  188   0 
    ID Exists
draft-hoeneisen-rfc5333bis                         2011-03-06  189   0 
    ID Exists
draft-ietf-6man-flow-3697bis                       2011-01-30  224   0 
    IESG Evaluation - AD Followup
draft-ietf-6man-flow-3697bis                       2011-01-30  224   3 
    IESG Evaluation - AD Followup
draft-crocker-dkim-rfc4871bis-doseta               2011-01-13  241   0 
    ID Exists
draft-yevstifeyev-httpbis-http-warning-registry    2011-01-11  243   0 
    AD is watching - External Party
draft-gont-tcpm-rfc1948bis                         2011-01-03  251   0 
    ID Exists
draft-gellens-mime-bucket-bis                      2010-12-23  262   5 
    IESG Evaluation - AD Followup
draft-ietf-httpbis-authscheme-registrations        2010-11-09  306   0 
    AD is watching
draft-ietf-eai-5738bis                             2010-11-09  306   0 
    AD is watching
draft-ietf-ccamp-asymm-bw-bidir-lsps-bis           2010-10-18  328   1 
    IESG Evaluation
draft-ietf-ccamp-asymm-bw-bidir-lsps-bis           2010-10-18  328   0 
    IESG Evaluation
draft-henderson-tcpm-rfc3782-bis                   2010-10-18  328   0 
    ID Exists
draft-faltstrom-5892bis                            2010-10-18  328   1 
    RFC Ed Queue
draft-ah-dnsext-rfc1995bis-ixfr                    2010-10-18  328   0 
    ID Exists
draft-blanton-tcpm-3517bis                         2010-10-18  328   0 
    ID Exists
draft-ietf-eai-rfc5337bis-dsn                      2010-10-18  328   0 
    AD is watching
draft-cui-ippm-rfc5136bis                          2010-10-16  330   0 
    ID Exists
draft-ietf-iri-4395bis-irireg                      2010-09-30  346   0 
    AD is watching
draft-ietf-eai-rfc5721bis                          2010-09-27  349   0 
    AD is watching
draft-ietf-dkim-rfc4871bis                         2010-08-16  391   3 
    RFC Ed Queue
draft-ietf-eai-rfc5335bis                          2010-07-05  433   0 
    AD is watching - AD Followup
draft-cooper-pkix-rfc2560bis                       2010-06-28  440   0 
    ID Exists
draft-ietf-ccamp-rfc5787bis                        2010-06-25  443   0 
    ID Exists
draft-ietf-eai-frmwrk-4952bis                      2010-06-25  443   4 
    RFC Ed Queue
draft-barnes-sipcore-rfc4244bis-callflows          2010-06-24  444   0 
    ID Exists
draft-henderson-hip-rfc5206-bis                    2010-01-03  616   0 
    ID Exists
draft-ietf-yam-5321bis-smtp-pre-evaluation         2009-11-11  669   2 
    DNP-waiting for AD note
draft-law-rfc4869bis                               2009-11-10  670   0 
    Waiting for AD Go-Ahead
draft-huang-behave-rfc3338bis                      2009-10-19  692   0 
    ID Exists
draft-huang-behave-rfc2767bis                      2009-10-19  692   0 
    ID Exists
draft-bishnoi-mpls-ldp-node-group                  2009-10-19  692   0 
    ID Exists
draft-faibish-nfsv4-pnfs-access-permissions-check  2009-10-18  693   0 
    ID Exists
draft-ietf-dnsext-rfc3597-bis                      2009-09-23  718   1 
    IESG Evaluation
draft-ietf-eai-rfc5336bis                          2009-07-06  797   0 
    AD is watching - AD Followup
draft-bishnoi-mpls-mldp-opaque-types               2009-07-06  797   0 
    ID Exists
draft-haddad-mext-mobisoc                          2009-07-02  801   0 
    ID Exists
draft-ietf-geopriv-rfc3825bis                      2009-06-08  825   8 
    RFC Published
draft-barnes-sip-rfc4244bis                        2009-03-04  921   0 
    ID Exists
draft-galvin-rfc3777bis                            2009-03-04  921   0 
    ID Exists
draft-chen-rfc4893bis                              2009-01-24  960   0 
    ID Exists
draft-ietf-payload-rfc3016bis                      2008-09-19  1087  1 
    IESG Evaluation - Revised ID Needed
draft-ellermann-idnabis-test-tlds                  2008-06-05  1193  0 
    ID Exists
draft-ietf-tls-rfc4347-bis                         2008-06-01  1197  2 
    RFC Ed Queue
draft-ietf-6man-node-req-bis                       2008-02-11  1308  0 
    IESG Evaluation - Revised ID Needed
draft-ietf-6man-node-req-bis                       2008-02-11  1308  1 
    IESG Evaluation - Revised ID Needed
draft-hoffman-tao4677bis                           2008-01-17  1333  0 
    ID Exists
draft-ietf-iri-3987bis                             2007-07-02  1532  0 
    AD is watching
draft-ietf-avt-rfc3189bis                          2007-05-11  1584  0 
    ID Exists
draft-fanf-smtp-rfc1845bis                         2007-01-25  1690  0 
    ID Exists
draft-ietf-dnsext-rfc2672bis-dname                 2006-09-27  1810  1 
    Waiting for AD Go-Ahead
draft-haddad-mipshop-mobisig-del                   2006-06-20  1909  0 
    ID Exists
draft-andreasen-sip-rfc3603bis                     2006-06-19  1910  0 
    ID Exists
draft-arkko-mip6-3775bis-ndmobext                  2006-03-01  2020  0 
    ID Exists
draft-ietf-avt-mib-rtp-bis                         2006-02-28  2021  0 
    ID Exists
draft-drage-sipping-rfc3455bis                     2005-10-20  2152  0 
    ID Exists
draft-eastlake-2606bis                             2005-10-18  2154  0 
    ID Exists
draft-ietf-l3vpn-2547bis-mcast-bgp                 2005-10-17  2155 
92   RFC Ed Queue
draft-cstoner-rfc2445bis                           2005-07-14  2250  0 
    ID Exists
draft-ietf-l3vpn-2547bis-mcast                     2005-06-01  2293  2 
    RFC Ed Queue
draft-daniel-natpt-bis                             2005-01-18  2427  0 
    ID Exists
draft-hares-idr-rfc2858bis-survey                  2004-07-01  2628  0 
    ID Exists
draft-chen-bgp-rfc2796bis-survey                   2004-05-01  2689  0 
    ID Exists
draft-hain-1918bis                                 2004-04-01  2719  0 
    ID Exists
draft-arkko-roamops-rfc2486bis                     2004-02-09  2771  0 
    ID Exists
draft-hoffman-rfc1738bis                           2003-09-01  2932  0 
    ID Exists
draft-hellstrom-avt-rfc2793bis                     2003-08-01  2963  0 
    ID Exists
draft-hoffkohn-rfc1738bis                          2003-06-19  3006  0 
    ID Exists
draft-allen-newsml-urn-rfc3085bis                  2003-05-01  3055  0 
    ID Exists
draft-howard-rfc2307bis                            2002-10-30  3238  0 
    ID Exists
draft-galvin-rfc2727bis                            2002-01-24  3517  0 
    ID Exists
draft-fishman-2436bis                              2001-12-01  3571  1 
    RFC Published
draft-guttman-svrloc-rfc2608bis                    2001-11-01  3601  0 
    ID Exists
draft-guttman-svrloc-rfc2610bis                    2001-08-01  3693  0 
    ID Exists
draft-moore-rfc1891bis                             2001-06-21  3734  0 
    RFC Published
draft-vaudreuil-1892bis                            2001-06-15  3740  0 
    RFC Published
draft-ietf-smime-rfc2630bis                        2001-04-01  3815  1 
    RFC Published
draft-ietf-snmpv3-usm-v2-rfc2574bis                2001-02-01  3874  0 
    RFC Published
draft-dally-ldapbis-rfc2256                        2000-11-16  3951  0 
    ID Exists
draft-hodges-ldapbis-ldapv3-ts                     2000-10-01  3997  0 
    ID Exists
draft-altman-rfc2941bis                            2000-10-01  3997  0 
    ID Exists
draft-altman-rfc2942bis                            2000-10-01  3997  0 
    ID Exists
draft-altman-rfc2944bis                            2000-10-01  3997  0 
    ID Exists
draft-vaudreuil-1893bis                            2000-06-15  4105  0 
    RFC Published
draft-guttman-svrloc-slpv2bis                      2000-05-19  4132  0 
    ID Exists
draft-biswas-pim-sm-qos                            1999-06-25  4461  0 
    ID Exists
draft-binkrich-mobisec                             1998-11-01  4697  0 
    ID Exists
---------------------------------------------------------------------------------------------

and these were given DEAD status:

draft-ietf-eai-5378bis                             2010-11-09  306   0 
    Dead
draft-ietf-yam-rfc1652bis-pre-evaluation           2009-08-17  755   0 
    Dead
draft-ietf-idnabis-mappings                        2009-05-25  839   0 
    Dead
draft-klensin-idnabis-protocol                     2007-11-02  1409  0 
    Dead
draft-borman-1323bis                               2007-07-10  1524  0 
    Dead
draft-duerst-iri-bis                               2007-07-02  1532  0 
    Dead
draft-ietf-dime-rfc3588bis                         2006-12-12  1734  0 
    Dead
draft-faltstrom-idnabis-tables                     2006-10-19  1788  0 
    Dead
draft-klensin-idnabis-issues                       2006-10-19  1788  0 
    Dead
draft-altman-telnet-rfc2941bis                     2006-04-17  1973  0 
    Dead
draft-altman-telnet-rfc2944bis                     2006-04-17  1973  0 
    Dead
draft-altman-telnet-rfc2942bis                     2006-04-17  1973  0 
    Dead
draft-adamson-rfc2847-bis                          2006-03-31  1990  0 
    Dead
draft-narten-ipv6-3177bis-48boundary               2005-07-12  2252  0 
    Dead
draft-clark-avt-rtcpxr-bis                         2005-07-01  2263  0 
    Dead
draft-even-avt-rfc3047-bis                         2005-06-14  2280  0 
    Dead
draft-costello-rfc3492bis                          2004-04-01  2719  0 
    Dead
draft-hoffman-rfc3490bis                           2003-10-06  2897  0 
    Dead
draft-hoffman-rfc3491bis                           2003-10-06  2897  0 
    Dead
draft-hoffman-rfc3454bis                           2003-10-06  2897  0 
    Dead
draft-kerr-avt-vorbis-rtp                          2003-01-08  3168  0 
    Dead
draft-antti-rfc2806bis                             2002-06-01  3389  0 
    Dead
draft-rfc-editor-rfc2223bis                        2002-02-22  3488  0 
    Dead
draft-ietf-mobileip-rfc3012bis                     2001-11-23  3579  0 
    Dead
draft-ietf-pppext-rfc2284bis                       2001-10-06  3627  0 
    Dead
draft-ietf-ppvpn-rfc2917bis                        2001-07-01  3724  0 
    Dead
draft-zeilenga-ldapv3bis-opattrs                   2000-06-29  4091  0 
    Dead
draft-vaudreuil-1983bis                            2000-04-25  4156  0 
    Dead

---

From: John C Klensin <[email-masked]>
Date: Sun, 11 Sep 2011 21:24:43 -0400

--On Sunday, September 11, 2011 18:01 -0400 Hector
<[email-masked]> wrote:

>> But I also don't see any advantage in prolonging the
>> discussion. If I correctly understand Jari's note, the IESG
>> has decided that there is adequate consensus for this move.
>> Either people will appeal that decision after it is formally
>> announced or they won't.  I would hope that, even if there is
>> an appeal, it would not reopen the discussions we have been
>> having over and over again.  If no one does, or if any appeal
>> that is ultimately filed is ultimately rejected, my hope is
>> that we can all pull together to try to make this work.
> 
> Do you have list of documents that might immediately benefit
> or would be top candidates to be moved to IS?  Top ten?
> 
> FWIW, I was wondering which BIS documents with no RFC
> publication dates would be candidates.  93 total.

Based on the first few, there are a bunch of errors in your
list.  I'd think the likely candidates of this type would be
documents that have been approved, or are near approval, for
Draft Standard.  The more interesting case would be documents
like 5321.  There is no question about either interoperability
or wide deployment and use, but, given outstanding errata, the
list of proposed changes in the YAM preevaluation document, and
other issues that have come up, I doubt that there would be
consensus for moving the existing document to full standard.  On
the other hand, this change does not increase the motivation to
do more work on it and may decrease it, so it is, as I said, and
interesting case.

A few comments about part of your list below...

> --------------------------------------------------------------
> DRAFT                                              INITDATE
> DAYS REVS CSTATE
> draft-ietf-appsawg-rfc3536bis                      2011-05-06
> 128   4     RFC Ed Queue
    Already published as RFC 6365, a BCP and hence irrelevant to
the "maturity level" discussion.

> draft-ietf-yam-rfc4409bis                          2011-05-06
> 128   0     Publication Requested
    In queue for publication as full/Internet standard.
Therefore presumably unaffected by this change.

> draft-ietf-appsawg-rfc3536bis                      2011-05-02
> 132   0     RFC Ed Queue
     See above.  You appear to have counted this document at
least three times.

> draft-iesg-rfc1150bis                              2011-04-22
> 142   0     RFC Ed Queue
> draft-iesg-rfc1150bis                              2011-04-22
> 142   0     RFC Ed Queue
      Published as RFC 6360, an informational document and hence
unaffected by this change.  Note: probably counted twice.

> draft-faibish-nfsv4-pnfs-block-disk-protection     2011-03-07
> 188   0     ID Exists
       I have no personal information on this document.

> draft-hoffman-rfc3536bis                           2011-03-07
> 188   0     ID Exists
       This document was replaced by
draft-ietf-appsawg-rfc3536bis and then by RFC 6365, for which
see above.  

And so on.  The number using these criteria would appear to be
somewhat smaller than you believe.

    john

---

From: Hector <[email-masked]>
Date: Sun, 11 Sep 2011 23:03:37 -0400

John C Klensin wrote:
> list.  

Yes, just imported into SQL database what Jari provided for the raw 
data and for the output shown below, the SQL query was:

SELECT
   draft,
   initdate,
   datediff(Now(),initdate) as days,
   revs,
   cstate
FROM idpub
   WHERE rfcdate = "0000-00-00" AND draft like "%bis%"
   ORDER by days;

Sure, there seems to have some book keeping aging in the data. i.e. 
duplicate records or new ones not replacing others (hence the dupes 
you saw), verifying some of the drafts with the actual, some had real 
revisions but the revision count field was 0.

I just wanted to give a rough output of those labeled as bis but did 
not have a RFC publishing date, sorted by the number of days from 
start to today.

> I'd think the likely candidates of this type would be

I would assume those documents that have been in the IESG hands will 
get flushed out quickly.

For example, lets used DKIM, a query of:

select draft,
       initdate,
       iesgdays as iesg,
       datediff(Now(),initdate) as days,
       rfcdate,
       revs, cstate
      from idpub
      where draft like "%dkim%"
      order by days;

Produces:

-------------------------------------------------------------------------------------------
DRAFT                                 INITDATE    IESG  DAYS   RFCDATE 
     REVS CSTATE
-------------------------------------------------------------------------------------------
draft-crocker-dkim-doseta             2011-01-13  114   241 
0000-00-00  0    ID Exists
draft-crocker-dkim-rfc4871bis-doseta  2011-01-13  114   241 
0000-00-00  0    ID Exists
draft-ietf-dkim-rfc4871bis            2010-08-16  304   391 
0000-00-00  3    RFC Ed Queue
draft-ietf-dkim-mailinglists          2010-05-08  344   491 
0000-00-00  2    RFC Ed Queue
draft-fenton-dkim-reputation-hint     2009-02-12  180   941 
0000-00-00  0    ID Exists
draft-ietf-dkim-rfc4871-errata        2009-01-26  87    958 
2009-08-27  3    RFC Published
draft-hallambaker-dkim-extensions     2008-07-02  196   1166 
0000-00-00  0    ID Exists
draft-ietf-marf-dkim-reporting        2007-12-03  468   1378 
0000-00-00  0    AD is watching
draft-ietf-dkim-deployment            2007-11-11  716   1400 
2010-06-01  2    RFC Published
draft-ietf-dkim-ssp-requirements      2006-08-10  317   1858 
2007-10-17  1    RFC Published
draft-hallambaker-dkimpolicy          2006-06-22  180   1907 
0000-00-00  0    ID Exists
draft-ietf-dkim-overview              2006-06-22  857   1907 
2009-07-08  2    RFC Published
draft-fenton-dkim-threats             2005-09-28  2136  2174 
0000-00-00  0    Dead
draft-ietf-dkim-threats               2005-09-28  102   2174 
2006-10-04  1    RFC Published
draft-dkim-pkix                       2005-09-09  185   2193 
0000-00-00  0    ID Exists
draft-ietf-dkim-base                  2005-07-12  225   2252 
2007-05-28  5    RFC Published
draft-ietf-dkim-ssp                   2005-07-12  452   2252 
2009-08-12  4    RFC Published
-------------------------------------------------------------------------------------------

With my familiarity with DKIM, the above seems to reflect pretty close 
to the history. I venture among those
yet to be published but in the hands of the IESG, seem to be good 
candidates.

draft-ietf-dkim-rfc4871bis            I expect this one for sure
draft-ietf-dkim-mailinglists          Maybe
draft-crocker-dkim-doseta             don't know enough about it but 
its Crocker :)
draft-crocker-dkim-rfc4871bis-doseta  don't know enough about it but 
its Crocker :)
draft-fenton-dkim-reputation-hint     don't know enough about it
draft-hallambaker-dkim-extensions     I don't recall this one
draft-ietf-marf-dkim-reporting        Seems like a moving target, so 
don't know.

and these probably not.

draft-hallambaker-dkimpolicy          POLICY is a deadly sin for DKIM!
draft-dkim-pkix                       don't know enough about it.

All pure SWAGGING on my part.

---

From: Sam Hartman <[email-masked]>
Date: Mon, 12 Sep 2011 07:32:48 -0400

>>>>> "Keith" == Keith Moore <[email-masked]> writes:

    Keith>     2) This will not do any good

    Keith> IMO, that is a valid objection.  Stability in our process is
    Keith> desirable; therefore change merely for the sake of change is
    Keith> undesirable.

"This will not do any good, stability is important, so this should not
be done," is an objection.  "This will not do any good," is neutral.
You believe that stability is important.  Others believe that forward
progress and being seen to do something are good.  I do tend to come
down on your side, and if I think something isn't going to do do good
I'm likely to actually state an objection. However for a lot of reasons,
I think the IESG should actually require people to present something
that is constructionally supportive or an objection before counting it
as such. "This will not do any good," is not such.

---

From: Keith Moore <[email-masked]>
Date: Mon, 12 Sep 2011 07:42:05 -0400

> as such. "This will not do any good," is not such.

I agree that a statement of the form "this will not do any good" is more compelling if it is supported by an argument as to _why_ it won't do any good.   Such a statement by itself should count against consensus, but it shouldn't sway anyone else into changing his opinion.

Keith

---

From: Martin Rex <[email-masked]>
Date: Mon, 12 Sep 2011 23:33:06 +0200 (MEST)

> 1) I support moving to a two level process.

I don't, and there were some other things in the document
(last time I read) that to which I dissent.

> 2) This will not do any good

These are not "objection", but when gauging "consent", then
counting them as "consent" rather than "no consent" or "dissent"
would IMO be inappropriate.

"objections" is stuff that needs to be addressed by issue resolution
prior to any gauging/judging of consent.

-Martin

---

From: Douglas Otis <[email-masked]>
Date: Mon, 12 Sep 2011 15:04:25 -0700

> advance more. It's not just about process.
Agreed. Well said.

-Doug

---

From: Hadriel Kaplan <[email-masked]>
Date: Fri, 16 Sep 2011 05:17:00 +0000

I thought the counting of votes was finished on this topic but people seem to keep emailing their support/lack-of, so naturally I will be a good lemming and do the same.

1) I am in favor of the two-maturity-levels draft and change.  I have consulted a textbook on Euclidean geometry and determined that the distance from level 2 to 1 is shorter than 3 to 1, getting us closer to the actual location most of us are at (which is of course 1 maturity level).  

2) I am strongly opposed to draft-loughney-newtrk-one-size-fits-all-01, because it is far too rational and sane, and would prevent this topic from continuing forever.  Furthermore, I am against any move to 1 maturity level because apparently there are one or two people with so much free time or posterity they actually bother moving PS to higher levels these days, and who are we to squash their hobby/passion/disorder?  (In fact, I was almost going to suggest we go to a 4 or 5 maturity level process just to give these people more harmless things to do, but I digress...)

3) The IESG should be applauded/thanked for recognizing there is only one maturity level (PS), and taking the steps necessary to treat potential RFCs as such from a quality perspective.  But they should be denigrated for not telling us they did that.  So they come out even.

4) Regarding the discussion in this thread about what types of comments should be counted or not: I believe we should produce a new RFC concerning what response phrases in emails are going to be counted or not for consensus counting, so that we may know what to say in the future to get our votes counted.  (Of course the big question everyone wants to know is when will such a new RFC reach the second maturity level?)

-hadriel

p.s. in all seriousness, I'm in favor of this two-maturiy-level draft.  I do not think it is "change for change's sake", but rather a change attempting to accommodate differing viewpoints of our present location and where we want to be.  If it fails to change the status-quo of 1 level, that's *OK*, we can try again - the Internet won't collapse because of this document, and neither will the IETF.

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 17 Sep 2011 08:39:11 +1200

+4 and rotfl

   Brian