Subject: [newtrk] [Fwd: I-D ACTION:draft-carpenter-newtrk-twostep-00.txt]

From: Brian E Carpenter <[email-masked]>
Date: Wed, 01 Jun 2005 10:22:12 +0200

Being in an impatient mood over the weekend, I put this together.
Before you read it, please note that it was constructed
specifically to *not* obstruct the discussion of the ISD proposal.
My view, though, is that something like what I am proposing
here would create a much cleaner context for any form of ISD
solution.

      Brian

-------- Original Message --------
Subject: I-D ACTION:draft-carpenter-newtrk-twostep-00.txt
Date: Tue, 31 May 2005 15:47:51 -0400
From: [email-masked]
Reply-To: [email-masked]
To: [email-masked]

A New Internet-Draft is available from the on-line Internet-Drafts directories.

	Title		: A two stage standards process
	Author(s)	: B. Carpenter
	Filename	: draft-carpenter-newtrk-twostep-00.txt
	Pages		: 8
	Date		: 2005-5-31
	
This document proposes several changes of principle to the Internet
    standards process, especially a reduction from three to two stages in
    the standards track.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-carpenter-newtrk-twostep-00.txt

.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: "Spencer Dawkins" <[email-masked]>
Date: Wed, 1 Jun 2005 06:44:12 -0500

> here would create a much cleaner context for any form of ISD

I don't believe impatience is a bad thing, at this point in the 
process :-) 

.

---

From: Joe Touch <[email-masked]>
Date: Wed, 01 Jun 2005 10:14:12 -0700

> here would create a much cleaner context for any form of ISD

Some feedback:

Aggregating the current steps may be useful, but it might be safer to
group PS+DS (using DS metrics as the hurdle), and leaving IS (STD) where
it is, than by basically truncating the process at DS and referring to
DS as IS.

I don't agree that STDs are unused; they are, IMO, the few protocols at
the foundation of the Internet and responsible for its stability. DS is
a clutter by comparison, and it is useful not to merge these two levels.

Finally, I don't agree that the proposed IS should be able to
normatively cite a PS for non-interoperable features. Those references
should be made to be only informative, i.e., in ways that do not affect
the spec of the PS specification for correctness or completeness.

Joe

.

---

From: Sam Hartman <[email-masked]>
Date: Wed, 01 Jun 2005 14:19:32 -0400

I strongly support the idea of having more open citations between
levels of the standards track.  I think that would be a qualitative
change that would make it easier to advance standards on the standards
track.  I believe that would make it more liely the standards track
would be used.

I suspect no matter what we'll do we'll get it wrong.  It seems that
there are some types of normative down references that will end up
being highly problematic and large classes that will be OK.  I don't
think we will do a good job of figuring out how to describe what down
references are acceptable without trying for a while.  So, my
recommendation would be to allow normative down references to proposed
(but not to internet draft) and then expect to refine things later.

--Sam

.

---

From: Pekka Savola <[email-masked]>
Date: Thu, 2 Jun 2005 08:24:41 +0300 (EEST)

> (but not to internet draft) and then expect to refine things later.

IMHO, it's either that or the process must be lightweight enough (and 
people and the IESG motivated enough) that if you want to move a spec 
from PS to IS (or DS), and you hit a normative reference to PS, you 
can reasonably easily also start the work to escalate that reference 
to IS (or DS).

Today (speaking from ex- WG chair's perspective) getting another WG 
(or no WG at all, anymore) to push its PSs forward to suit your need 
for normative references is painful and time consuming at best.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Sam Hartman <[email-masked]>
Date: Thu, 02 Jun 2005 02:18:32 -0400

>>>>> "Pekka" == Pekka Savola <[email-masked]> writes:

    Pekka> On Wed, 1 Jun 2005, Sam Hartman wrote:
    >> I suspect no matter what we'll do we'll get it wrong.  It seems
    >> that there are some types of normative down references that
    >> will end up being highly problematic and large classes that
    >> will be OK.  I don't think we will do a good job of figuring
    >> out how to describe what down references are acceptable without
    >> trying for a while.  So, my recommendation would be to allow
    >> normative down references to proposed (but not to internet
    >> draft) and then expect to refine things later.

    Pekka> IMHO, it's either that or the process must be lightweight
    Pekka> enough (and people and the IESG motivated enough) that if
    Pekka> you want to move a spec from PS to IS (or DS), and you hit
    Pekka> a normative reference to PS, you can reasonably easily also
    Pekka> start the work to escalate that reference to IS (or DS).

I disagree.  I think that if the process is going to be light enough
weight to use, it needs to not block on normative references to
somewhat unclear specs that everyone manages to implement anyway.  For
example TLS is at proposed; it's going to be there for a while.

As a practical matter though there are a lot of TLS implementations
and they do interoperate.  So I don't think this should block IMAP,
SIP, etc.
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 02 Jun 2005 13:39:20 +0200

Joe,

Joe Touch wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> 
> 
> Brian E Carpenter wrote:
> 

But that would make it much harder to get on the standards track at all.
The criticism the IESG has received is that it's been *tougher*
than required by RFC 2026 for PS, not that it's been lax. So I don't
see how your proposal would help with timeliness,or respond to the
complaints from the community.

> the foundation of the Internet and responsible for its stability.

I didn't mean to suggest the *specifications* are unused. I assert that
the *designation* STD is unused - once it's been assigned to a document,
it is very rare for it to be cited as an STD. (Except of course by
the very small number of people who are actually aware of the formalities
of the standards track; but we don't count - we aren't the end users
of RFCs.)

> DS is

Well, I disagree. Again, this is something nobody outside the small number
of people above perceives. IMHO the DS/STD distinction is meaningless
formality already, so we'd better just scrap it.

> the spec of the PS specification for correctness or completeness.

Disagree, for reasons Sam has stated. Again, meaningless formality.

    Brian

.

---

From: Joe Touch <[email-masked]>
Date: Thu, 02 Jun 2005 08:41:34 -0700

> complaints from the community.

Your proposal doesn't address admission to PS, since it didn't
substantively change the requirements or process to admit to PS.

Granted, my suggestion makes it harder to get into PS (and the zoo there
 IMO needs higher admissions criteria, not lower), but this proposal
does not make it easier.

> of the standards track; but we don't count - we aren't the end users

It is useful to distinguish citation by those maintaining the protocols
and by "the rest of the community". Removing STDs to respond to the
latter hurts the former. IMO, ignorance is better treated with education
than by omitting material from the 'curriculum'.

> formality already, so we'd better just scrap it.

Given DS is a smaller clutter than PS, the difference between how it is
merged (with PS or STD) isn't significant enough to debate, I guess.

> Disagree, for reasons Sam has stated. Again, meaningless formality.

OK, there I'm confused. We're still talking about protocol specs, right?
At some point, the minutae of how they're written is largely meaningless
formality by definition, but it's precisely that formality that makes
them implementable and ensures interoperability.

So long as there is a difference between normative and non-normative
references at all, there needs to be careful consideration of when to
use each.

Normative references to non-normative protocols doesn't make sense; I
don't see how it hurts the doc to force them to be non-normative refs.

And I entirely disagree with Sam's point - this can't be an 'experiment'
that we change later, since we can't undo the results.

Joe

---

From: John Leslie <[email-masked]>
Date: Thu, 2 Jun 2005 14:44:53 -0400

> http://www.ietf.org/internet-drafts/draft-carpenter-newtrk-twostep-00.txt

   To simplify, Brian proposes:

1) to retain "Proposed Standard" as is;
2) to rename "Draft Standard" to "Interoperable Standard";
3) to eliminate the third stage ("Standard");
4) to allow normative references to Proposed Stds in Interoperable Stds
5) a bunch of discussion points and transition issues.

> here would create a much cleaner context for any form of ISD

   I think we need guidance from Scott Bradner on that question.

   My understanding of consensus leaving IETF62 was that those present
wanted an ISD system to document interoperability as it happened (as well
as other details) in a mechanism simpler than issuing RFCs; and that we
would wait for an indication from IESG whether they would support NEWTRK
discussing this further.

   The only reaction I've heard of from IESG is that they felt it sounded
like too much work; and they (pointedly?) did not ask us to discuss the
details.

   Brian now comes up with something more on-charter; and I presume he
believes the IESG _could_ support it (though he is careful to point out
he hasn't asked any of the members to commit to that).

   I find myself confused when Brian says this will "*not* obstruct the
discussion of the ISD proposal". He calls for a single "snapshot" of
progress to interoperability, listing which features of the "Proposed
Standard" are interoperable at that moment in time -- sitting around for
all time as the main reference to the "Standard". I cannot see this as
"a much cleaner context for any form of ISD solution".

   The rest of the stuff seems like a worthwhile improvement; but I'm
very hesitant to discuss it until we clear up the question of how much
of the ISD proposal we will continue to pursue.

--
John Leslie <[email-masked]>
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Sam Hartman <[email-masked]>
Date: Thu, 02 Jun 2005 15:52:05 -0400

>>>>> "John" == John Leslie <[email-masked]> writes:

    >> http://www.ietf.org/internet-drafts/draft-carpenter-newtrk-twostep-00.txt

    John>    To simplify, Brian proposes:

    John> 1) to retain "Proposed Standard" as is; 2) to rename "Draft
    John> Standard" to "Interoperable Standard"; 3) to eliminate the
    John> third stage ("Standard"); 4) to allow normative references
    John> to Proposed Stds in Interoperable Stds 5) a bunch of
    John> discussion points and transition issues.

    John>    The only reaction I've heard of from IESG is that they
    John> felt it sounded like too much work; and they (pointedly?)
    John> did not ask us to discuss the details.

I'd read the consensus statement as:

1) The IESG is not at all sure that the ISD proposal is a good idea.
   We'd think it reasonable if newtrk decided not to follow it any more.

2) If you do decide to spend more efforts on the ISD proposal make
    sure you realize that you may not come to consensus and the IESG
    believes that even if you do come to consensus the IETF as a whole
    may not support that consensus.  In other words you're taking a
    significant risk.

3) If you do decide to go forward with ISDs, the IESG tried to point
   some useful directions that the IESG believes it would be worth
   considering.  The IESG also tried to enumerate some specific
   concerns with the current proposal.

Of course this reading of the consensus statement is an individual
opinion.  The only thing the IESG as a whole has consensus on is the
text that Brian posted.
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 03 Jun 2005 12:08:29 +0200

> 5) a bunch of discussion points and transition issues.

That is a good summary, thanks.

> like too much work;

I think that is an unfair reading. I think it is more work than people
may realise, but in my mind there could certainly be an ISD-like
form of document that would be relatively easy to produce and maintain.
I think we have a tendency to over-formalize, in fact, and something
pretty simple could be pretty effective.

> and they (pointedly?) did not ask us to discuss the
> details.

No, because the message was not about the details.

> he hasn't asked any of the members to commit to that).

I have no idea. I really, really did not discuss it with any other ADs.

> "a much cleaner context for any form of ISD solution".

A simpler standards track and allowing IS->PS downrefs actually makes
the structure of ISDs much simpler, IMHO, and probably they could
be generated almost automatically. I wouldn't be against designing
the system for IS, PS, and errata simultaneously with ISD and with a
*requirement* that ISDs can be generated automatically (except for
manually inserted comments).

> of the ISD proposal we will continue to pursue.

Understood, but divide and conquer has merit too.

    Brian

.

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 03 Jun 2005 12:16:57 +0200

>    We'd think it reasonable if newtrk decided not to follow it any more.

I'm not sure about the second sentence. Personally, I think the notion
of providing the users of our specifications with guidance about what
packages of specifications go together has great potential value.

 >     may not support that consensus.  In other words you're taking a

That is certainly the IESG view. Of course, there's an interesting
meta-point here: the IESG won't survive if it appoints itself as
both judge and jury on process changes. If there is a clear IETF
consensus for a process change (note: I didn't say newtrk consensus),
the change will happen. But we aren't anywhere near there yet.
 >    concerns with the current proposal.

Yes, and I think people in newtrk may have missed that bit.
Maybe we should have spelled out the three bullets as Sam has
now done.

    Brian

.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Fri, 3 Jun 2005 07:00:37 -0500

I am glad this conversation is starting back up, pre-Paris... 
questions below.

Spencer

> pretty simple could be pretty effective.

Just to confirm - my understanding is that the IESG felt like ISDs 
would be {too much work | more work than people realize} FOR THE IESG, 
right?

To pick one AD comment (not at random):

> I would like to understand the answers to the following questions:
>
> - Who is expected to write the ISD text for a document that doesn't 
> fit into an existing one?
>
> - When is that text written and who reviews it?
>
> - Who is expected to maintain that text as the standard changes?
> - Will the contents of an ISD (or changes to the contents) by 
> reviewed by the WG that produces the standards track document?  By 
> the IETF community in a LC process?

[deleted non-question text, down to]

> For instance, who is expected to do all of the work outlined in 
> section 6, "Transition"?  I agree (as it says in the document) that 
> ISDs will be useless without this work, but I think that this 
> document is equally useless without some understanding of how all of 
> that work will get done.
>
> I wonder if we are seriously underestimating how much work it will 
> take to produce ISDs and how long it will take before we have enough 
> of them that they are a useful source of information for 
> implementers.

Do we (newtrk) know the answers to these questions?

Do the answers contain the words "somebody besides the IESG"?

Have we written them down?

Spencer 

.

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 06 Jun 2005 15:25:55 +0200

Spencer Dawkins wrote:
> I am glad this conversation is starting back up, pre-Paris... questions 
> below.
> 
> Spencer
> 
> Just to confirm - my understanding is that the IESG felt like ISDs would 
> be {too much work | more work than people realize} FOR THE IESG, right?

Not sure. If a clear and simply structured ISD was put on the IESG's
virtual desk, I'm not sure it would add much workload for the IESG.
But an unclear ISD would of course be harder to deal with.

>> - Will the contents of an ISD (or changes to the contents) by reviewed 
>> by the WG that produces the standards track document?  By the IETF 
>> community in a LC process?
> 
> 
>> ISDs will be useless without this work, but I think that this document 
>> is equally useless without some understanding of how all of that work 
>> will get done.
>>
>> of them that they are a useful source of information for implementers.
> 
> 

Indeed.

    Brian

.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 9 Jun 2005 14:57:23 -0700

> >  5) a bunch of discussion points and transition issue

There are two questions that need to be answered:

1.  What significant problems do these changes solve?

2.  What are the incentives for using the changed conditions, as opposed 
to the current ones?

Unfortunately, I believe that the answer for each is: none.

Draft is not getting used.  Changing the label does nothing to change 
that.

Unless and until we change incentives and/or change definitions, the 
community will continue to ignore anything after proposed.

  d/
  ---
  Dave Crocker
  Brandenburg InternetWorking
  +1.408.246.8253
  dcrocker  a t ...
  WE'VE MOVED to:  www.bbiw.net

.

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 13 Jun 2005 10:20:16 +0200

> 1.  What significant problems do these changes solve?

-- clarifies that moving up from PS is explicitly tied
to interoperability
-- removes the sense of struggling towards an unattainable
goal (yet another bureaucratic step to finally reach STD status)

In other words, directly tackles problem 2.4 in RFC 3774

> 2.  What are the incentives for using the changed conditions, as opposed 

-- clearer and more attainable goals
-- moving to Interoperable Standard specifically may *not* require a
    new RFC even if some features are listed as not shown to interoperate

(Remember that my starting position was to move to a one level
standards track. I detected resistance to that, but I'd go back to
it in a second if people wanted it.)

    Brian

.

---

From: Bruce Lilly <[email-masked]>
Date: Mon, 13 Jun 2005 08:50:42 -0400

> -- clarifies that moving up from PS is explicitly tied

No:
1. it appears to be clear from BCP 9 (RFC 2026) that there is already
   such a condition (however, see my June 8 list message).
2. a list of partially- or non-interoperating features (see below) is
   not the same as actual interoperability.  The relevant "problem"
   is that some people don't believe that mere markings on virtual
   paper -- moreover with no regard for the factual accuracy of what
   those markings represent -- constitute interoperability, and that
   "problem" is "solved" by explicitly *removing* ties to actual
   interoperability. 

> goal (yet another bureaucratic step to finally reach STD status)

"unattainable" is too strong; as there are full standards, achieving
that status clearly is attainable.
 
> In other words, directly tackles problem 2.4 in RFC 3774

2.4 says "Three Stage Standards Hierarchy not properly Utilized".
Throwing out the three-stage process doesn't address lack of proper
utilization of the process anymore than throwing out the baby addresses
the problem of disposal of waste bath water.

The second bullet point of 2.4 asserts that there is a belief that the
quality bar for PS has been raised; leaving aside the minor matter of
veracity of that belief and/or assertion, as has already been noted here
the proposal does not appear to address PS -- indeed the draft explicitly
states that there is no change to PS.

The third bullet point addresses lack of enforcement of implementation
or deployment experience, and further weakening the "requirement" for
interoperable implementations doesn't address the lack of enforcement,
it codifies it -- the problem is exacerbated.

The penultimate bullet point discusses failure to carry out periodic
reviews, and that appears to be "addressed" by the draft proposal in
the same manner as the standards track process itself (throwing out
the baby by eliminating the requirement).  As the IESG has been tasked
with carrying out such reviews (BCP 9), it is difficult to see this
as anything other than "appoints itself as both judge and jury on
process changes".

The last bullet point discusses lack of maintenance, and the first
bullet point of the section notes that specifications rarely progress;
the practice of closing down WGs noted in the last bullet point has
been recognized as a major contributing factor in the lack of
progression, as discussed here and on the IETF discussion list. The
draft proposal doesn't address that issue.
 
> -- clearer and more attainable goals

i.e. "lowering the quality bar"

>     new RFC even if some features are listed as not shown to interoperate

Apparently already the de-facto situation.  Usually there is a new RFC
because of correction of errata, boilerplate changes, etc., but non-
interoperating features has not prevented progression on the current
Standards Track, at least recently.

Simply listing non-interoperating features does not constitute actual
interoperability (see above and my June 8 list message).

> standards track.

Effectively, we already have that via BCP RFCS, which (per BCP 9) are
supposed to meet the requirements for a full Standard.  If -- as sometimes
claimed -- the quality bar has been raised to require interoperating
implementations at Standards Track entry, and if there is some
hypothetical specification which meets the requirements for full Standard
when the specification is prepared, there is a mechanism to give the
specification full Standard status in one step via BCP.

There are more than 2 questions of significance.  Obviously, one could
ask:
3. What significant problems would the proposed changes introduce?
4. Do we really care "to make the Internet work better", i.e.
   interoperabilty, or is the Mission Statement a hollow one?
   Indeed, that question is fundamental; if we do care about quality --
   making the Internet work better in reality -- then anything that
   lowers the quality bar is counterproductive.

.

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 14 Jun 2005 16:29:18 +0200

Bruce,

>    such a condition (however, see my June 8 list message).

Yes, for people who know 2026 as well as we do. But for our users,
the people who read RFCs rather than write them, the name says
it all.

>    "problem" is "solved" by explicitly *removing* ties to actual

Nothing we write is the same as actual interoperability. The proposal
doesn't remove the ties between what we write and interoperability;
it just moves them around a bit.

> that status clearly is attainable.

So rarely that nobody cares, IMHO.
> the problem of disposal of waste bath water.

Er, your metaphor is a bit complicated for me, but in my book abolishing
a process gets rid of its problems; the question is whether it also
sacrifices useful goals.

> states that there is no change to PS.

Indeed, because fixing that sub-problem is an operational question.

> it codifies it -- the problem is exacerbated.

But my proposal doesn't weaken (or strengthen) that requirement.
It just moves it around a bit.

> as anything other than "appoints itself as both judge and jury on

This isn't the IESG, it's newtrk, and newtrk is here to propose
changes. But, with my limited experience of the IESG workload, I
want to state as my opinion that the probability of any IESG operating
under current conditions actually implementing such a periodic
review is zero. What use is a provision that, as a matter of running
code, is certain not to be executed?
> draft proposal doesn't address that issue.

It's somewhat decoupled, but I think the point is valid. On the one hand
we don't want self-perpetuating WGs, but on the other hand if we close
them down ASAP we lose any chance of a reasonably prompt interoperability
review. I agree something needs to change in this area.

> i.e. "lowering the quality bar"

No, not in reality. Abolishing the bar that almost nobody tries to jump over
has no particular effect at the entry level, and most documents stay at the
entry level.

> Standards Track, at least recently.

Hmm. Examples?
> interoperability (see above and my June 8 list message).

No,of course not - it would however avoid the need to republish an RFC
if only minor features had not be demonstrated.

> specification full Standard status in one step via BCP.

Well, BCPs are not meant for that as I read 2026.

> 3. What significant problems would the proposed changes introduce?

I haven't spotted any, but maybe I have rose-tinted glasses.

>    lowers the quality bar is counterproductive.

I think that's arguable. You may make it work better by shipping something
in a big hurry despite imperfections. But in any case, I don't agree
that reducing the number of standards levels need have any adverse effect
on the quality of our specs.

    Brian

.

---

From: Bruce Lilly <[email-masked]>
Date: Wed, 15 Jun 2005 11:49:20 -0400

[trimming to address the most important issues]

> it just moves them around a bit.

I believe that the change from BCP 9 (a.k.a RFC 2026) "'interoperable'
means to be functionally equivalent or interchangeable components of
the system or process in which they are used" and "The requirement
for at least two independent and interoperable implementations applies
to all of the options and features of the specification" to the draft
under discussion "applies to each of the options and features of the
specification considered individually" is much more than "moves them
around a bit"; it changes the character of the requirement from assessing
interoperability in terms of functional components to something which
is considerably less meaningful w.r.t. interoperability as currently
defined in BCP 9.

> So rarely that nobody cares, IMHO.

If nobody cares, why are we here?  Let the sleeping dog lie. [I believe
that "nobody" is too strong an assertion also]

> It just moves it around a bit.

See above.

> code, is certain not to be executed?

Presumably the provision is there because the WG, the IETF (via Last
Call review), and the IESG felt that it was an important step in
managing the collection of specifications that constitute Standards
Track RFCs.  IESG has apparently decided to second-guess that
consensus decision by not carrying out the process specified in BCP 9.
Not merely judge and jury, but (non-)executioner as well.

> review. I agree something needs to change in this area.

This is also related to the periodic reviews, since per BCP 9 the WG
Chair is responsible for documenting interoperability in a report for
advancement to DS.  No WG, no Chair, no report, no advancement.
 
> has no particular effect at the entry level, and most documents stay at the

There are other factors in lack of advancement (see above).  Changing the
requirement for interoperability to something (difficult to concisely
characterize exactly what that is) considerably less related to true
interoperability certainly lowers the quality bar.  Whether or not that's
a de-facto process change vs. a de-jure process change is another matter,
outlined in detail in my June 8 "Interoperability" message, and related
to the "judge and jury" comments.

> > Standards Track, at least recently.

We discussed (in the sense of BCP 9 section 6.5.2 second paragraph) the
case of advancement of the 2476bis draft to DS despite lack of
demonstrated functional component interoperability (See
http://www.ietf.org/IESG/Implementations/rfc-2476-and-draft-gellens-submit-bis-implementation.txt
and the spreadsheet matrix derived from it at
http://users.erols.com/blilly/rfc-2476-gellens-implementation.sxc
).  There is also the advancement of a 2234 successor draft to DS
despite the fact that *both* cited implementations (
http://www.ietf.org/IESG/Implementations/RFC2234-implementation-report.txt
) explicitly fail to
use the 2234 and draft provisions specifying CRLF line endings, one fails
to accept binary, decimal, and hexadecimal literals using %B, %D, %X in
spite of the fact that 2234 and the draft explicitly state case
independence of the 'b', 'd', and 'x' (it could be argued that the
specification is not sufficiently clear, since the case-independence part
is in a separate section), one accepts (illegal) empty rules, one fails
to accept a specified string literal of the form %x0D.0A.0B.  And there
are specification features known to cause confusion which have not been
used, and which could and should be removed as part of the fine-tuning
of the specification.

> if only minor features had not be demonstrated.

I believe that the provision for removal of features not demonstrated as
interoperable removes features which may be used to exploit security
vulnerabilities and enhances interoperability (in the BCP 9 functional
component sense).  I further believe that that is a valuable step in
refining a specification as it progresses from entry level through
fine-tuned to cast in concrete.

> Well, BCPs are not meant for that as I read 2026.

I agree that BCP 9 specifies other uses.  However, in the hypothetical
case of a fully-fledged, truly interoperable specification with wide
deployment experience in mission-critical applications, BCP *could* be
used.  Indeed there is a proposal currently in Last Call for consideration
as BCP which may well be more suitable as a Standards Track specification
precisely because of fine-tuning and deployment issues.
 
> I haven't spotted any, but maybe I have rose-tinted glasses.

I have mentioned the functional component interoperability issues, and
the current provision for removal of unimplemented potential security
loopholes/interoperability sandbars.

> in a big hurry despite imperfections.

Well, I'm not sure that "a big hurry" is possible with current procedures
unrelated to the Standards Track (e.g. IESG review *after* IETF Last Call
rather than *concurrent* review, RFC Editor backlog), and the nature of
the beast (getting e.g. security considerations right takes time and
effort).  And if the "imperfections" impair interoperabilty or cause
havoc, one may well make the Internet work much worse.

> But in any case, I don't agree 

Agreed that all else being equal (particularly the quality-assurance
provisions w.r.t. interoperability and large-scale deployment), the number
of steps need not be related to quality.  However, if weakening or removing
those QA provisions is specified as (part of) the reduction of steps, then
clearly quality may suffer.
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 16 Jun 2005 19:49:49 +0200

> is considerably less meaningful w.r.t. interoperability as currently

Bruce, I think we've had this conversation, but iirc our rules on
what interoperability means were intended very precisely to ensure
that the *specification* was interoperable with itself and not that
implementations were conformant. If someone wants to check and
certify conformance, that's fine but it's not IETF business.

My changes are specifically aimed at the target of checking
that all the features in the spec are interoperable (or not), not
at whether the implementations are complete.

The other change I suggested would allow us to state that a spec
has been proved interoperable except for certain specified features,
in order to avoid the need to update an RFC just to remove
those features. That's a point we can certainly argue, and I'd like
to hear other opinions.

> that "nobody" is too strong an assertion also]

Because I believe that by making the end of the standards track more
attainable, we reduce the cost/benefit ratio of investing effort
after reaching PS. And that will cause more specs to be advanced,
which is presumably an implied goal of this WG.
> Not merely judge and jury, but (non-)executioner as well.

That was certainly the view at the time RFC 2026 was approved.
But we do have running code proof that the IESG doesn't have
cycles. If you want the job done, you have to provide the cycles -
which would imply a new committee, or offloading some other duties
from the IESG.
> advancement to DS.  No WG, no Chair, no report, no advancement.

Indeed. And no maintenance of ISDs, either.

> interoperability certainly lowers the quality bar.

I dispute this, see above.

   Whether or not that's
> to the "judge and jury" comments.

De facto, we run an approximation to RFC 2026 that matches available
human resources.
> used, and which could and should be removed as part of the fine-tuning

Interesting. So if we are operating that way, doesn't it tell us
something about what's realistically possible? It seems to me that
unless we can find a few million to fund an IETF Interoperability
Institute, it's very hard to do better. I really don't see an easy
answer here, other than dealing with what we know to be pragmatically
possible today.
> component sense).

It doesn't follow that such features imply vulnerabilities (they might,
of course). But I really don't see a logical difference between
deleting text and re-issuing the RFC compared with an external
statement that the feature is removed.

   I further believe that that is a valuable step in
> fine-tuned to cast in concrete.

And that is where we differ. I think it's a clerical matter without
deep significance.

> loopholes/interoperability sandbars.

See above

> havoc, one may well make the Internet work much worse.

Indeed. Circumstances vary and in many cases you're quite correct.

> clearly quality may suffer.

Yes, but what we are seeing I think is that BCP 9 sets a standard that
we can't meet in practice, and that tends to make people give up.

    Brian (thinking it's time I put my AD hat on...)

.

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 16 Jun 2005 22:58:24 -0700 (PDT)

On Thu, 16 Jun 2005, Brian E Carpenter wrote:
> has been proved interoperable except for certain specified
> features, in order to avoid the need to update an RFC just to
> remove those features. That's a point we can certainly argue,
> and I'd like to hear other opinions.

I am strongly in favor of that idea.  In fact, I would go even
further:  I would like to see the mechanism for documenting
interoperability be fully decoupled from the actual specification.
Interoperability reports would state explicitly which features are
covered and would remain valid even if the underlying technical
specification was revised to include new features.

Mike Heard

.

---

From: Joe Touch <[email-masked]>
Date: Fri, 17 Jun 2005 08:49:44 -0700

> specification was revised to include new features.

Not all features can be deprecated from specs by a line-item veto.

Determining which ones can be, or which subsets constitute coherent sets
is a nightmare.

Joe
.

---

From: "C. M. Heard" <[email-masked]>
Date: Fri, 17 Jun 2005 09:10:28 -0700 (PDT)

> Not all features can be deprecated from specs by a line-item veto.

Just to make it clear, I do NOT think that it should be a requirement
to deprecate/remove features (minor or major) from a spec as a condition
for advancement of the rest.

> Determining which ones can be, or which subsets constitute coherent sets

Not necessarily.  For example, when a MIB module is developed it is a
requirement that every object or notification defined therein reside
in at least one OBJECT-GROUP or NOTIFICATION-GROUP (respectively).  This
forces the module developer to make exactly such a determination.
 
Mike

.

---

From: Joe Touch <[email-masked]>
Date: Fri, 17 Jun 2005 09:38:03 -0700

> for advancement of the rest.

That's the net effect, isn't it?

Or, let me state it in the positive - the subset that's advanced needs
to be consistent and complete...

> Not necessarily.

Sure - but it's very hard to know when it is and when it is not.

>... For example, when a MIB module is developed it is a

Is it a requirement that no groups have internal interdependencies,
e.g., that one part is computed from another (I'm not a MIB expert, so
pardon if that's a silly question).

Regardless, my point is that allowing the process is dangerous, since
it's not always clear when it can be usefully applied.

Joe
.

---

From: Bruce Lilly <[email-masked]>
Date: Wed, 22 Jun 2005 13:26:28 -0400

[apologies for the delay in responding]

> certify conformance, that's fine but it's not IETF business.

Let's distinguish:
1. interoperability testing for the sake of certifying conformance
2. listing independent interoperable implementations as a way of
   ensuring the clarity and utility of a specification

The first is irrelevant.  The second is what BCP 9 requires.

> at whether the implementations are complete.

However, I believe that it fails to do so.  Consider a hypothetical
specification which (being an optimist) I hope would never pass IESG
muster:

Provision 1: Implementations MUST do X.
Provision 2: Implementations MUST NOT do X.

(for any value of X).  Yes, the specification is self-contradictory.
(A real specification is likely to be more complex, and self-contradiction
may be much more difficult to spot than in this simplified hypothetical
case). Nevertheless, it may be the case that one can find two independent
implementations that meet provision 1, and another two independent
implementations that meet provision 2.  You say that that demonstrates
that all of the features in the specification are interoperable, and
that therefore the specification is suitable for advancement as written.
I believe that it does not adequately demonstrate interoperability, and
that *fully-conforming*, *functionally-equivalent* implementations, as
specified in BCP 9, are necessary to demonstrate that.  In this
hypothetical case, the fact that NO implementation is able to meet all
of the provisions indicates that there is a serious problem with the
specification.

> which is presumably an implied goal of this WG.

But if, as you say, nobody cares about reaching the end of the Standards
Track, then even if there is zero cost in doing so, it does not follow
that more specifications will be advanced -- they might well not be
advanced *because* "nobody cares".

> which would imply a new committee, or offloading some other duties

Well we also have courtroom evidence that O.J. Simpson can't wear O.J.
Simpson's gloves.  Sometimes a demonstration which is claimed to provide
proof of something simply demonstrates what the demonstrating party would
like others to believe.  Note that the review need not consume many IESG
"cycles"; a WG Chair is supposed to produce documentation for advancement
to Draft, and if the documentation is inadequate (e.g. contains obviously
erroneous statements) it could (and should) be sent back.  If the criteria
for advancement aren't met, the review of that specification for that
period is over, and it is again reviewed in another 12 months.  Once the
criteria are met, of course a standards action is required; that is a
bread-and-butter procedure (I-D, Last Call, etc.).  Now it may well be
the case that that procedure can be streamlined -- the payoff would be
across the board (Informational, Experimental as well as Standards Track
and BCP) and would presumably also apply to ISDs.

> Indeed. And no maintenance of ISDs, either.

Therefore ISDs can't solve that problem; they can (all else remaining equal)
only make matters worse (by adding one more item for which there is no
manpower).  That's not a criticism of ISDs per se; it simply means that
the problem of premature WG closings needs to be addressed independently
of ISDs.
 
> I dispute this, see above.

If the change would result in declaring a clearly deficient (or maybe one
that is deficient, but not quite so clearly deficient) as satisfactory, the
quality bar is clearly being lowered.  As demonstrated by the simplified
hypothetical case above, I believe that's the case.
 
> something about what's realistically possible?

Not necessarily.  I'm not convinced about the Simpson gloves either.

> It seems to me that 
> Institute, it's very hard to do better.

That's a strawman based on an irrelevant criterion; the demonstration
of independent, fully-conforming, functionally equivalent interoperable
implementations is not for the sake of certifying conformance, it is
to demonstrate the soundness of the specification.  And it requires no
Institute; it is the WG Chair's responsibility to collect the information.
The quality bar that a WG Chair needs to clear is as low as it could
possibly be; two is the minimum number of implementations one could have
that qualifies as "interoperability".

> I really don't see an easy 

It seems possible to look at a list of RFCs which supposedly support a
claim and easily pick out obvious bogus ones (xx99, xx00).  That's not
not impossible, it's not rocket science.

> of course).

True that there's not necessarily any vulnerabilities.  However it's also
true that a feature which is not present can't be exploited (can't lead
to congestion, retransmissions, etc.).

> But I really don't see a logical difference between 

The difference is one which the community and the IESG has in the not-so-
distant past seen fit to choose reissuing, even though that meant resetting
from full Standard to Proposed, not once, but at least twice.  I refer to
STD 10 and STD 11, RFCs 821 and 822 plus others amending them, plus bits
and pieces of other RFCS not officially part of STD 10 or STD 11 but which
amended them (notably RFC 1123).  There is a considerable advantage in
having a complete and coherent specification in one place, rather than a
specification plus a large collection of "ignore section 3.1", "change
'foo' to 'bar' in section 4.5", etc. amendments.  A complete and coherent
specification is much easier for implementers to understand and implement
correctly, and that promotes interoperability.
 
> we can't meet in practice, and that tends to make people give up.

Maybe.  Maybe it demonstrates only an unwillingness.  Maybe it indicates
procedural or administrative issues.  I don't think it's a clear case of
"can't".
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 26 Jun 2005 00:14:51 +0200

> of the provisions indicates that there is a serious problem with the

There was also a famous real case in OSI days of a TP4 implementation
that passed the conformance test with flying colours because, whatever
it received, it returned an immediate RESET, which the spec said was
always a valid response. But are we really after requirements that
cater mathematically for such pathological cases, or are we after
a pragmatic standard?  I think that's what we're arguing about,
throughout this thread.

    Brian

.

---

From: Bruce Lilly <[email-masked]>
Date: Sun, 26 Jun 2005 09:57:17 -0400

On Sat June 25 2005 18:14, Brian E Carpenter wrote:

> always a valid response.

Fascinating.  So specifications sometimes need to be revised to say
what is really meant, to close such loopholes.

> But are we really after requirements that 
> cater mathematically for such pathological cases,

I suspect that the short answer is "yes".  After all, we require that
mere drafts containing ABNF be checked, that MIBS be verified, that
boilerplate text be absolutely, positively, character-by-character
verbatim (even where that makes no logical sense), etc.  Why would we
expect that the standards process itself would not be correspondingly
stringent?  If such cases aren't handled by the process, how can we
hope to reach the goal of true interoperability?
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html