Subject: [newtrk] In support of Larry's proposal

From: Sam Hartman <[email-masked]>
Date: Wed, 03 Aug 2005 04:02:24 -0400

Hi.  Larry's proposal to base our groupings on interoperability
reports is the neatest thing I've heard in the process reform
discussion.  My initial impression is quite positive.  I would support
advancing this proposal to a point where we could try experiments with
it.  I think I could volunteer as an AD to find working groups to
conduct such experiments and to do my part of the experiments.  I do
not have time to write text for Larry's proposal but I do have time to
review text.

I also believe that SRDs could be combined with Larry's proposal in
some way and think that could work reasonably well.  (As an aside, I
think the SRD proposal is useful in and of itself and believe that
something along those lines could work.  I prefer things based on
Larry's proposal but believe I could successfully implement what would
be required of me as an AD under a fleshed out version of the SRD
proposal and believe doing so would not introduce unacceptable costs.)

I like Larry's proposal because it focuses on interoperability.  It
creates a new class of referenced item called an interoperability
report (IR).  we could set things up so that you only get an IR when
you actually manage to have two interoperable implementations.  That
is nice because it focuses significant effort on interoperability.
People could learn that if you are given a citation for an RFC and an
IR that contains that RFC, that citation rightly has more standing
than a citation for just an RFC.

Being able to have an IR citation might well become a goal of protocol
advocates.  In theory STD numbers might serve this purpose today.  In
practice I've been involved in efforts like Kerberos that would like
to have an STD number and that would attach some value to that.
However getting an STD number has required significantly more effort
than we could justify because of normative downreferences and so
justifying effort on working towards draft (much less full) has been
difficult.  I think IRs might be attainable in a since that STDs are
not.

If getting an IR citation becomes a goal, then we will have a
significant focus on documenting and testing interoperability.  John
has described all the benefits of doing this and has described why he
believes that style of thought is the key advantage of the IETF
process.

On the flip side, you would not need to have complicated discussions
of grouping in order to publish a proposed standard.  You don't need
to decide what group a proposed standard would belong to until you
want to join it to an IR.

The decision of when one version of a standard replaces another would
become explicit.  I think it is good.  In the mail community there
seems to be a strong consensus that RFC 282? should replace RFC 82?.
However I'm involved in an effort in the Kerberos community where the
opposite result is desirable.  We just published RFC 4120, 3961 and
3962.  Assuming we get interoperability documentation together these
three documents would likely meet the criteria for an IR on Kerberos.
However we're also working on a "next version".  When that comes out,
it will be a while before there is enough deployment that we would
want to replace the Kerberos IR with the new version.  If the feature
matrix in an IR was a consensus item, both the email and kerberos
communities could get what they want.

I heard and understand Bruce's objections to Larry's proposal.
However I believe that Larry's proposal strikes a better balance
between the conflicting goals than anything else I've seen.  Even
given Bruce's concerns I support the proposal.

--Sam

.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Bruce Lilly <[email-masked]>
Date: Wed, 3 Aug 2005 09:02:29 -0400

> not have time to write text for Larry's proposal but I do have time to
[...]
> difficult.  I think IRs might be attainable in a since that STDs are

We currently have implementation reports, which are required for
advancement to Draft status.  However, if I understand your comments
re. Kerberos correctly, there is some concern about downward references
(even with consideration of RFC 3967).  It might be worthwhile
exploring specifically what that concern is (as a side discussion).

> believes that style of thought is the key advantage of the IETF

As noted above, we have IRs, and they are required for advancement to
DS.  But they are unstructured (as opposed to RFCs (structure defined by
RFC 2223) and Internet-Drafts (in the form of RFCs, and with 1id-guidelines
and 1id-checklist as additional documentation of the format and content)),
and it appears that the IESG doesn't read them.  That is related to an
ongoing process discussion -- no matter what the on-paper procedure says,
if the IESG and other parts of the IETF don't actually follow the process,
we have a problem.

> want to join it to an IR.

For the oft-stated example of putting together an RFP, one is still faced
with choosing which IRs to include and exclude in order to specify a set
of technologies of interest.  There is a danger that there would be a
tendency to look at the products listed in IRs as a supplier "short list"
(no matter what "does not constitute an endorsement" disclaimers are made),
and that may have undesirable consequences in WG composition and direction.
 
> want to replace the Kerberos IR with the new version.

I'm not convinced that specification versions would be clarified.  In
essence, 2821 is a newer version (than 821) of a specification for the
same technology; it is not a specification of a different version of
the technology itself.  Other technologies may have separate versions
of the technology per se (e.g. S/MIME versions, IP versions).

> If the feature 

In order to be meaningful in terms of relationship to the specification,
the conformance criteria must necessarily be those specified in the
technology specification (i.e. those specified as being important for
interoperability and to prevent damage, viz. those using BCP 14 imperative
keywords or their equivalents).  That is why 2026 refers to "functionally
equivalent or interchangeable components of the system or process in
which they are used", and "requirement [...] applies to all of the
options and features of the specification".  The conformance criteria
need to be objectively and impartially established, such that any
observer can confirm (or not) that all of the specified technology is
covered.  Consensus (as the term has been used in the IETF) is rather
nebulous and open to subjective interpretation; there is the danger of
a WG being "stacked" with vendor-affiliated members to skew the choice
of features in a manner favorable to the vendor's interests if the
selection is left to consensus.  An objective choice of criteria
protects against that.

I believe the following are among the set of minimal issues that would
need to be addressed for such a proposal to advance:
1. establish confidence that the relevant review body can and will pay
   significantly more attention to the content of IRs than is apparently
   currently the case with RFC 2026 and the IESG; this is fundamental --
   there is not much point in specifying a new procedure unless there is
   the ability and will to abide by the procedure.  If necessary, there
   may need to be provision of a separate standards review/approval body,
   leaving IESG to manage WGs (indeed, establishing a separate group with
   the mandate to review standards and with approval authority might
   obviate any changes to the existing BCP 9 procedure save for the
   current references to the IESG as the responsible group). 
2. define a suitable structure for IR format.  Ideally, as interoperability
   reports are required for advancement from entry-level (PS) to DS, each
   PS should contain a summary of conformance criteria.  However, there
   is not now any such requirement or even suggestion, which probably
   explains why many WGs find advancement to DS difficult (i.e. because
   that step hasn't been planned for) and why IRs vary markedly in format,
   content, and quality.   A simple table of requirements and prohibitions
   should suffice for a summary of conformance criteria; an IR could then
   be a matrix of criteria vs. implementations.

Both items could be of benefit regardless of what direction is chosen; if
the IESG cannot manage standards review and approval in addition to WG
management, establishing a separate body to offload standards review and
approval would address that issue, whether we keep BCP 9 processes (modulo
the players), reduce the standards track to two steps, or eliminate the
standards track entirely.  And there seems to be some potential benefit to
formalizing interoperability reporting and encouraging standards track
specification developers to plan for interoperability testing and reporting
at an early stage of specification development.

So I would support working on either or both of the above if deemed within
WG scope.  We could consider them separate from any other issues (ISDs,
SRDs, other proposals), deferring specifics on specific other process
changes and summary document formats until these issues are resolved.

However, I note that there is discussion planned in Paris on final
disposition of ISDs (I am not in Paris, so cannot attend).  While discussion
of IESG comments on the ISD (and other) proposals would certainly help
clarify some of the issues, I'm not sure that final disposition is
possible or necessary at this juncture.

As I won't be in Paris on Friday, it would be presumptuous of me to suggest
an agenda item formally,  Instead, I'll informally suggest the following
topics which might be of interest, and which those who will attend are
welcome to formally suggest adding to the agenda if they wish:
o It has been stated that the IESG doesn't have the cycles to follow the
  BCP 9 process.  Dialog with the IESG regarding this might be illuminating;
  i.e. roughly how are IESG cycles spent among the various IESG tasks at
  present (esp. taking into account issues that may be decreasing in light
  of recent structural changes in the IETF administrative area)?  Which of
  those tasks could conceivably be offloaded (to some subordinate body, to
  IAB or other existing bodies), to a hypothetical separate-but-equal body?
o Would it be practically feasible to offload Standards Track and BCP RFC
  issues from the IESG, while leaving review of Informational and
  Experimental RFCs under IESG jurisdiction?  Or vice versa?
o What specific issues have precluded adequate IESG review of existing
  implementation reports in a manner consistent with BCP 9 definitions for
  advancement to Draft status?  In particular, why did nobody catch the
  obviously bogus references to xx00 and xx99 RFCs in the ABNF implementation
  report?
o If there are no overriding issues (e.g. apathy, lack of technical
  expertise), would a formal requirement or recommendation for a set of
  conformance criteria in entry-level (PS) Standards Track RFCs assist in
  review for advancement to DS?
o Would a formally specified format for implementation reports assist in
  review of specifications for advancement to DS (subject to the same
  caveats as above w.r.t. overriding issues)?
o Would a formal ballot for use at Last Call reviews assist the IESG in
  assessing Last Call comments?
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Sam Hartman <[email-masked]>
Date: Wed, 03 Aug 2005 09:16:58 -0400

Bruce, I think it is fairly clear that the IESG believes that RFC
2026's requirement for periodic review is impractical.  I think one of
the purposes of creating the newtrk working group in many peoples'
minds is to change the process and remove that review.

I respect that you disagree with people who believe that the 2026
periodic review is impractical.

But it sounds like you believe that the IESG is acting in bad faith on
this issue.  We are clearly not doing what 2026 asks us to do.
However I think we've raised the issue to the community and asked
people to look at changing 2026.  There is an ongoing forum for the
discussion of that and other changes: newtrk.

I think we are taking a reasonable approach to raising that concern.
I believe the IESG is acting in good faith to try and get the
requirement of periodic review removed from the process.

I completely agree with you that if a community consensus emerges to
preserve this requirement and the IESG still fails to meet it, then
the IESG will be working in bad faith.

But let's separate the issue of doing something you disagree with from
acting in bad faith.

--Sam

.

---

From: Sam Hartman <[email-masked]>
Date: Wed, 03 Aug 2005 09:18:55 -0400

I agree that we have interoperability reports today.  However they are
not commonly used other than as part of advancing to draft or
standard.  I believe Larry's proposal would change that and would
increase the number of technologies with interop reports.  Interop
reports today are not citable; I think that would be an important
change.

I agree with you that the feature matrix for an interop report needs
to come from the standards.  However that is not a purely mechanical
process: I believe that IETF consensus is required to approve such a
matrix.
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 03 Aug 2005 15:30:25 +0200

> process: I believe that IETF consensus is required to approve such a

Sam, I have multiple interpretations of your last sentence:

- "I believe that IETF consensus is currently required to approve the 
matrix before one starts filling in interoperability reports" - this is 
clearly not consistent with current practice, where implementors generally 
work out the matrix at the same time as they generate the reports, and the 
first time the community sees it is when the interoperability reports get 
cited in a Last Call for Draft.

- "I believe that the IETF is implicitly approving the matrix and the 
interoperability reports based on it as part of its approving the documents 
for Draft Standard" (this is consistent with existing practice, but I don't 
know if we've ever stated explicitly that we're doing that, or even been 
aware that we're doing so)

- "I believe that the IETF should switch to working out the matrix ahead of 
interoperability testing, that an IETF consensus process should be used to 
approve the matrix, and that this should be done as part of adopting 
Larry's proposal".

I'm sure there are a couple of other interpretations. Can you clarify?

                          Harald

.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 03 Aug 2005 15:35:01 +0200

> minds is to change the process and remove that review.

Note that the "-cruft-" effort, which is on the agenda for Friday, was a 
last-gasp effort to do something resembling that periodic review.
If that effort isn't successful, I am very much in favour of dropping all 
requirements for periodic review from the standards process.

                     Harald

.

---

From: Pekka Savola <[email-masked]>
Date: Wed, 3 Aug 2005 16:39:02 +0300 (EEST)

> requirements for periodic review from the standards process.

FWIW, IMHO decrufting is and would be very useful even if RFC 2026 (or 
its predecessor) didn't have any requirements for periodic review.
(And it should be applied to all the docs, not just standards track.)

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Sam Hartman <[email-masked]>
Date: Wed, 03 Aug 2005 09:45:43 -0400

>>>>> "Harald" == Harald Tveit Alvestrand <[email-masked]> writes:

    Harald> --On 3. august 2005 09:18 -0400 Sam Hartman
    Harald> <[email-masked]> wrote:
    >> I agree that we have interoperability reports today.  However
    >> they are not commonly used other than as part of advancing to
    >> draft or standard.  I believe Larry's proposal would change
    >> that and would increase the number of technologies with interop
    >> reports.  Interop reports today are not citable; I think that
    >> would be an important change.
    >> 
    >> I agree with you that the feature matrix for an interop report
    >> needs to come from the standards.  However that is not a purely
    >> mechanical process: I believe that IETF consensus is required
    >> to approve such a matrix.

    Harald> Sam, I have multiple interpretations of your last
    Harald> sentence:

"I agree with you that the IETF needs to take a consensus action that
approves the test matrix at some point.  Under the current process
that happens during last call to draft.  I think under Larry's
proposal that would need to be an explicit consensus action of some
kind."
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: "Larry Masinter" <[email-masked]>
Date: Wed, 3 Aug 2005 07:11:28 -0700

I'm not sure I made a "proposal", since a "proposal" has more
details worked out. I think I had an "idea", and the idea needs
some refinement.

Based on a few conversations this week, let me try to refine
my idea a bit -- if you're around in Paris and would like to
talk about this f2f, let me know.

I think there are a couple of different steps in creating a 
useful interoperability information:

(a) deciding on what constitutes a "feature", and which
  collection of features should be contained in a single
  report.

(b) reporting on implementations -- their identity, source,
  independence. 

(c) reporting on tested interoperability, with other
  independent implementations

I think "(a)" needs to be the result of a "rough consensus"
process, and that the generation of "what constitutes a feature"
requires a little thought. Sometimes you can break a spec
up by chapter, sometimes you think that every "MUST/MAY/SHOULD"
might be a separate feature, and sometimes it requires you to
write something. The collection of features may well cross
multiple RFCs, and the document that identifies or describes
each feature kind of corresponds (in scope) to what people
have been talking about with respect to ISDs.

I think (b) and (c) should not be -- can not be -- the result
of a consensus process. Individuals can report on implementations --
their own, someone else's that they're willing to test. Since
the IESG doesn't actually currently review the accuracy
of implementation reports -- and no one else does either --
it's better to make these clear, authored, individual statements.

I can imagine a "report" consisting of two parts: the
identification of the implementation, and then, for each
feature (as generated by a), either
   * this was not implemented
   * this is implemented
   * this is implemented and successfully tested against
    implementations X, Y, Z, ....

For each feature, the implementor should also be encouraged
to comment on the specification, special interpretations,
difficulties, etc.

My main concern in revising the process is to eliminate
work for the IESG that the IESG doesn't actually do
(like 'review interoperability reports'), and give the
work to groups that have the motivation to actually do
it. For example, the industry groups that run interoperability
tests and events are motivated to create the matrix, and
individuals with implementations may well be motivated
to report their implementations.

Given this focus on interoperability statements, we may later
want to update the matrix/schema (if the specs change), but
updating the schema requires some rough consensus.

I'm less certain about how to create a citable document
based on interoperability reports, except through the
desire to update the specifications based on errors
or flaws found during interoperability testing. So
I'm imagining other organizations will still cite RFCs.
But if we have some other way of discovering those
specifications which have "multiple independent interoperable
 implementations", then perhaps we would then only
need two states in standards track: "proposed standard"
and "full standard".

Larry

.

---

From: Pekka Savola <[email-masked]>
Date: Wed, 3 Aug 2005 17:59:51 +0300 (EEST)

On Wed, 3 Aug 2005, Pekka Savola wrote:
> FWIW, IMHO decrufting is and would be very useful even if RFC 2026 (or its 
> predecessor) didn't have any requirements for periodic review.

Oops, sorry for a brain fart -- I meant s/predecessor/successor/.
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Bruce Lilly <[email-masked]>
Date: Wed, 3 Aug 2005 15:55:23 -0400

On Wed August 3 2005 09:16, Sam Hartman wrote:
> 2026's requirement for periodic review is impractical.

I think it may be useful to distinguish the built-up backlog of PS and DS
RFCs that have not been reviewed from what would be required in a steady-
state condition once that backlog has been removed.  I also think it is
worth considering a longer review cycle -- 24 months/12 months is an
aggressive target.  I also think that it would be helpful to have a more
clear understanding of *why* those in the IESG who believe that review is
impractical feel that way; whether it's an issue of lack of availability
of some resource, a structural/organizational issue, etc.

> I think one of 

Yes, that has been proposed; one point that I've made is that an organization
that wishes to be known as an SDO but which does not wish to review
specifications is at best an unusual organization.
 
> periodic review is impractical.

Actually I agree that if we look at the backlog plus ongoing production of
Standards Track documents that would require future review and factor in
the disbanding of WGs, and look at the past IESG workload (particularly with
a lot of administrative tasks) it is impractical.  But that cannot be blamed
entirely on 2026 (that document is responsible neither for the backlog
nor for disbanding WGs).
 
> this issue.

I have tried to be careful not to ascribe motivations, but merely to
observe several examples of what appear to be process failures.

> We are clearly not doing what 2026 asks us to do. 

Agreed.

> requirement of periodic review removed from the process.

Off the top of my head, I cannot think of a single SDO (or SDO-like
regulatory agency) that does not have some sort of periodic review
process.  Removal of periodic review seems to me to be tantamount to
saying "let's not be an SDO".
 
> the IESG will be working in bad faith.

Or there is some structural or resource-limited problem that needs to
be resolved, or the task needs to be transferred to another group.  I
believe that 2026 was when approved community consensus.  We are agreed
that the IESG has not met the requirements set forth in 2026.  It is not
clear whether there has been a change in IETF community consensus to
abandon review (though it is clear that there are some in the IESG who
would favor that).  I'm still not prepared to ascribe the failure of the
IESG to meet 2026 requirements to bad faith, at least not in the absence
of a better understanding of specific reasons given for why reviews have
not been made, why reviews which have been made have deviated from 2026
processes, and why WGs are being disbanded before WG chairs (as designated
by 2026) can prepare implementation documentation for advancement to DS.

> But let's separate the issue of doing something you disagree with from

As far as I'm concerned they are separate.
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: James M Galvin <[email-masked]>
Date: Thu, 04 Aug 2005 15:35:40 +0200

Having talked to Larry this morning, I wanted to add my support for
his idea with a hopefully minor clarification.

I see his idea as an incremental change to the status quo with a lot
of value.  We currently have a 3 step process: proposed, draft, and
full.

For documents expected to progress on the standards track, the
working group submits "two" documents to be reviewed as part of
entrance to proposed.  The protocol itself and the interoperability
"checklist."  As Larry said, the checklist might be as simple as
putting all the MUST, SHOULD, MAY into a table, but there is likely
to be at least a little bit of writing.  The important thing is it
has community consensus.  One detail to consider more carefully is
that every additional RFC that updates the original protocol must
also update the checklist.  This suggests that checklist document has
normative references to all documents that an implementor needs.

Next, we create an option for the promotion to draft to be a strictly
automated, administrative task.  Two or more implementors simply
submit their completion of the checklist.  Just as we have today,
nobody really double checks the forms.  As soon as two are received,
the protocol automatically advances.  This removes a work item from
the IESG.  If there is a need to review the checklist then the
working group does what it already does today - reconvenes to review
the feature list and updates the checklist, and perhaps the protocol
specification if appropriate.  In this case we also do what we do
today, which is the documents are resubmitted to the IESG, and a
decision is made as to whether we need to recycle at proposed or the
advancement can still continue.  And we maintain community review and
consensus on changes.

Essentially, the incremental change is to formalize the
interoperability/implementation reports.  With that conceptual change
we make two changes to the existing process (which I think are minor)
and we have a pretty big win in terms of streamlining the process and
speeding up advancement of protocols, without loss of quality.

No changes are proposed for the promotion from draft to full standard
at this time, although perhaps a discussion of eliminating draft
standard and advancing directly to full standard deserves further
consideration.

Jim

--On Wednesday, August 03, 2005 7:11 AM -0700 Larry Masinter
<[email-masked]> wrote:

> newtrk
> resources:_____________________________________________________ web
> user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
> mhonarc archive:
> http://darkwing.uoregon.edu/~llynch/newtrk/index.html

.

---

From: James Galvin <[email-masked]>
Date: Thu, 04 Aug 2005 15:46:15 +0200

also update the checklist.  This suggests that the checklist document 
has normative references to all documents that an implementor needs.

Jim

.

---

From: John C Klensin <[email-masked]>
Date: Thu, 04 Aug 2005 10:13:09 -0400

Jim, Larry,

I think this is a good idea.  But I'm concerned that it would 
make a serious existing problem worse.   Our big problem these 
days, or at least one of them, is getting specifications to 
Proposed and getting them published.  You are effectively adding 
two additional obstacles to our being able to do that quickly 
and efficiently.  The first of those involves creating the 
checklist.  In my opinion, that would be worth it, but it isn't 
any more zero-cost than ISDs or SRDs are.  The second is more 
serious.  Part of the problem of getting things to Proposed 
today, IMO, is that we have become obsessed with getting them 
perfectly right because, probably, they will never be revised 
(at Proposed) or be revised for Draft.  If you change things in 
a way that substantially guarantees no revision process for 
Draft, you reinforce that obsession at, IMO, precisely the time 
that we should be encouraging more of an "it is just Proposed, 
flag the problem and move on" attitude.

      john

--On Thursday, August 04, 2005 15:46 +0200 James Galvin 
<[email-masked]> wrote:

> Having talked to Larry this morning, I wanted to add my
> support for his idea with a hopefully minor clarification.
>
> I see his idea as an incremental change to the status quo with
> a lot of value.  We currently have a 3 step process: proposed,
> draft, and full.
>
> working group submits "two" documents to be reviewed as part
> of entrance to proposed.  The protocol itself and the
> interoperability "checklist."  As Larry said, the checklist
> might be as simple as putting all the MUST, SHOULD, MAY into a
>...

.

---

From: James Galvin <[email-masked]>
Date: Fri, 05 Aug 2005 10:01:40 +0200

To the extent a goal is to speed up getting to proposed, you may be 
right, depending on what one believes is the real reason for the time 
it takes to get to proposed.

On the other hand, streamlining the advancement process is an overall 
improvement both in terms of process and speed, which is a "win" all 
by itself.  A benefit which I believe is greater than the burden of 
creating the checklist.

Jim

--On Thursday, August 04, 2005 10:13 AM -0400 John C Klensin 
<[email-masked]> wrote:

> Jim, Larry,
>
> I think this is a good idea.  But I'm concerned that it would make
> a serious existing problem worse.   Our big problem these days, or
> at least one of them, is getting specifications to Proposed and
> getting them published.  You are effectively adding two additional
> obstacles to our being able to do that quickly and efficiently.
> The first of those involves creating the checklist.  In my opinion,
> that would be worth it, but it isn't any more zero-cost than ISDs
> or SRDs are.  The second is more serious.  Part of the problem of
> getting things to Proposed today, IMO, is that we have become
> obsessed with getting them perfectly right because, probably, they
> will never be revised (at Proposed) or be revised for Draft.  If
> you change things in a way that substantially guarantees no
> revision process for Draft, you reinforce that obsession at, IMO,
> precisely the time that we should be encouraging more of an "it is
> just Proposed, flag the problem and move on" attitude.
>
>       john
>
>
> resources:_____________________________________________________

.

---

From: John C Klensin <[email-masked]>
Date: Sat, 06 Aug 2005 07:11:36 -0400

--On Friday, 05 August, 2005 10:01 +0200 James Galvin
<[email-masked]> wrote:

> To the extent a goal is to speed up getting to proposed, you
> may be right, depending on what one believes is the real
> reason for the time it takes to get to proposed.

Personally, I think there are a dozen reasons, perhaps several
dozen, why it takes us so long to get to Proposed.  I even
believe that, almost every time someone tries to make a decision
based on "the real reason", it leads our collective thinking
into the weeds.  

Some of those reasons could be fixed by making procedural
changes, others by making attitude changes, and still others not
at all (at least without sacrificing, e.g., quality).   But the
fact that there is no "the reason" should not prevent us from
trying to fix high-leverage factors nor should it suggest that
we not be cautious about adding additional factors.

> On the other hand, streamlining the advancement process is an
> overall improvement both in terms of process and speed, which
> is a "win" all by itself.  A benefit which I believe is
> greater than the burden of creating the checklist.

I agree.  But my main concern was actually not with the
checklist but with the mechanism chosen for speeding
advancement.  It is quite clear to me --without concluding it is
"the reason"-- that we are making efforts to get a document to
be completely comprehensive and exactly right, technically and
editorially, at Proposed Standard.  Whether successfully or not,
WGs and document editors spend time on it, the responsible AD
and then the IESG spends time on it, and the RFC Editor spends
time on it.  Cumulatively, it certainly represents a lot of
time: if we could avoid documents entirely and do things by
direct brain-to-brain transfer between the originating WG and
the final standards-user, we would see a quicker process.
Clearly, there is a balance to be found between "document that
is good enough, but achievable" and "document that has achieved
a higher level of perfection" and that reaching that balance for
a given document requires discretion and good judgment.

If we establish a firm norm that says, as I think Larry's
proposal does (the absence of something specific, in I-D form,
is a problem at this level of detail), that specifications will
normally advance from Proposed to Draft when checkoff items are
completed, rather than requiring revision of the RFC text, that
creates considerable _additional_ pressure on getting the text
perfectly correct at Proposed.   That is not, IMO, helpful.
Worse, it may not save any time in the overall process: it might
just shift more editorial work from pre-Draft to pre-Proposed.

    john

.

---

From: Sam Hartman <[email-masked]>
Date: Sat, 06 Aug 2005 09:23:41 -0400

>>>>> "John" == John C Klensin <[email-masked]> writes:

    John> certainly represents a lot of time: if we could avoid
    John> documents entirely and do things by direct brain-to-brain
    John> transfer between the originating WG and the final
    John> standards-user, we would see a quicker process.  

This is undoubtably true.  However a lot of the effort is actually
spent clearing up details of the proposal in the WG's mind and no
efficiencies in dealing with the document would remove this effort.
For example even if you understood what Larry wanted perfectly, you
would still be left with a partially formed muddle.  The idea is just
at that stage of development.

    John> If we establish a firm norm that says, as I think Larry's
    John> proposal does (the absence of something specific, in I-D
    John> form, is a problem at this level of detail), that
    John> specifications will normally advance from Proposed to Draft
    John> when checkoff items are completed, rather than requiring
    John> revision of the RFC text, that creates considerable
    John> _additional_ pressure on getting the text perfectly correct
    John> at Proposed.  That is not, IMO, helpful.  Worse, it may not
    John> save any time in the overall process: it might just shift
    John> more editorial work from pre-Draft to pre-Proposed.

I agree that this is a potential problem.  I am not sure that I would
consider it a blocking problem especially for the experimental phase.
However I'd invite you to think about ways of avoiding the problem; I
will do the same.  Larry already has a mechanism to collect comments
associated with testing events--things you needed to interpret in
order to understand the spec, things that might require updating.  The
question is how to combat a desire to get things perfect to avoid a
round of revisions.  That is almost certainly going to be an attitude
issue more than a process issue.  The process clearly will allow
revision.

--Sam
.

---

From: Sam Hartman <[email-masked]>
Date: Thu, 04 Aug 2005 10:01:48 -0400

>>>>> "James" == James M Galvin <[email-masked]> writes:

    James> Having talked to Larry this morning, I wanted to add my
    James> support for his idea with a hopefully minor clarification.

    James> I see his idea as an incremental change to the status quo
    James> with a lot of value.  We currently have a 3 step process:
    James> proposed, draft, and full.

    James> For documents expected to progress on the standards track,
    James> the working group submits "two" documents to be reviewed as
    James> part of entrance to proposed.  The protocol itself and the
    James> interoperability "checklist."  As Larry said, the checklist
    James> might be as simple as putting all the MUST, SHOULD, MAY
    James> into a table, but there is likely to be at least a little
    James> bit of writing.  

I would object to requiring an interop framework be submitted with
each PS.  I think that raises the bar for PS too high.

I think thet interop frameworks should be a new class of approved item
that can be submitted when it is appropriate.

In general I would expect WG charters to include doing the interop
framework.

I also think it is important to assign some sort of identifier the
first time an interop framework has two full interop reports; his
might be like STD numbers today.

.

---

From: John Leslie <[email-masked]>
Date: Sat, 6 Aug 2005 11:47:21 -0400

> based on "the real reason", it leads our collective thinking

   Listening to Ted Hardie in the NEWTRK session at IETF63, I began
to see our problems in a new light.

   The audio archive doesn't seem to be up yet: as nearly as I can
remember, Ted said standards and specifications are different things,
and we may need to move to a different model of specifications. Ted
sees specifications as more important than standards.

   I tend to agree -- or at least agree with what I think Ted said.

   "Specifications" describe what something is; "standards" describe
how to measure something. We've named a three-level "standards"
track, but it's getting populated with "specifications" IMHO.

   We list many "protocols" as being "proposed standards", while
in fact they describe how to do something rather than how to measure
compliance. It is not clear there _is_ any interest in progressing
these to "standards" which describe how to measure compliance.

   Maybe our problem is the lack of a named "specifications track".

--
John Leslie <[email-masked]>
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: John C Klensin <[email-masked]>
Date: Sat, 06 Aug 2005 12:08:40 -0400

--On Saturday, 06 August, 2005 09:23 -0400 Sam Hartman
<[email-masked]> wrote:

>     John> documents entirely and do things by direct
> brain-to-brain     John> transfer between the originating WG
> and the final     John> standards-user, we would see a quicker
> process.  
> 
> This is undoubtably true.  However a lot of the effort is
> actually spent clearing up details of the proposal in the WG's
> mind and no efficiencies in dealing with the document would
> remove this effort. For example even if you understood what
> Larry wanted perfectly, you would still be left with a
> partially formed muddle.  The idea is just at that stage of
> development.

Of course, we are in complete agreement about this.

>     John> If we establish a firm norm that says, as I think
> Larry's     John> proposal does (the absence of something
> specific, in I-D     John> form, is a problem at this level of
> detail), that     John> specifications will normally advance
> from Proposed to Draft     John> when checkoff items are
> completed, rather than requiring     John> revision of the RFC
> text, that creates considerable     John> _additional_
> pressure on getting the text perfectly correct     John> at
> Proposed.  That is not, IMO, helpful.  Worse, it may not
> I agree that this is a potential problem.  I am not sure that
> I would consider it a blocking problem especially for the
> experimental phase. However I'd invite you to think about ways
> of avoiding the problem; I will do the same.  Larry already
> has a mechanism to collect comments associated with testing
> events--things you needed to interpret in order to understand
> the spec, things that might require updating.  The question is
> how to combat a desire to get things perfect to avoid a round
> of revisions.  That is almost certainly going to be an attitude
> issue more than a process issue.  The process clearly will
> allow revision.

Agree about the attitude issue, but certain process models may
support bad attitudes and vice versa.  We also, as others have
pointed out, need to be careful that an IETF-sanctioned
interoperability report living document or checklist doesn't
back us into the various conformance testing --or at least
certification of testing suites/methods-- without our noticing.

    john

.

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 09 Aug 2005 15:44:48 +0200

>    Maybe our problem is the lack of a named "specifications track".

Well, quoting RFC 2026:

3.  INTERNET STANDARD SPECIFICATIONS

    Specifications subject to the Internet Standards Process fall into
    one of two categories:  Technical Specification (TS) and
    Applicability Statement (AS).

So this is apparently only a naming issue. Very few RFCs actually
fall into the AS class.

     Brian

.

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 09 Aug 2005 16:06:47 +0200

> just shift more editorial work from pre-Draft to pre-Proposed.

Only if you *require* a new RFC to go with the implementation
report. Combining this style of implementation report with
a one stage process seems entirely possible to me - then we
would have standards track documents with implementation reports,
and inferior ones without them.

     Brian

.