Subject: RE: [newtrk] question two for newtrk

From: Larry Masinter <[email-masked]>
Date: Thu, 01 Jul 2004 09:36:08 -0700

My opinion is that we're better off keeping three named stages
and trying to fix the progression; I think
the road to fixing the progression is to add more "mini-states"
between PS, DS, S using a more formal and recognized process
for identifying and fixing problems (not just "errata"), documenting
feature implementations and interoperability, and maintaining
accountability and responsibility for standards progression beyond PS.

I think the current three stages are coherent and are useful;
that is, it is useful to know (PS) if the specification seems stable
and technically sound, it is useful to know (DS) if there
are multiple independent interoperable implementations (where
the implementations represent separate exercise of licensing
process for any known IPR), and it is useful to know (S) when such
technical specifications have been widely deployed, etc.

The actual states corresponding to the IETF approval process
do happen and are good to document. Rather than changing the states,
fix the process to make it easier for the documents to
match the states.

Even if some other arrangement might be better (two states
instead of one, etc.), the benefit of the improvement
is unlikely to exceed the cost of making the change in the first place,
in terms of confusion, recategorizing old specifications, and
suffering the interim period where some specs are "old track"
and other specs are "new track".

Larry

.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 1 Jul 2004 11:31:59 -0600 (MDT)

On Thu, 1 Jul 2004, Larry Masinter wrote:

> I think the current three stages are coherent and are useful; that
> is, it is useful to know (PS) if the specification seems stable and
> technically sound,

I agree that "technically sound" and "free of known bugs" is a useful
bit. It would be a requirement for the first stage, regardless of
whether there are more stages. Thus, it is not a differentiator when
it comes to deciding on the number of stages.

As for "seems stable", I am not sure what you mean. For me, "stable
for X days" means that no RFC bugs where reported or RFC changes made
for the last X days.

There is no applicable history behind the first publication of an RFC
so it seems meaningless to talk about any stability as a PS
requirement. PS bugs can be found (and sometimes are found) the same
day PS is released. HTTP/1.1 (RFC 2616) is a good example.

> it is useful to know (DS) if there are multiple independent
> interoperable implementations

Why do we need a whole stage for that? Would an official table of
known independent interoperable implementations be sufficient for any
RFC user who cares about implementation experience? Some RFC users may
wait for 5 implementations, some for 1, and some would not care at
all. Why assign a label based on some arbitrary number of
implementations that would be too small for some class of RFC users
and too big for another?

> and it is useful to know (S) when such technical specifications have
> been widely deployed, etc.

Same as above. A deployment record can be maintained for each RFC. If
there are classes of RFC users that care about "wide deployment",
these classes would probably have different notion of "wide". So,
again, we would be assigning a label based on some subjective notion
of "wideness" that would be most likely inappropriate for some classes
of RFC users.

> The actual states corresponding to the IETF approval process do
> happen and are good to document.

The characteristics of RFCs you describe indeed exist, but different
RFC users define them differently and so the moment when they "happen"
is often different for different classes of RFC users. It is essential
to maintain data that enables users to characterize a given RFC based
on that user criteria. I do not see why it is useful to assign labels
(not to mention stages!) that cannot reflect RFC user value system.

Can you expand your "it is useful" assertions above into "it is useful
because ..." explanations, based on an RFC user point of view, and
given several different classes of RFC users?

> Even if some other arrangement might be better (two states instead
> of one, etc.), the benefit of the improvement is unlikely to exceed
> the cost of making the change in the first place,

Yes, but only if you expect IETF to disappear in the nearest future.
If IETF continues to function for at least 10 years, then correctly
fixing such a major problem now will definitely pay off long-term,
IMO.

> suffering the interim period where some specs are "old track" and
> other specs are "new track".

What confusion would a one-stage track model cause for the old RFCs?
Old RFCs are RFCs so it does not matter what their old label was.

I do not see a need to recategorize old specifications if a one-stage
track model is adopted.

I am not sure what "suffering" you expect from a one-stage track. We
will simply stop assigning labels to RFCs. Who is going to suffer and
why?

Thanks,

Alex.

.

---

From: Larry Masinter <[email-masked]>
Date: Thu, 01 Jul 2004 16:29:06 -0700

Alex, let me turn the question around. If you're proposing
changing from "three states" to "one state", what is the
benefit?

> (not to mention stages!) that cannot reflect RFC user value system.

I disagree with "cannot". If you had said "often do not", I wouldn't
have a problem.

The labels defined in RFC 2026 correspond to values of many users.

> given several different classes of RFC users?

I don't have quantitative data, but then, neither do you.

I can speak from a personal and anecdotal experience:

- Personally, I believe that the categories (PS, DS, and S) have
  been useful to me.

- I have given many (well, at least 8) tutorials that have included
  a section on the IETF standards process, and the various levels of
  specifications. The audiences have been interested in the topic,
  and my judgement by questions and discussions is that the documented
  levels are interesting and useful. (People paid attention, asked
  questions, didn't through tomatoes, etc.)

> fixing such a major problem now will definitely pay off long-term,

The short-term benefit is not worth the short-term cost, and the
long-term benefit is difficult to measure and likely to depend on
other factors such that it's not possible to predict and not worth
optimizing.

I'm not sure what you're proposing "fixing" that will have a "pay off".

Perhaps you're imagining some cost savings that I don't
see here. How will having only one state decrease the amount
of work? Surely documents will be revised and revisions will also
have to be approved. Will documents be revised less often?
Will there be fewer reviews? Will users be less confused?

Larry

.

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 02 Jul 2004 16:48:13 +0200

> it is useful to know (PS) if the specification seems stable
> technical specifications have been widely deployed,

True. But knowing those things does *not* require three different
coarse-grained labels. See draft-loughney-what-standards-01.txt
for one approach to this.

> and other specs are "new track".

Don't agree. It can be done overnight. If for example we went to
a single level, all PS, DS and S would be munged into the
single level - a bunch of pages at rfc-editor.org would need
to be updated. Everything goes to "new track" on the flag day.
Remember, we already have a single-step track (BCP) and it works
just fine.

     Brian
.

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 2 Jul 2004 09:50:46 -0600 (MDT)

On Thu, 1 Jul 2004, Larry Masinter wrote:

> Alex, let me turn the question around. If you're proposing changing
> from "three states" to "one state", what is the benefit?

A single-stage track is better than a multi-stage track because every
stage in a single-stage system corresponds to a well-defined,
objective characteristic that is the same regardless of an RFC user
class.

  ID stage (0):  specification not certified by IETF
                 to have no known technical bugs at publication time

  RFC stage (1): specification certified by IETF
                 to have no known technical bugs at publication time

Note the fundamental difference in approach with what have been
proposed by some (proposals do vary) proponents of a multi-stage
track:

  0: specification not certified by IETF
     to have no known technical bugs at publication time;
     please stay away

  1: specification certified by IETF
     to have no known technical bugs at publication time but
     not recommended for wide deployment as it is yet untested,
     may be unstable, and may never be widely deployed;
     please implement

  2: specification certified by IETF
     to be "stable enough" for some RFC users but
     possibly not widely deployed;
     please deploy

  3: specification certified by IETF
     to be "deployed widely enough" for some RFC users;
     this is the final stage so if you have been waiting
     for something, it's time to stop waiting!

The problems with stages (2) and (3) above are:

	- IETF attempts to certify something that is highly
	  dependent on RFC user class (stages 2 and 3)

	- IETF attempts to certify something it has no
	  direct influence over (stability and deployment
	  depend primarily on IETF users, not IETF)

	- stages may imply advice as to when to implement
	  or deploy, which is usually meaningless to many
	  user classes (including stage 1, in some proposals)

	- stages are arbitrary ordered: a buggy specification
	  may be stable and widely deployed; an unstable specification
	  may be widely deployed, etc.

As a result:

	- the stages are ignored by many IETF user classes,
	- the pressure to move to the "next" stage is minimal,
	- little work is done to move things to the "next" stage,
	- only the first stage is used for most RFCs,
	- the time/efforts bar to enter the first stage becomes higher
	  than intended ("because that's our only chance!"),
	- the documented procedures do not correspond to practice and
	  IETF loses more relevance/users,
	- a NEWNEWTRK working group is formed.

> I don't have quantitative data, but then, neither do you.

I did not ask for quantitative data. I asked for "because ..."
explanations/reasons to accompany existing assertions within a stated
framework/perspective. It is impossible to argue with an assertion
without assuming some reasons behind it, and I do not want to guess
what your reasons are.

>   questions, didn't through tomatoes, etc.)

The above is a proof that things like "stability" and "deployment" are
widely understood as very important. Nobody argues with that. Any
standards track must provide data that enables a knowledgeable IETF
user to decide whether an RFC is "stable enough" or "deployed enough"
(among other things). That's not a differentiation/explanation I am
looking for.

What are the reasons to believe that "stable enough" or "deployed
enough" means the same to such RFC user classes as folks implementing
a protocol on some internal backend invisible to clients, folks
implementing a protocol in a software appliance, folks doing silicon
implementations, folks implementing protocol testers, folks starting
up an independent certification lab, folks lobbying government, etc.

> Perhaps you're imagining some cost savings that I don't see here.

I am "imagining" solving a problem identified by the Problem WG, that
NEWTRK charter tells us to solve. Please see our charter and Problem
WG statements for reasons why the problem is worth solving (I hope
that we do not have to debate that!).

> How will having only one state decrease the amount of work? Surely
> documents will be revised and revisions will also have to be
> approved. Will documents be revised less often? Will there be fewer
> reviews?

All other factors being equal, simpler is always better. While
revisions will always happen, the debate whether "X" is a bug is
usually very specific and pretty short. The debate whether "Y" is
"stable enough" or "deployed enough" has exactly the opposite
characteristics. So, yes, we will be saving time/efforts/resources with
fewer stages.

A simple example is a protocol that, after a few revisions, does not
change. A year or so later, with a meaningful multi-stage track,
somebody will need to step up and ask, "Should we spend some resources
on debating and possibly moving this protocol to the next stage?".
With one-stage track, there is not reason to appoint a person/process
to ask the question, there is no question, there is no debate, and
there is no move.

Depending on proposals, a multi-stage track would require remapping of
old RFCs, changing our processes, our software, etc. So no matter what
this WG does, there will be short-term extra work. I have already
provided reasons why a one-stage track may actually require less
short-term work (no reason to remap old RFCs and/or fewer software
changes are required; we would not be changing stages, we would be
collapsing them; that's usually "cheaper").

> Will users be less confused?

With one-stage track, there are fewer things to be confused about.
There are fewer states/variables/changes. So, yes, I have reasons to
hope that users will be less confused.

Alex.
.