Subject: [newtrk] question one for newtrk

From: [email-masked] (Scott Bradner)
Date: Thu, 27 May 2004 09:26:50 -0400 (EDT)

Spencer asked:
  "Is this the draft that was to come, or should we wait for another?"
   :-} ... to start discussing, I mean.
Yes, the Black/Carpenter ID is the one I was referring to.

Given this ID and the Dawkins/Crocker/Perkins one
(draft-dawkins-pstmt-twostage-02.txt) I'd like to get a series of focused
discussion going about  the different aspects of the work newtrk is
supposed to be doing. I'd like to try to reach consensus on enough issues
to have at least one revision of one or both of these IDs before San
Diego.

First I'd like to focus on the question of the level of IESG review for
whatever is the first formal stage of a new IETF standard track. 

I think that over the years (including mostly the years I was on it) the
IESG has toughened up the review and requirements for publishing a
Proposed Standard.  RFC 2026 (and RFCs 1310 and 1602 before it) assumed
that the IESG review would make sure that :

   "A Proposed Standard specification is generally stable, has resolved
   known design choices, is believed to be well-understood, has received
   significant community review, and appears to enjoy enough community
   interest to be considered valuable."

But, in particular, the review assumed:

   "further experience
   might result in a change or even retraction of the specification
   before it advances"

It seems to me that the IESG no longer assumes that a PS will get revised.
Considering that few PSs do get revised this may be a reasonable
assumption but it also might be part of the reason that few PSs get
revised - they are seen as "good enough" since they got such a careful
review and were generally revised a number of times during the review
process.

So the question I'd like the WG to first discuss is:

  "Should the first stage of IESG review of a standards track document be
  done under the assumption that few documents will be revised after first
  publication (and this must be "correct" the first time) or should it
  assume that we will have a functioning multi-stage IETF standards track?

(leaving aside the number of stages for now)

(This question relates to the first part of section 2.1 in the
Black/Carpenter ID.)

thanks

Scott

.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Keith Moore <[email-masked]>
Date: Thu, 27 May 2004 10:39:46 -0400

> I think that over the years (including mostly the years I was on it)
> the IESG has toughened up the review and requirements for publishing a
> Proposed Standard.  RFC 2026 (and RFCs 1310 and 1602 before it)
> assumed that the IESG review would make sure that :
> 
>    "A Proposed Standard specification is generally stable, has
>    resolved known design choices, is believed to be well-understood,
>    has received significant community review, and appears to enjoy
>    enough community interest to be considered valuable."
> 

IMHO, that's an entirely reasonable assumption.

But you didn't cite the entire criteria for PS.  2026 also says 

 "A Proposed Standard should have no known technical omissions with
 respect to the requirements placed upon it."

And I believe that in the vast majority of cases, it is the failure of
working groups to address _this_ requirement that causes IESG to find
reason to insist on changes.

Somewhere embedded here is a question as to what constitutes due
diligence to identify "technical omissions" on the part of either the WG
or IESG.  As the Internet has grown larger and more diverse, the number
of technical aspects that have to be considered has also become more
diverse.  The damage that is done when technically inferior protocols
are widely deployed has also increased.  Which means that it is entirely
reasonable for IESG to push back on WGs that haven't done due diligence.

In terms of encouraging deployment, the label we put on these documents
is largely irrelevant.  What matters to the industry is the appearance
of stability and agreement.  If we changed the label for the first
formally approved documents to Frob, we'd still have widespread
implementation of Frobs, regardless of what the actual criteria were.

IMHO, because poorly designed specifications have so much potential to
do harm, we have a responsibility to ensure that specifications that
we claim are stable and have agreement, are also technically sound
and suitable for wide deployment.  Relaxing our criteria for the first
formally approved documents won't help us meet this responsibility.

And in the long term, IETF's continued credibility as a standards-making
organization hinges on how rapidly we can obtain agreement on technical
specifications but also on how well our technical specifications
actually work when widely deployed.  To do this we must consider how
to approve _good_ specifications more quickly, rather than how to 
approve _incomplete_ specifications more quickly.

>   "Should the first stage of IESG review of a standards track document
>   be done under the assumption that few documents will be revised
>   after first publication (and this must be "correct" the first time)
>   or should it assume that we will have a functioning multi-stage IETF
>   standards track?

The first stage of IESG review of a standards track document should be
done long before a working group thinks a specification is anywhere
close to final form.  Ideally there should be review of design goals,
detailed problem scope, security architecture, and impact on other
constituencies long before a protocol specification is written.

To the extent that our problem is that working groups are failing to
recognize and accommodate technical requirements early in their design
processes, we cannot fix it by changing how IESG works after the WGs
think their designs are complete.

.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 27 May 2004 10:39:24 -0400

I would like to see an initial standards level which says "yes this has 
been reviewed, but be warned it may change."
The problem currently is that once deployments get out there, the bar to 
change gets set very high.  In several working groups I have seen folks 
objecting to changes to things that are still I-D because they have 
implemented and delivered to customers already.  At I-D level, we can say 
"that's your problem."  But if we have a first standardization level that 
we expect people to implement to, and we expect to make changes, we need to 
be very clear what degree of implementation is an impediment to change and 
what degree of backwards compatibility WGs are expected to maintain across 
such changes.

Yours,
Joel M. Halpern

.

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 27 May 2004 11:10:01 -0600 (MDT)

> or should it assume that we will have a functioning multi-stage IETF

A) Multi-stage IETF standards track is required for quality standards:

I believe that humans cannot do informal things correctly without
testing. In IETF scope, testing means implementing IETF standards and
using those implementations in real and simulated life. Thus, every
nontrivial IETF standard that has not been implemented is guaranteed
to be buggy, regardless of how well IETF/IESG review system works.

Consequently, the "standard must be correct the first time" is not a
requirement we can comply with unless we invent a new sub-standard
category for protocols that are specified but not tested and not
called "standards". The latter would effectively result in a
multi-stage IETF standards track anyway!

If IETF cares about quality of its standards, it must test and fix
them. A multi-stage standards track is then unavoidable.

B) Quality standards may not be something we can produce:

IMHO, IETF currently lacks infrastructure and possibly culture that
would allow efficient standard fixing. A meaningful multi-stage
standards track would require fixing widely deployed protocols. I do
not know if we have the guts and the resources to do that. Patching
workarounds is always easier short-term and is usually "not IETF
problem".

Thus, it may be better to remove standard "correctness" (i.e.,
quality) from our stated goals and focus on standards sufficient to
get the [increasingly patched] code "run". We should have a
single-stage standards track then. There is no reason to pretend we
can improve a deployed standard.

HTH,

Alex.
.

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 27 May 2004 11:28:50 -0600 (MDT)

Is there consensus that by "multi-stage standards track" we mean a
small finite set of predefined stages (probably 2 or 3 stages total)?

Is assigning a standard revision numbers (a potentially infinite set
of increasing versions) under consideration? If we use revision
numbers, we can have a singe-stage standards track but still apply bug
fixes to our standards (by releasing the same standard with a higher
revision number). Each revision would solve backword compatibility
problems as appropriate for that revision, of course.

For example, using the later scheme, we would not approve HTTP/1.1 as
the "last stage standard" today, but publish HTTP/1.2 with known
errata. Then wait another X years and publish HTTP/1.3 with then-known
errata. Etc.

Thanks,

Alex.
.

---

From: Keith Moore <[email-masked]>
Date: Thu, 27 May 2004 14:18:23 -0400

> There is no reason to pretend we can improve a deployed standard.

we have managed to do so in the past on several occasions.  

of course it's better if we do a good job defining the specification
before we start using the word "standard" to refer to it.  even then,
sometimes it will need revision.  vendors will generally implement 
those revisions if they think it will help them sell their products 
and it is not too onerous for them to do so.

Keith

.

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 27 May 2004 13:10:26 -0600 (MDT)

> we have managed to do so in the past on several occasions.

Sure. What I am trying to say (and the above quote sounds wrong out of
its context), is that we have to decide what our primary goal is and
that decision will determine the number of stages on the standards
track.

If our primary goal is to get the code running, there is no reason for
multiple stages. The code will most likely run, more-or-less, at the
first stage. If, on a rare occasion, we decide to improve the standard
based on implementation experience, we do not need a second stage to
do those exceptional improvements.

If our primary goal is to get the code running well, than we have to
accept that our yet-unimplemented standards are not good enough. We
then need several stages or revisions to do the standard right, based
on implementation experience.

Alex.
.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Thu, 27 May 2004 13:12:31 -0700

>   assume that we will have a functioning multi-stage IETF standards track?

Demonstrating my tendency to write very long sentences......

I don't want to endorse the implicit assumption that the stages of the 
standards track are hierarchical, or even have separate names - I think 
that in many cases, the "recycle at current level" is the right thing to do 
- in fact so often that I am not sure having "higher levels" is right.

Statements of perceived stability may actually survive better (see 
what-standards).

But I think the other strawman - "must be correct the first time" - is just 
unachievable, so that's most certainly wrong.

                   Harald

.

---

From: Keith Moore <[email-masked]>
Date: Thu, 27 May 2004 16:28:58 -0400

> multiple stages. 

As much as I like the sentiment,  I think we may have extrapolated too
far from the popular "rough consensus and running code" quote.  That,
and conditions have changed a bit.

There are typically several years' worth of development in the
difference between a prototype that "runs" in a limited set of
test cases for small sets of users from the same platform in ideal
conditions, and a protocol that can be reliably used by hundreds of
millions of users with various kinds of hosts and networks and wide
differences in the quality of connectivity between  those networks. 

In my view, the first formally approved, permenantly published version
of a standards-track specification should be 

a) Good enough that we don't know of any changes that appear to be
necessary for the code to be deployed and used (for its stated purpose)
by the Internet as a whole, and

b) Extensible enough that we believe that if we do find bugs or
limitations, we can change the protocol in a way that is not onerous to
implementors or users, and which allows for graceful transition to the
new version.

Which is not to say that "snapshots" or "prototypes" or some such aren't
useful, but there's a strong argument for very clearly labelling them
"temporary", ensuring that they are not confused with stable
specifications, and automatically removing the documents from official
repositories - say in one or two years after publication.

Keith

.

---

From: Keith Moore <[email-masked]>
Date: Thu, 27 May 2004 17:10:08 -0400

> But I think the other strawman - "must be correct the first time" - is
> just unachievable, so that's most certainly wrong.

If we insist that specifications be provably correct before approval, we'll
never manage to produce anything useful.  (perhaps correct, but not useful).

On the other hand, if we fail to do due diligence towards producing robust
specifications before we approve them, we'll just be helping to accelerate 
the demise of the Internet's flexibility.  Instead of having a network
which is versatile enough to support a wide variety of applications, we'll
have a network where a small number of applications compete for the
network to be optimized to support them, and other applications have to
masquerade as those popular applications.

We need to have a better understanding about which kinds of robustness
and completeness are absolutely necessary before wide deployment, versus
which kinds of robustness and completeness can be added later after
they are justified by wide deployment.

.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Thu, 27 May 2004 14:23:16 -0700

> they are justified by wide deployment.

Amen.

.

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 27 May 2004 15:47:48 -0600 (MDT)

On Thu, 27 May 2004, Harald Tveit Alvestrand wrote:

> I don't want to endorse the implicit assumption that the stages of
> the standards track are hierarchical, or even have separate names -
> I think that in many cases, the "recycle at current level" is the
> right thing to do - in fact so often that I am not sure having
> "higher levels" is right.

I agree, and [re]cycle counters are the revision numbers I asked
about. They are the tags that distinguish one cycle from another (you
have to tag cycles so that people and computers can refer to a given
cycle/version).

Cycling at a single level is still a multi-stage approach in a sense
that the first cycle most likely does not produce a "correct"
specification, but future cycles may achieve desired protocol quality.

Alex.
.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 27 May 2004 16:12:11 -0700

Joel,

JMH> I would like to see an initial standards level which says "yes this has 

Interestingly, the existing, formal requirements for the different levels would
satisfy your goals, I believe.  The problem is that the labels have been
administered in a way that makes the bar for each label much higher than
was intended.

This sounds less like a problem with the labels and more a problem with
how we run things.

d/
--
 Dave Crocker <mailto:[email-masked]>
 Brandenburg InternetWorking <http://www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>, <fax:+1.866.358.5301>

.

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 28 May 2004 11:36:35 +0200

> specification, but future cycles may achieve desired protocol quality.

I seem to detect violent agreement that we have to iterate to improve
the quality of our specifications. But we need to avoid the administrative
aspect of this iteration taking over and making the iteration hard
to achieve. IMHO the current three stage process is baroque and has
become an administrative burden. That's why I am still very much inclined
to collapse the three stages into one, plus a WG snapshot mechanism.
I like Alex's idea of an explicit version number system for when a
standard does get iterated.

    Brian
.

---

From: Keith Moore <[email-masked]>
Date: Fri, 28 May 2004 10:35:34 -0400

> administered in a way that makes the bar for each label much higher 

Circumstances changed.  The Internet grew, and the consequences of 
standardizing protocols that didn't have adequate security or 
scalability got worse.  The Internet also became more diverse, and it 
became much more likely that some new protocol would interfere with the 
operation of other protocols.

> This sounds less like a problem with the labels and more a problem with

Or maybe it's a problem with our expectations.

Bottom line: It's more difficult than it used to be to produce a 
specification that will work well and do minimal harm.

And yet people keep arguing that we need to be less diligent...

Keith

.

---

From: John C Klensin <[email-masked]>
Date: Fri, 28 May 2004 14:44:15 -0400

--On Friday, 28 May, 2004 10:35 -0400 Keith Moore
<[email-masked]> wrote:

> Circumstances changed.  The Internet grew, and the
> consequences of standardizing protocols that didn't have
> adequate security or scalability got worse.  The Internet also
> became more diverse, and it became much more likely that some
> new protocol would interfere with the operation of other
> protocols.
> 
>> This sounds less like a problem with the labels and more a
>> problem with how we run things.
> 
> Bottom line: It's more difficult than it used to be to produce
> a specification that will work well and do minimal harm.
> 

Keith,

On this one, the argument isn't about "less diligent"...

I'd like to drop down to a specific, concrete example in the
hope that we can at least agree on where we are disagreeing.

In the early days of the network, a number of protocols were
developed whose mode of functioning was

	* Sender opens TCP connection and sends a single-line
	string.  Depending on the protocol, that string may be
	more or less well specified.
	
	* Receiver sends some stuff (which, depending on the
	protocol, may be more or less well specified) and closes
	the connection.  

Some of them got documented into RFCs, some didn't.  And some
ended up being very heavily used, and some didn't.  The two sets
of categories are not necessarily the same.  To be even more
specific, let's take one of those, "whois" as an example.

Now, "whois" is fairly widely implemented and fairly well
understood to be part of the Internet protocol suite.  I suspect
that there are large fractions of the end user community who, if
asked to list applications-level Internet protocols in order of
importance would like "Web" (meaning http and https), "Email"
(meaning SMTP, POP3, and maybe IMAP, "web access" and some
proprietary stuff), and then "whois".  You will even note that,
for that group, "whois" is the first of those protocols they
actually know by name.   I am _not_ making this up.

Now, if we were designing "whois" today, we would want selective
access to information, more finely-grained detail, precise
specification of queries and responses, protection against
various [in]security-related attacks, and so on.  Whoops, we are
doing that redesign and have a whole WG devoted to it.

But it doesn't follow that "whois" is broken.  It is very widely
deployed, in extremely heavy use (the registry and registrar
figures about how heavily are, to me, appalling) and there is no
question that multiple independent implementations exist (the
terrible simplicity of developing an implementation may have
something to do with this, a lesson that, IMO, we should be
applying much more often).  Clearly the quality of the documents
haven't prevented any of that deployment or implementation, even
though a lot of bits about queries and responses could be better
specified (and, for specific groups of servers, have been,
although mostly outside the IETF process and the RFC series.  So
it meets our criteria for IS, right?   Well, no -- because some
IESG members have decided it (and "finger", for which a serious
attempt to better define and promote the thing was attempted)
are ugly, insecure, and won't scale (an astonishing argument for
something as heavily used as "whois", but that is another
issue).   Want to move it to Historic?  Which of "unused,
unimplemented, or unscalable" would you like to identify them
with? 

Where we have a problem here is that IESG (or particular ADs, or
particular people who speak up a lot on various mailing lists)
keep confusing "IS" with "recommended" or even "mandatory".
They have apparently decided that single-turnaround "open
connection/ send stuff/ get other stuff back" protocols are
reprehensible and shouldn't make it onto the standards track,
much less progress if they are already there.  Instead, ideas
for these sometimes useful quick-turnaround,
trivial-to-implement things are burdened down with enough
machinery for multiway handshakes, signed messages, PKI
structures, privacy protections, and so on to create... well,
the Wasa out of a sea kayak.  That is, IMO, just wrong and it,
and its generalizations, are causing a lot of nasty symptoms.
And some of the perpetrators and their sympathizers classify the
objections to those symptoms, and maybe the underlying problem,
as wanting to be "more lax" or "less diligent".   That is
problematic, not only on its face, but because there really are
some proposals out there that I think are to be "more lax" or
"less diligent" -- confusing this group of problems with them
blocks this class of protocols while getting complaints about
the _really_ "less diligence" proposals less serious attention
than they deserve.

And _this_ problem has a solution, one that has been on the
books for a long time.  It is for those who dislike these
ultra-simple applications protocols to write the AS that says
"simple single transaction-pair protocols, while once a
generally good idea, are considered really harmful and should
not be used on the public Internet unless secured in other
ways".  Specify the ways and the tradeoffs if you like.   Move
the thing down the standards track: having "whois" or "finger"
at IS _and_ with a status of "not recommended" has never been
inconsistent (or at least wasn't before we stopped using
"mandatory/ recommended/ optional/ not recommended".

But blocking consideration of these things -- whether for
advancement on the standards track or for assignment of port
numbers -- because they offend contemporary aesthetics, is just
not how we are going to make progress.

    john

.

---

From: Keith Moore <[email-masked]>
Date: Fri, 28 May 2004 16:35:25 -0400

Wow.  Lots of issues touched on in one message.

1. Regarding your example - I don't know how typical it is.  At least I
don't think it's typical of things IESG got bogged down in when I was on
IESG.  I have seen cases like you describe, but I've also seen cases of
protocols where the designers simply failed to try to understand the
security requirements, scalability requirements, impact on other
protocols, etc.   Maybe those cases aren't entirely representative
either, but they do involve some protocols that have (or seem likely to
have) a big impact on the net.

2. On protocol design - It has become very difficult for us to make 
simple protocols like "whois".  To some degree this may be because scale
and diversity have forced us to make protocols more complex.  (Handling
of character data is a good example).  But to some degree this may be
because we are doing more design by committee.  And even when we find
designers with clear vision and good intiution about what is needed,
they have to sell that vision to a more diverse set of interests and
backgrounds than they used to.

(fwiw, the whois scalability argument completely eludes me)

3. Does wide deployment and heavy use equate to soundness of design
(or lack of "brokenness")?   I don't think so, but I often hear 
arguments to that effect.  

Actually it seems to me that one of our most important jobs is to try to
arrange that widely deployed and/or heavily used protocols are less
broken.  Except for security, we can tolerate some degree of brokenness
in most rarely used protocols.  But brokeness in widely used protocols
is often extremely expensive.

One of the things I think we need to understand better is -
What is the right tradeoff between technical soundness and 
timeliness?  It probably varies depending on how widely the
protocol will be used.  But it seems that it's very easy for
us to delay things for months or even years in an attempt to
make a protocol less broken, but after which the protocol is
hardly improved at all.  There is a great deal of difficulty 
associated with fixing even minor problems that are discovered 
late in the process.   One answer is to try to recognize when
we are at the point of diminishing returns and ship whatever
we have at that point.  Another answer - one that I prefer -
is to try to identify problems earlier in the process, when
it is easier to fix things. 

4. I haven't seen a lot of cases where ADs 'keep confusing "IS"
with "recommended" or even "mandatory"'.  I do think that several ADs
believe that a protocol that exists in a vacuum - i.e. the absence of 
_something else_ in that space which is labeled as "recommended" or 
"mandatory", that "IS" will be _effectively_ "recommended" or 
"mandatory" in a practical sense.  And I think that's often a realistic
first-level assessment of how the market will behave.

I'm all for having explicit recommendation labels and applicability
statements.  But the market is going to find the peg that is the closest
fit for whatever hole it's trying to fill, and cram that peg into that
hole regardless of whether the peg is the right shape, color, strength,
etc.

Does that mean we should constrain ourselves to only making pegs that
happen to fit the largest holes in the market?  I don't think so.  But
it might mean that we need to be very clever about what we call those
pegs and how we sell them.  

(Even that might not help.  WebDAV was not supposed to be a remote file
system access protocol, but that's often how it ends up being used. 
Either NFS or CIFS would have been MUCH better for that purpose.)

Protocols sometimes get used for different things, and in different
ways, than intended by their designers.  If we evaluate the soundness
of a protocol based on some anticipated use case, we might end up
insisting on complexity and expense for features that are never used
in practice, because the protocol isn't used as we anticipated.
I don't know how to solve this problem.  

5. Regarding AD bias against simple protocols - I agree that it would
be unfortunate for a protocol to be rejected merely because it was
simple.  Though I can't recall a recent case where that's happened.

What I have often seen that resembles this, is a case where somebody
has a proposal for a protocol that tries to solve problem X, and it 
gets shot down because various people try to evaluate that protocol 
according to how well it solves problem X' or even problem Y.  

Of course attempts to _use_ protocol X for problem Y are quite 
common, so maybe this isn't as unreasonable as it seems.  But 
(from personal experience) it's really annoying when it happens
to _your_ proposal.

Keith
.

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 28 May 2004 15:19:14 -0600 (MDT)

> arguments to that effect.

And then those same people complain when they have to patch their
extremely popular OS every week or, as was the case just a few years
ago, save their document every two minutes to avoid loosing data when
their extremely popular editor crashes.

Quality has nothing to do with popularity, neither in real nor in
virtual world. And quality is a changing target (as opposed to
popularity).

> timeliness?

Call me an idealist, but I do not see a direct relationship between
technical soundness and timeliness. That is, it should not take us
much longer to develop quality protocols. If it does, the solution is
not compromising quality, but solving the core problems that cause
delays or produce ugly protocols. Unfortunately, I am not very
optimistic that those core problems can be solved, but that's another
question., and we should keep trying anyway.

> ways, than intended by their designers.  If we evaluate the
> soundness of a protocol based on some anticipated use case, we might
> end up insisting on complexity and expense for features that are
> never used in practice, because the protocol isn't used as we
> anticipated. I don't know how to solve this problem.

The above problem (wrong protocol scope) cannot be solved. Protocol
scope is what we take as a starting point, when WGs are formed. At
that time, we cannot reliably predict whether the scope is correct.
However, reviewers should try to identify non-essential features for a
given protocol scope and suggest moving those to separate document(s).
Protocol extension mechanism would have to support that, of course.

Alex.

.

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 28 May 2004 15:35:17 -0600 (MDT)

On Fri, 28 May 2004, Brian E Carpenter wrote:

> I seem to detect violent agreement that we have to iterate to
> improve the quality of our specifications. But we need to avoid the
> administrative aspect of this iteration taking over and making the
> iteration hard to achieve.

Cycling must be lightweight to make sense. Once IETF errata
maintenance is improved, we can simply cycle once there is enough
errata to warrant the next cycle. IETF/IESG would have to determine
what is "enough" in every case. If IESG approval was not required, we
could cycle with every errata entry, but I do not think it is a good
idea to remove Last Call and IESG review as a prerequisite for cycles.
After all, wrong errata can break the protocol.

Original protocol (first cycle) must accommodate cycles through minor
version numbers or other mechanisms. When doing testing, different
versions must be used to make sure implementations handle them as
specified. This is essential to ensure backward [non]compatibility
when cycling.

> ... I am still very much inclined to collapse the three stages into
> one, plus a WG snapshot mechanism.

If cycling is the way to go, we should try to use the same mechanism
for snapshots. That is, the first cycle is the first snapshot and then
the revision counters are incremented forever. At some point, the
draft becomes a standard, but that does not reset the cycle counter
back to zero.

Alex.
.

---

From: Keith Moore <[email-masked]>
Date: Fri, 28 May 2004 22:04:22 -0400

> for snapshots.

not clear.  I believe we occasionally need to do prototyping of 
protocols for the sake of  "proof of concept" or "proof of feasibility" 
or to better understand requirements of an eventual standard.  Those 
"proof of feasibility" protocols need specifications, but not complete 
specifications.  Deployment of such implementations should be strongly 
discouraged.  Nor is there any reason to assume that the eventual 
standard will resemble the prototype specification in any useful way.   
It would be confusing and misleading to treat the eventual standard as  
simply a revision of a prototype specification.

.

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 28 May 2004 20:28:19 -0600 (MDT)

On Fri, 28 May 2004, Keith Moore wrote:

> > If cycling is the way to go, we should try to use the same
> > mechanism for snapshots.
>
> protocols for the sake of "proof of concept" or "proof of
> feasibility"  or to better understand requirements of an eventual
> standard.  Those "proof of feasibility" protocols need
> specifications, but not complete specifications.  Deployment of such
> implementations should be strongly discouraged.  Nor is there any
> reason to assume that the eventual standard will resemble the
> prototype specification in any useful way.

True. The above arguments are arguments for snapshots. They are not,
IMO, arguments against reusing the cycling approach for snapshots. For
example, all snapshot cycles can use zero as a major revision number
or some simple convention like that.

Imagine that IPv4 did not have private IP addresses but there was a
different protocol for folks to play with IP-like things in the lab.
Not a very good design, although it would make distinction more
obvious and completely prevent leaking private IP addresses into
globally routable space.

> It would be confusing and misleading to treat the eventual standard
> as simply a revision of a prototype specification.

There is no perfect solution, I guess. Somebody will always be
confused. Nevertheless, millions of people working on and using
software do not find devel/stable distinction difficult to comprehend.
In this context, I do not see any substantial difference between
standards and software.

We want a distinction, but, IMO, it should be done on a higher level
than protocol names or revision numbers. Simplicity and uniformity in
low-level mechanisms should be preserved.

Alex.
.

---

From: Keith Moore <[email-masked]>
Date: Fri, 28 May 2004 23:57:14 -0400

>>   I believe we occasionally need to do prototyping of

I hesitate to use the term "snapshot" because it's already associated 
with so many things.  But I am firmly of the opinion that prototype 
specifications need to expire,  and be removed from the official 
repositories.

> software do not find devel/stable distinction difficult to comprehend.

Heaven forbid we should strive for clarity in publishing specifications.

There are not even close to millions of people working on RFCs, and 
even with that much smaller number of readers there is a widespread 
belief that all RFCs are in some sense "standard" or "approved" or 
"endorsed" by the community.

> low-level mechanisms should be preserved.

There is good reason for almost everything else about the mechanisms 
for producing prototype specifications and the mechanism for producing 
standard specifications to be different.  They need different formats, 
different acceptance criteria, different approval processes, different 
lifetimes.  There are also good reasons to have different publication 
mechanisms.

.

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 31 May 2004 10:48:36 -0600 (MDT)

Let me try to illustrate my point with specific examples below...

> "endorsed" by the community.

[I mentioned the number of software users to illustrate that a uniform
cycling/revision system work well for many. Its power is not in
numbers but in simplicity and uniformity.]

Yes, majority of RFC users believe that all RFCs are "stable".  That
belief cannot be changed by new cycling system or deletion of expired
documents from some repository. It can only be changed by not using
the term "RFC" with something that we believe is not mature, stable,
and endorsed enough.

Software releases use "development/stable" or similar naming
conventions to do the trick because the word "release" alone does not
mean much for most people. It is too late to assign new adjectives to
"RFC" because "RFC" already implies "stable" to most IETF users.
"Snapshot" could be that new word to indicate "unstable" documents.

IMO, we still can and perhaps should use the same low-level cycling
and numbering mechanism for standard track documents. As the document
matures, we simply change the high-level name from "snapshot" (or
whatever) to "Internet Standard" (or whatever). Here are examples of
IETF document revisions, for two documents:

	IETF Document {
		ID:        0-3345-a
		Revision:  0.2
		Status:    WG draft
		Name:      "Our new TCP optimizations for Gbit links"
		...
	}

	IETF Document {
		ID:        0-3345-a
		Revision:  0.3
		Status:    WG Snapshot
		Name:      "Fastee: TCP optimizations for Gbit links"
		...
	}

	IETF Document {
		ID:        3-1234-c
		Revision:  0.1
		Status:    individual draft
		Name:      OPES protocol for adapting hyperimages
		...
	}

	IETF Document {
		ID:        3-1234-c
		Revision:  1.2
		Status:    Internet Standard
		Name:      Hyperimage Adaptation Protocol
		...
	}

The first two examples are revisions of the same document (the
document ID is the same). It is possible that the first revision or
both are expired by now (not all attributes are shown above).

The last two examples are revisions of the same document. Revision
0.1, an individual draft, probably expired long time ago.  Revision
1.2 is a "stable" standard.

We see a clear status labeling distinction and a uniform low-level
numbering scheme. We can assign RFC number(s) to a document that
reached "Internet Standard" status, but that's a separate question of
"backward compatibility".

Is this a single-stage track? Yes, in a sense, iff there is only one
"stable" stage called "Internet Standard" or somesuch. No otherwise.

N.B. Document IDs can be symbolic (like current drafts) rather than
numeric (as in the above examples) but should not depend on document
status, title, authorship, etc. as the latter may change.

> I am firmly of the opinion that prototype specifications need to
> expire, and be removed from the official repositories.

A uniform cycling approach with revision counters can be used
regardless of whether we delete expired documents from IETF
repositories. It is a low-level mechanism independent on the
higher-level things like "expiration" or "archive-ability".

> > We want a distinction, but, IMO, it should be done on a higher
> > level than protocol names or revision numbers. Simplicity and
> > uniformity in low-level mechanisms should be preserved.
>
> for producing prototype specifications and the mechanism for
> producing standard specifications to be different.  They need
> different formats, different acceptance criteria, different approval
> processes, different lifetimes.  There are also good reasons to have
> different publication mechanisms.

As the above examples try to illustrate, the "everything else" can be
different even if the low-level document identification and revision
interface is uniform. I assume the advantages of uniform low-level
interface are obvious from document maintenance, tracing, addressing,
and search point of view. I do not know what are the advantages of
having two low-level interfaces (handling exactly the same thing --
document identification and revision cycling!).

Alex.
.

---

From: Larry Masinter <[email-masked]>
Date: Mon, 31 May 2004 12:46:27 -0700

I think the internet community would be best served if
NEWTRK would make the fewest number of changes to standards
track as possible; whenever you change the process, you wind
up with some of your documents under the old process and some
under the new process, with even more confusion.

If people can't tell which RFCs are stable and which ones
aren't, it isn't really necessary to change the titles,
add version numbers, etc. 

You could give people more information by annotating every
RFC (even the old ones!) with

 See http://www.ietf.org/status/rfcNNNN.html for more information.

and then providing more information: has it been implemented,
are there open issues that haven't been resolved, is it believed
to be obsolete, etc.

I'm puzzled why people are so eager to change the current
standards track rather than work on ways to make it work better
first, and changing it only when absolutely necessary.

> the term "RFC" with something that we believe is not mature, stable,

I disagree "it can only be changed".  If it were easy to determine
the status of an RFC in an obvious way, we wouldn't have as much
trouble. The problem right now is that the status of the RFC that
is *in* the RFC doesn't change over time, and so nobody can trust
it. It says "Proposed Standard" and you have no idea how many
people use it, and whether errors have been discovered.

> "Snapshot" could be that new word to indicate "unstable" documents.

I don't think we can fix things merely by applying new words,
because no one pays any attention to the status of a document
when the status itself doesn't track over time.

You can't really tell if a 'proposed standard' is stable WHEN
IT ISSUES. You can only tell after some time. I don't think
the nomenclature for software releases is applicable for standards,
because software updates are (appropriately, I think) more frequent
than standards updates. A software 'release' need only be stable
until the next release. This isn't the case with standards.

> and numbering mechanism for standard track documents.

<clip discussion of examples>
I think it's inappropriate because the entire system proposed
is predicated on assigning release 'numbers' when the documents
are finished, while what we mainly need is retrospective information
on older specifications about whether they remain valid after
they've been issued.

Larry
-- 
http://larry.masinter.net

.

---

From: Keith Moore <[email-masked]>
Date: Mon, 31 May 2004 18:21:53 -0400

> the term "RFC" with something that we believe is not mature, stable,

I had pretty much reached the same conclusion.  So basically I don't 
think we should be using the RFC label for specifications that we don't 
consider stable.  It follows that if we think there's a good chance 
that the specification will need to change (for instance, as a result 
of interop tests among early implementations) we shouldn't be calling 
that specification an RFC.

Now if it turns out that zero changes are needed to migrate a document 
from prototype to RFC, then we would benefit from having the same 
mechanism for publishing both kinds of documents.  However based on 
experience both with moving I-Ds to RFCs and moving Proposed Standard 
RFCs to Draft Standard RFCs, I'd expect that to be a rare occurrence. 
So using the same mechanism for publishing both kinds of documents is 
optimizing for the rare case.  To put it more bluntly, it's a waste of 
time and effort.

Keith

.

---

From: Keith Moore <[email-masked]>
Date: Mon, 31 May 2004 18:38:44 -0400

> first, and changing it only when absolutely necessary.

I don't think that's what's happening.  I think there are lots of ideas 
for how to make small tweaks to the process (not just the standards 
track) but people disagree on which small tweaks are useful or 
desirable, so we end up with lots of different tweaks being proposed.

The major premise of this group - that we can fix IETF productivity 
problems by only changing how we label its output - is an example of 
the desire to fix large problems with small tweaks.

I don't think anybody wants to make large changes if small tweaks would 
do the job.  Whether small tweaks can actually do the job is a separate 
question that I don't think we've tried very hard to answer.  Most 
people seem to accept that premise on faith, or intuition, or perhaps 
out of fear of opening Pandora's box.

Personally I'm puzzled why people are so eager to change how we label 
our output rather than work on ways to make the output better.  But 
maybe that's just me.

Regarding our standards track - I keep coming to the conclusion that 
it's really really difficult to improve on 2026 for stable documents, 
except perhaps to get rid of Full Standard.  I keep finding that we 
need something very close to Draft Standard (stable, well-defined, 
tested with multiple implementations, and suitable for wide deployment) 
and something very close to Proposed Standard (believed to be stable 
and well-defined, suitable for implementation and testing, but not yet 
suitable for wide deployment).  There are at least two problems here - 
one is that Proposed Standards are too easily confused with mature 
specifications that are suitable for wide deployment, and another is 
that we haven't paid enough attention to the engineering process 
_prior_ to Proposed Standard.

(fwiw, I also share your concern that we don't have good enough ways to 
gather and disseminate feedback on already published specifications)

.