Subject: Comments on draft-carpenter-newtrk-questions-00.txt

From: Randall Gellens <[email-masked]>
Date: Wed, 12 Jul 2006 16:18:19 -0700

As Ted said during tonight's plenary, the draft and the plenary 
discussion seem to be proposing either making (presumably 
significant) changes in the standards process, or doing nothing, when 
there may be other possibilities.  Ted mentioned that there were 
other things that could be done to improve the linkage between our 
specifications and running code.  It seems to me that if efforts in 
that area succeed, we'd make it much easier to advance 
specifications, since we'd have more implementations.

I'd also like to note that our specifications are not atomic; 
advancing from PS to DS means showing at least two interworking 
implementations of every feature and option.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Nothing in the world is more dangerous than sincere ignorance and
conscientious stupidity.
          --Martin Luther King, Jr.

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Fred Baker <[email-masked]>
Date: Wed, 12 Jul 2006 19:32:14 -0400

> implementations of every feature and option.

Yes. The questions, at least in my mind, are:

(1) what does it take to demonstrate DS qualifications, and
(2) what is the additional value of Full Standard?

I would like to believe that a well documented interoperability test  
constitutes DS qualification; the current DS qualification sets the  
bar somewhat higher than that, and I note that few documents actually  
achieve that, even though we can daily see implementations  
interoperating in the field at PS.

I'm not sure that Full Standard means a lot besides "obsolete".  
Personal opinion.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Wed, 12 Jul 2006 19:33:25 -0400

Hi, Randall,

> I'd also like to note that our specifications are not atomic; advancing 
> from PS to DS means showing at least two interworking implementations of 
> every feature and option.

Well said.

"moving to full standard for SIP requires moving to full standard for 
HTTP/1.1..."

This was actually the scariest thing I've heard in a SIP meeting, and that 
covers some ground.

Spencer

---

From: Eliot Lear <[email-masked]>
Date: Thu, 13 Jul 2006 08:45:55 +0200

> interoperating in the field at PS.

Some data to Fred's point:

By RFC, not by STD (obviously):

Status	1999	2000	2001	2002	2003	2004	2005
-------------------------------------------------------------
PS	102	119	71	105	103	131	169
DRAFT	6	6	2	4	7	7	3
STD	3(*)	2	0	8*	3	0	1

(*) 3 in 1999 were SMIv2 6 in 2002 were SNMP.

These are rough based on 10 minutes of scripting I did back in March.  I believe there are two reasons for the huge gap between PS and DRAFT:

 - it's difficult to get there (interop requirements, picking out
   uncommonly used features, etc)
 - nobody wants or needs to do the work (what GM in her right
   mind would want her experts working on something that neither
   generates new features nor fixes product bugs)

If Iljitsch's proposal is that the IESG "makFrom [email-masked] Thu Jul 13 02:48:13 2006
Date: Thu, 13 Jul 2006 08:45:55 +0200
From: Eliot Lear <[email-masked]>
To: Fred Baker <[email-masked]>
Cc: Randall Gellens <[email-masked]>,
	Iljitsch van Beijnum <[email-masked]>, [email-masked]
Subject: Re: Comments on draft-carpenter-newtrk-questions-00.txt

> interoperating in the field at PS.

Some data to Fred's point:

By RFC, not by STD (obviously):

Status	1999	2000	2001	2002	2003	2004	2005
-------------------------------------------------------------
PS	102	119	71	105	103	131	169
DRAFT	6	6	2	4	7	7	3
STD	3(*)	2	0	8*	3	0	1

(*) 3 in 1999 were SMIv2 6 in 2002 were SNMP.

These are rough based on 10 minutes of scripting I did back in March.  I believe there are two reasons for the huge gap between PS and DRAFT:

 - it's difficult to get there (interop requirements, picking out
   uncommonly used features, etc)
 - nobody wants or needs to do the work (what GM in her right
   mind would want her experts working on something that neither
   generates new features nor fixes product bugs)

If Iljitsch's proposal is that the IESG "makes a call" based perhaps on somebody's request with some modest effort to demonstrate that a spec is ready for the next step, I think that actually would be a fine two-step approach.

Eliot

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 13 Jul 2006 07:29:48 -0400

there is one useful aspect of our DS contortions.  When we do the 
work, we actually figure out which features turn out not to be used, 
and get them out of the spec.
OSPF had ToS routing in its PS specification.  It turned out that 
there was only one implementation.
So when the protocol was ready to advance, that feature was removed.
Knowing that the feature was not being used proved very helpful to us later.

Yours,
Joel

At 02:45 AM 7/13/2006, Eliot Lear wrote:
>These are rough based on 10 minutes of scripting I did back in 
>March.  I believe there are two reasons for the huge gap between PS and DRAFT:
>
>If Iljitsch's proposal is that the IESG "makes a call" based perhaps 
>on somebody's request with some modest effort to demonstrate that a 
>spec is ready for the next step, I think that actually would be a 
>fine two-step approach.
>
>Eliot
>

---

From: Eliot Lear <[email-masked]>
Date: Thu, 13 Jul 2006 20:30:11 +0200

Joel,
> Knowing that the feature was not being used proved very helpful to us

I agree, but I wonder if there is another way to accomplish the same
goal without quite the level of formalism.

Eliot

---

From: Pekka Savola <[email-masked]>
Date: Thu, 13 Jul 2006 14:03:51 +0300 (EEST)

> Status	1999	2000	2001	2002	2003	2004	2005
...
> I believe there are two reasons for the huge gap between PS 
> and DRAFT:
>

A part of that might also be caused by normative references problems. 
I don't think we have much data on that as we haven't run an 
experiment (yet).

>   generates new features nor fixes product bugs)

Some of this would in fact usually be documenting the product bugs 
that were caused by ambiguous or incorrect specification.  You're 
right that once you've figured out a way to fix a spec problem in YOUR 
implementation, there may be marginal interest in fixing it in the 
spec.  However, I believe doing so will reduce the load on customer 
support (and ultimately also engineering which will need to answer the 
escalated support issues), because many of the support issues are 
actually caused by interoperability between different products or 
vendors.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Eliot Lear <[email-masked]>
Date: Thu, 13 Jul 2006 20:50:54 +0200

> actually caused by interoperability between different products or

I'm not saying don't fix interoperability problems that actually have
impact in a two-step approach.  In fact if there *are* interoperability
problems, that's an argument to recycle the spec, IMHO.  I am saying
that having that as a formal requirement demonstrably deters people from
actually doing the work.  Doing that function survey is pain in the butt.

Eliot

---

From: Henning Schulzrinne <[email-masked]>
Date: Thu, 13 Jul 2006 09:01:58 -0400

Has this been exercised in the past, say, 5 years?

At least for widely-implemented protocols, such as SIP, there is no  
reasonable way to say "there is only one implementation that does X",  
as there is no plausible way to catalog all such implementations.  
Most of the implementors don't show up at IETF meetings and  
implementations are written by dozens of small start-ups, open source  
systems and other non-traditional sources.

This provision actually discourages any DS effort: the danger is that  
you can't find an implementation that does use a certain feature, you  
deprecate it and then find that there was some SDO or implementor  
somewhere that actually did find this extremely useful. That just  
makes everyone look silly.

On Jul 13, 2006, at 7:29 AM, Joel M. Halpern wrote:

> work, we actually figure out which features turn out not to be  
> used, and get them out of the spec.
>> If Iljitsch's proposal is that the IESG "makes a call" based  
>> perhaps on somebody's request with some modest effort to  
>> demonstrate that a spec is ready for the next step, I think that  
>> actually would be a fine two-step approach.
>>
>> Eliot
>>

---

From: Tony Hansen <[email-masked]>
Date: Thu, 13 Jul 2006 15:11:59 -0400

A variety of people at the plenary last night said "declare victory".
But I know that different people took that statement to mean different
things. To me, declare victory means "recognizing the reality of the
single level standard as it appears to be and moving on". This doesn't
mean to stop working on newtrk, but to refocus newtrk on recognizing
that fact. Put an end to the arguments about whether we should go to
1-level, 2-level or 3-level, and move on from there.

When a new spec is published on the standards track, it's meant to be
the new standard, and the industry should be using it, and that's what
industry is (more or less) doing. (The less comes into play once in a
while when a non-clueful company puts out an RFP/RFQ for an old standard
because "that's the standard".)

On the flip side is the question of when a standard is no longer a
standard. For this I think it should be based on whether there is a
group willing to work on: doing interop testing & maintaining an errata
list for the spec and/or working on a new version of it.

If no one is willing to do the testing necessary to find and generate an
errata list, the spec should automatically become Historic. (Pick a time
span, say 2 years.) So the burden then is laid on updating the errata
list in order for the spec to remain a standard. (There would be an
opportunity for an empty errata to be created only if there is interop
experience to back it up and only after a given time has elapsed.) If
people are interested in the standard, they should be willing to do the
minor amount of work to keep it from becoming Historic.

	Tony Hansen
	[email-masked]

---

From: Eliot Lear <[email-masked]>
Date: Thu, 13 Jul 2006 21:25:15 +0200

Tony,

> minor amount of work to keep it from becoming Historic.

We are in this position because we are not today documenting existing
practice, having not properly understood the lack of desire to do what
you class as "minor amount of work".  If the work truly is minor, then
again I have no problem with this approach, as it's actually a two step
approach that hides behind the word Historic.  But let us not get
ourselves into a position where we're marking widely used specs Historic
left and right simply because we couldn't get someone to jump through hoops.

And I think we're at the point where I'm violating my own rule.  There
is no concrete proposal here that I can argue in favor of or oppose, so
I think I'd better shut up and wait for one.

Eliot

---

From: Douglas Otis <[email-masked]>
Date: Thu, 13 Jul 2006 18:06:04 -0400

On Jul 13, 2006, at 3:25 PM, Eliot Lear wrote:

> We are in this position because we are not today documenting  
> existing practice, having not properly understood the lack of  
> desire to do what you class as "minor amount of work".

The SRD approach generates Name.Serial references to sets of  
documents after expending a "minor amount of work."  This minor  
effort reduces other efforts expected of the RFC editor, IESG, IETF  
WGs, and of those attempting to compile useful references.  When  
other standard bodies want to coordinate IETF changing references,  
Name.Serial makes predicting future references simple.  These stable  
and recognizable references should assist someone wanting to specify  
a level of compliance by picking the Serial number.  These references  
will likely be viewed as either Current or Stable by the community,  
rather than less meaningful Draft-Standard, Proposed-Standard, etc.   
When joined together in sets, the status of individual RFCs become  
far less relevant.  Once Serial.Name is instantiated, outmoded  
unsupported designators could be themselves made historic. : )

> If the work truly is minor, then again I have no problem with this  
> approach, as it's actually a two step approach that hides behind  
> the word Historic.  But let us not get ourselves into a position  
> where we're marking widely used specs Historic left and right  
> simply because we couldn't get someone to jump through hoops.
>
> And I think we're at the point where I'm violating my own rule.   
> There is no concrete proposal here that I can argue in favor of or  
> oppose, so I think I'd better shut up and wait for one.

I think there was general consensus for going forward with the SRD  
draft in Paris.  It is hard to understand who's oxen was being gored,  
and why the newtrk group failed to move forward, but the view that  
this represents additional effort is greatly mistaken.  When seen in  
the larger picture, this approach represents a significant reduction  
in efforts expended by all involved.

-Doug

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Fri, 14 Jul 2006 00:07:21 +0200

>I think I'd better shut up and wait for one.

Dear Eliot,
IMHO the ambiguity is in the IETF mission. This is why I oppose RFC 
3935 and the sentences I recently quoted about "influence" and a 
technology not being neutral to IETF values. The IETF should publish 
recipes. Not to want to provide guidance. Guidance should belong to 
groups committing to deliver comprehensive, consistant, and 
maintained network systems, with source code, updates, docs, faqs, 
support, etc. They should be the IETF deliverable users. Legitimately 
looking for user QA, and their needs to be considered.

Idealy Internet users should be provided every month a CD with the 
current release of their favorite netix brand. Like Red Hat, Debbian, 
etc. Let imagine the Linux situation if no distribution existed. The 
IETF is NOT "to influence people in the way they design, use and 
manage the Internet for it to work better" (cf. RFC 3935). It is to 
help development teams to better design solutions addressing the end 
users needs.

The only way to see an Internet architectural evolution is in having 
user internetwork operating systems, using the different IETF, IEEE, 
MPEG, ITU, ETSI, etc. propositions and the common bandwidth 
convergence in an innovative way. The danger in not preparing, 
supporting, and even opposing (as for the Multilingual Internet) such 
a necessary/observed evolution, is that these systems will cicumvent 
the IETF and will not benefit from its community/experience, and 
worse, to delay them.

jfc

---

From: "David Harrington" <[email-masked]>
Date: Thu, 13 Jul 2006 19:23:33 -0400

Hi,

I believe the rule was applied to the Entity MIB and the RMONMIB work,
IIRC.

dbh 

> -----Original Message-----
> From: Henning Schulzrinne [mailto:[email-masked]] 
> Sent: Thursday, July 13, 2006 9:02 AM
> To: Joel M. Halpern
> Cc: [email-masked]

> > Knowing that the feature was not being used proved very helpful to

> > us later.
> >> > I would like to believe that a well documented interoperability

> >> test
> >> March.  I believe there are two reasons for the huge gap between

> >> PS and DRAFT: