Subject: [newtrk] On the priorities of process change

From: Pekka Savola <[email-masked]>
Date: Thu, 4 Aug 2005 09:42:24 +0300 (EEST)

(This might have been appropriate to the IETF list but decided to go 
for a smaller scope this time.)

Bert and Sam's comments on the plenary made me think of the process change
priorities a bit.

I'm symphatetic to the belief that we must not work on too many things in
parallel; if we do, there is a danger that the community is not able to give
sufficient concentration to the process improvement effort.

In that light, I've gotten the feeling that either:
  a) we must get to a resolution on SRD/ISD Very Soon Now (so that we 
can move on to focus on other things), or

  b) it might be best to abandon (or put on a back burner) the SRD/ISD 
approaches, because AFAICS, in the grand scheme of things, I don't 
think they actually provide significant improvement on timeliness or 
quality.

(But rather, the approaches "just" improve the external representation 
of our work and make us more compliant with RFC2026. Maybe the 
decision to charter the WG was a bad one, years ago, or maybe it was 
an attempt to collect "low-hanging fruit" without realizing that we 
didn't know where the fruit was and low or high it was.)

It seems that we should be better focusing on the more pressing 
problems (which are out of scope of this WG):
  - improving the IESG management/leadership capabilities to better lead,
    steer, help, review, etc. [Harald's twolevel, John's standard review
    panel, etc. proposals]
  - improving the document workflow (up to 'Publication Requested').
    I doubt this requires process changes, rather than cultural changes or
    tools improvements.  However, I doubt this could get proper attention
    prior to the IESG's load on the day-to-day techincal job is decreased.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Douglas Otis <[email-masked]>
Date: Thu, 4 Aug 2005 10:42:46 +0200

On Aug 4, 2005, at 8:42 AM, Pekka Savola wrote:

>  a) we must get to a resolution on SRD/ISD Very Soon Now (so that  
> we can move on to focus on other things), or
>
>  b) it might be best to abandon (or put on a back burner) the SRD/ 
> ISD approaches, because AFAICS, in the grand scheme of things, I  
> don't think they actually provide significant improvement on  
> timeliness or quality.

The SRD proposal specifically avoids making any changes to the  
process, beyond including updates to the tiny SRD document when  
submitting I-Ds for IESG action.  The intent is to offer a means to  
facilitate and consolidate information related to specific  
endeavours.  Having an asynchronous Status database that references  
the consolidation documents related to such specific endeavours  
permits the IESG and the IETF to communicate with the greater  
community, independent of a more formal RFC process.  This (reviewed  
or moderated) information should help provide a means to discover the  
information needed to better manage the many RFCs.  As a side  
benefit, the structural SRD overlay should provide stable reference  
anchors and improved presentation and event notifications.

-Doug
.

---

From: Pekka Savola <[email-masked]>
Date: Thu, 4 Aug 2005 14:34:41 +0300 (EEST)

I changed the subject..

On Thu, 4 Aug 2005, Douglas Otis wrote:
> The intent is to offer a means to facilitate and consolidate 
> information related to specific endeavours.  Having an asynchronous Status 
> database that references the consolidation documents related to such specific 
> endeavours permits the IESG and the IETF to communicate with the greater 
> community, independent of a more formal RFC process.  This (reviewed or 
> moderated) information should help provide a means to discover the 
> information needed to better manage the many RFCs.  As a side benefit, the 
> structural SRD overlay should provide stable reference anchors and improved 
> presentation and event notifications.

As a first observation, Bill Fenner's scripts could allow automating 
the populating the lists for references.  That is, we could produce 
HTML pages today which list all the docs (whether RFCs or drafts) 
which refer to <RFC X>, for {all X}.  It can't classify them reliably, 
though for later docs, norm/informative split could be automatic, and 
it could look up the category/maturity of the document from a 
database.

So, the first meta questions are:
  - is the classification ("SMTP documents") sufficiently valuable to
    be worth the effort?
  - is the classification of references, so far as it can't be made
    automatically, worth the effort?

A few more specific ones:
  - As far as I understand, SRDs are supposed to be done for all the
    existing docs (at least stds track)?  How do you make the initial
    classification ("SMTP" is a relatively simple example)?  Who would
    make that classification?

  - how much trouble would it be maintaining and moderating the
    information that goes on the SRDs?

I.e., my question is, to what extent this could be automated so that 
we don't need to do anything at least at this point before we know 
whether this is really useful or not?

---

From: Larry Masinter <[email-masked]>
Date: Thu, 04 Aug 2005 05:57:10 -0700

After a couple of conversations this week, I wonder if
we might have some time on the agenda to talk about the
"implementation/interoperability framework" idea in the 'newtrk'
meeting.

I believe that the list of "features"  with references to specific
paragraphs/chapters/sections of RFCs might contain some of the
information currently proposed for an ISD, but put into a much
more operational form.

To take the example of 'mixer', Harold has offered
draft-ietf-newtrk-sample-isd-00.txt. Consider instead
what an "implementation/interoperability framework" for
mixer might contain. It would need to cite all of the
relevant RFCs, and contain a list of features. If necessary,
the commentary about status of different drafts would also
be contained in it. So almost everything that is in Harald's
example ISD could appear in the implementation framework
document.

With implementation/interoperability frameworks and individual
publication of implementation/interoperability reports that
fill out the framework, we could make the progression from
PS to DS more "automatic": a document could reach DS if there
were multiple independent interoperable implementations of
every feature. The IESG wouldn't have to deliberate or review
this.

In addition, promotion from DS to Standard could also be
assisted, by asking implementation reports to report on 
"order of magnitude of number of copies of implementation
in operational use". I think there would still need to be
some judgment about veracity of such claims and also what
constitutes "widespread deployment" (which might differ
based on the type of specification).

In that sense, I think we could keep the three-step process,
but make the steps after PS more automatic, at least from
the IESG point of view.  Of course, it may well be valuable
for individual groups to revise specifications to remove
unimplemented 'features'.

One thing that would change is that we would ask groups
that create Proposed Standards to also create new feature-lists
or propose updates to existing feature-lists. Most likely,
the schema for an implementation report, when there are multiple
versions of the specification, would allow implementations
to report on which version of which specification they
implemented.

Robert Snively sent me a IEEE proforma for interoperability
http://grouper.ieee.org/groups/802/3/efm/public/comments/d3_0/pdfs/PICS-59.p
df
I think we should (at least for now) keep the IETF focus on
independent interoperability testing rather than "conformance",
but it is useful to see that this having a standard way
to report implementation not a new idea in the standards world.

Larry
-- 
http://larry.masinter.net

.

---

From: Douglas Otis <[email-masked]>
Date: Thu, 4 Aug 2005 15:16:41 +0200

On Aug 4, 2005, at 1:34 PM, Pekka Savola wrote:

>> information related to specific endeavours.  Having an  
>> asynchronous Status database that references the consolidation  
>> documents related to such specific endeavours permits the IESG and  
>> the IETF to communicate with the greater community, independent of  
>> a more formal RFC process.  This (reviewed or moderated)  
>> information should help provide a means to discover the  
>> benefit, the structural SRD overlay should provide stable  
>> reference anchors and improved presentation and event notifications.
>>
>
> As a first observation, Bill Fenner's scripts could allow  
> automating the populating the lists for references.  That is, we  
> could produce HTML pages today which list all the docs (whether  
> RFCs or drafts) which refer to <RFC X>, for {all X}.  It can't  
> classify them reliably, though for later docs, norm/informative  
> split could be automatic, and it could look up the category/ 
> maturity of the document from a database.

Bill's efforts would be instrumental at establishing the initial  
rough drafts.  This effort should be reviewed however.  Part of that  
review process should include relevant WG and/or interested parties  
to correct omissions or mis-classifications.  There is a SRD  
classification currently called "companion" which is intended to  
include closely related endeavours, and perhaps general definitions.   
Maybe there should be another category "definitions" to split the  
"companion" category.  As the number of classifications is minimal,  
adding this category may provide better clarity regarding when direct  
RFC or SRD references should be used.  I may be overly optimistic  
thinking that reference classifications will not prove to be too  
contentious.

>    automatically, worth the effort?

SMTP may have been a poor example, and perhaps should be removed from  
the draft.  The benefit of creating the SRD document set should be  
proportional to the difficulty.  This work must be done when  
attempting to understand what is involved within a particular  
implementation.  The SRD structure conveys this effortlessly in  
various forms, and provides a means for individuals to quickly  
understand RFC relationships to the endeavour and each other.  Having  
a means to consolidate information under specific endeavours, rather  
than specific RFCs, provides stable reference anchors.  An anchor  
permits a communication channel that would not otherwise exist.  From  
my perspective, it would be worth the effort to include all useful  
RFCs, where those not included could be described as cruft.

>    make that classification?

Newtrk could make an initial effort to overview the assembly of the  
SRD documents, perhaps together with an effort of the tools group.  I  
understand there should be a change in the method currently used to  
reference RFCs, for example.  The reference databases used in RFC  
edit process should be available from the ietf.org web site, for  
example.  There should also be tools created that cross-reference  
SRDs to RFCs.  The Status database may be useful for collecting  
feedback as this work proceeds.

Perhaps after a few iterations, these SRD documents could be  
presented to relevant working groups, where possible, for closer  
review.  Once these SRD documents appear to be stable, the IESG would  
need to decide whether to accept the SRDs by then requiring the  
modification of these documents as a requirement for I-D draft  
submission.  The "wip" SRD (modified SRD) should also be useful to  
track current work, in addition to providing a stable reference for  
completed work.

>    information that goes on the SRDs?

The typical SRD document should be about a half of a page of text.   
Once there is a complete set of SRD documents created, these could  
act as templates, when making changes to an existing RFC.  Where a  
new endeavour in involved, a new "wip" SRD would indicate this new  
territory, but even this document should be able to use a similar SRD  
as a starting point.  While this may not be completely free of  
effort, it should not encumber the submission process.  If anything,  
it should help track the initial efforts made early in the process,  
and ensure those interested in the endeavor are made aware of the  
document's progress.

> I.e., my question is, to what extent this could be automated so  
> that we don't need to do anything at least at this point before we  
> know whether this is really useful or not?

I think even the initial efforts should be able to prove their worth  
by way of improved presentation, without requiring efforts beyond a  
fine tuning of various script outputs.  The SRDs could be used as  
inputs to a graphical display as well.  Nothing would be required  
beyond efforts creating the initial set of SRD documents, unless SRDs  
are considered helpful by the IESG.  Even then, should it be later  
decided to be more effort than benefit, SRDs can be dropped without  
any other change.

The SRD experience could help with any follow-on efforts to deal with  
RFC categorization.  Perhaps an IESG Note will include this  
information within the Status database, which changes as needed.   
Again, the SRD draft does not attempt to change the standards  
process, but it does open new avenues for conveying IETF products.   
As example, their could be a version/interoperability section within  
the Status database.
Each Status partition could be used as a signal for different events.

-Doug
.

---

From: John C Klensin <[email-masked]>
Date: Thu, 04 Aug 2005 10:01:14 -0400

--On Thursday, August 04, 2005 09:42 +0300 Pekka Savola 
<[email-masked]> wrote:

> (This might have been appropriate to the IETF list but decided
> to go for a smaller scope this time.)
>
> Bert and Sam's comments on the plenary made me think of the
> process change priorities a bit.
>...
> It seems that we should be better focusing on the more
> pressing problems (which are out of scope of this WG):

While we might disagree on details, this is in essence the 
reason I decided, after seeing the initial IESG reactions to the 
ISD proposal, that proposal was no longer of interest, at least 
temporarily.    A debate over, e.g., the differences between 
ISDs and SRDs is largely irrelevant while, as the saying goes, 
Rome burns.   It seems to me now that the critical-path issue is 
to either significantly reduce IESG workload so that the process 
issues can be considered or to get consideration and evaluation 
of process changes out of the hands (and queue/ bottleneck of 
the IESG).

I would, however, caution against doing anything significant 
that changes or eliminates the multistage standards process 
without careful consideration of implications, the real meaning 
of interoperability conditions, tightly-bound access to those 
conditions, etc., lest with throw the entire IETF away with the 
proverbial bathwater.

     john

.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Thu, 04 Aug 2005 15:35:01 +0200

> example ISD could appear in the implementation framework

It would take me about one month to write an implementation framework 
document for MIXER, I think. There are LOTS of features in MIXER.
The sample ISD (despite its name, it was never a draft) took me an hour.

                     Harald

.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Thu, 04 Aug 2005 15:33:42 +0200

--On 4. august 2005 14:34 +0300 Pekka Savola <[email-masked]> wrote:

> As a first observation, Bill Fenner's scripts could allow automating the
> populating the lists for references.  That is, we could produce HTML
> pages today which list all the docs (whether RFCs or drafts) which refer
> to <RFC X>, for {all X}.  It can't classify them reliably, though for
> later docs, norm/informative split could be automatic, and it could look
> up the category/maturity of the document from a database.

An automatic classifier would group all documents that use ABNF into a 
single cluster. And it's a normative reference too.

If we want clustering to happen, it HAS to be done by hand. Tools can help.

.

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 09 Aug 2005 16:41:22 +0200

John C Klensin wrote:
> While we might disagree on details, this is in essence the reason I 
> decided, after seeing the initial IESG reactions to the ISD proposal, 
> that proposal was no longer of interest, at least temporarily.    A 
> debate over, e.g., the differences between ISDs and SRDs is largely 
> irrelevant while, as the saying goes, Rome burns.   It seems to me now 
> that the critical-path issue is to either significantly reduce IESG 
> workload so that the process issues can be considered or to get 
> consideration and evaluation of process changes out of the hands (and 
> queue/ bottleneck of the IESG).
> 
> I would, however, caution against doing anything significant that 
> changes or eliminates the multistage standards process without careful 
> consideration of implications, the real meaning of interoperability 
> conditions, tightly-bound access to those conditions, etc., lest with 
> throw the entire IETF away with the proverbial bathwater.

I'm forming the view that I need to propose a roadmap for a consistent
set of process documents. That may seem like a bureaucratic approach,
but that isn't quite the intent - as Pekka says, we need to prioritize
the order in which we change the engines, but as John says the various
engines all need to end up pointing in the same direction. So an
overview of where we want to go is in order.

Not all of this, in fact only a small part, is in newtrk's charter.

     Brian

.