Subject: [newtrk] My rough notes from NEWTRK (please report errors!)

From: "Spencer Dawkins" <[email-masked]>
Date: Fri, 5 Aug 2005 11:46:55 +0200

New IETF Standards Track Discussion WG (newtrk)

Friday, August 5 at 0900-1130
=============================

CHAIR: Scott Bradner <[email-masked]>, attending via Jabber
GUEST CHAIR: Harald Alvestrand <[email-masked]>
Notes: Spencer Dawkins <[email-masked]>

reading list
- newtrk list archives
- http://darkwing.uoregon.edu/~llynch/newtrk/index.html
- draft-ietf-newtrk-repurposing-isd-03.txt
- draft-lear-newtrk-decruft-experiment-01.txt

AGENDA: The agenda was bashed...

- Brian - does "next steps" include drafts for a new standards track?

- Sam - SRD and ISD in one big bullet - issues and next steps in one big bullet - Larry's proposal on the list fits somewhere

- John - please don't conflate SRD and ISDs - they are different - and should we be discussing stuff that's not a draft?

- Harald - SRD is probably a comment on ISDs, so this could be OK

- Spencer - there are two basic issues that have been raised - who will write ISDs, and who will approve them. All other is details.

- We are likely to accept only one proposal of ISD and SRD

- Interest in Larry's proposal on implementation reports? Some, but tabled for now

- Interest in one-step/two-step? Brian - need to have interlocking proposals that make sense, not fragments

- John - happy to start one two-step, but need more drafts on one-steps, no opportunity to do drafts

- Scott - need to get metadocuments first

1 - introduction & status - chair

2 - discussion on the issues with ISD proposal raised by the IESG and others 

o What is an ISD?

- About half the room has read the ISD drafts (good work!)

- We are confused - both implementers and procurement types can't refer to what our "standards" really contain

- ISDs become the standards - RFCs are no longer standards

- as per plenary - very difficult to talk about concepts at the IETF - always too much detail or too little, never "just right"

- Brian (channeling IESG comments, as IESG chair) - proposal was discussed at the retreat, slides plus document

- Proposal doesn't explain what problem is being solved - several IESG members do think metadocuments is the right way to go

- SRD doesn't have the same problems - because it's not meeting the same requirements

- ... and then IESG went into BOF mode, and ...

- Ted - two problems mentioned by Spencer, plus who is going to read these documents? Our standards process is now actively in the way of our specifications process - how to iterate? This is really moving to a different model for standards, but not for specifications, and specifications are more important

- Leslie - like metadocuments or overviews, need to be clear about whether this is lightweight or heavyweight - reviewed? normative? Need to pick one answer

- Pete - have followed up with Ted after the retreat - don't even understand whether new specifications update older specifications very easily - who is the audience? Implementers? Procurement people? then ISDs are appropriate. Other SDOs? ISDs would not be appropriate for particular specifications - but this is not the goal anyway.

- Alex - like to step back from IESG perspective - people come here because they can get things done here. Process overhead is a price. The acceptable price is limited. Don't increase red tape. We have applicability statements today, and people aren't writing them - why?

- Spencer - should we be worrying about "standards" at all - IETF-approved specifiations, and be done?

- Dave - Ted making distinction between specifications and standards - ISD goal is actually tying pieces together. If we care about our stuff gets used, maybe we should help people figure out how to use them. We are never done. How does the outside world take, and work with, snapshots?

- Sam - this isn't the biggest problem we've got, but ... I am a proponent of metadocuments, but not ISDs. Would like to discuss concerns I have with ISDs - too much text, updates would be contentious, concerned about ISDs as references in some cases - inter-document references to RFCs, not to ISDs, concerned about errata in the ISD, especially if it's not consensus-driven, and concerned about whether metadocuments are needed all the time.

- Ted - believe ISDs would be harder for implementers - the way this is put together, including the errata and commentary, really makes it harder. In really complex cases, we still need RFCs anyway.

- Doug - database isn't the same weight as an RFC, but it would be useful to have a central location for a set of documents. SRD tries to limit itself to half a page. Concerned that ISDs will get out of sync if there's a heavyweight review.

- Aaron - Scott is participating in Jabber discussion - need to include him in verbal discussion - Brian will channel Scott - folklore should be written in RFCs if it's important.

- Carosco - de jure or de facto doesn't matter for single specifications, but compound documents do matter, and it doesn't matter whether they have the same name or different names. It would be nice if the IETF does have compound documents.

o Doug Otis - what is an SRD? 

- Flat system is difficult to deal with... Trying to start with ISDs and stay in sync with standardization process

- IESG has to take a look at SRDs, but SRDs should help their process, not slow them down.

- Should be a status database for issues tied to documents

- Not authoritative information - stuff that hangs around in SRDs should become an RFC.

- Want stable references for IETF products

- Link to database with partitions (IESG notes, IETF notes, security alerts, ...)

- Larry - lot of stuff besides SRDs in the database - where does it come from?

- John - SRDs looked very lightweight at the beginning, but are gaining weight - what's the current difference from ISDs? WG approval, IESG approval... did we converge?

- Alex - STD groupings not used today - when you refer to a spec, you refer to a specific revision. This isn't useful. There are problems with the standards track, but this isn't addressing any of the problems I'm hearing about. 

- Doug - SRDs do have serial numbers...

- Sam - while I don't like ISDs, SRDs provide an interesting starting point for compound documents. I like "very little text" within them. No descriptive text - that's the critical feature. Automated tools are OK, anything more is not helping. Handles the grouping problem, separates out other problems. Going to have a nice long debate about who updates text in the database. SRDs need not cover all documents, especially at first. Don't make them mandatory - run an experiment.

- Scott - ISD could use standards action text.

- Spencer - question Harald's statement that we have to pick between ISDs and SRDs - we COULD do both, if the same people don't have to write both and read both - can't go back to nonexistent working groups, so what then? Can't extrapolate from STDs, because we have so few - nothing advances.

- Larry - it all boils down to what we get and who does the work.

- Aaron - does this work without status database? SRDs is lightweight, database is not.

- David Kessens - very worried about what I'm seeing - we're adding complexity and work, doesn't matter about the details.

- Dave Crocker - concern for the effort to create these things, but more concern about how they will be used. We don't spend nearly enough time on this.

- Harald - who believes there are proposals that are useful? more than believe there is no such proposal in the room, but the people who believe "is no such proposal" also believe "cannot be such a proposal".

- Pekka - generating something automatically has hope. Anything more does not. Consensus will kill this.

- Harald - who believes ISDs are viable? Document is full of options - John doesn't know how to answer this question! 

- Brian - assume ISDs are part of the standards process? 

- Larry - can we have the discussion that was on the agenda before we start asking questions? Have talked to people at his company who read RFCs - key thing is what do we have to do interoperate with the greatest number of implementations?

- Sam - would be happy with a mechanism that does not complicate the standards process except when using it adds value. Don't want to get sucked into discussion of grouping text, don't want to have to group things when no grouping is required.

- Pete - would be happy with grouping documents that go through the standards process, as long as they aren't required? Sam - yes.

- John - ISDs and SRDs are converging, except whether one is normative - <but the room was very confused about which of the alternatives was normative - apparently both would be approved> - we're headed off to a dark place

- David Kessens - are you sure there is consensus that ISD/SRDs should exist?

- Alex - run an experiment? 

- Dave - suggestion that product manager is a consumer was good - imagining useful product data sheets that used BCPs? helping both both product managers and consumers.

- Sam - this is a valid use of BCP under RFC 2026, and we have running code in OPSEC. Not QUITE the same, but overlap. Support experiments in this space, don't forget success criteria, don't forget incentive to participate in the experiment, need tools support to succeed and avoid too much work.

- Carosco - name doesn't matter, want to specify what a tender must contain. People know RFCs are standards ("oops!"). Grouping a whole set of documents saves one or two pages in a 200-page tender.

- Bob Braden - we do a great job of turning out specifications and a horrible job of modularity. RFC database contents are really ugly. I feel really sorry for implementers! I understand concern about additional work, but do see additional benefits.

3 - future of ISD proposal

-  SHOULD A TYPE OF GROUPING DOCUMENT EXIST? 25 out of 40 yes, 4 no

- SHOULD THEY BE MANDATORY? <in some cases?> run an experiment? room sense is "no" - use the PROTOS model?

- SHOULD GROUPING DOCS BE APPROVED? depends on the text they contain? - approved by the working group? - show the IESG a couple of examples? About half the room was willing to say "yes, approve them", but this also includes question of normative text in these documents, and what "approval" means.

- SHOULD THEY BE ABLE TO CONTAIN NON-AUTO-EXTRACTED/AUTO-GENERATED TEXT? Sense of room is that a lot of people think "yes"

- SHOULD THEY NORMALLY CONTAIN TEXT? No sense of the room. Get a concrete proposal and discuss on the list.

3.5 - implementation reports

- have seen documents generated by IETF participants in an informal way - HTTP, WEBDAV - formalize what we already do anyway

- get IESG out of business of approving interoperability reports they can't verify anyway

- Sam - experiments are useful, intersection with 2-step is interesting

- Spencer - is additional work for the IESG still on the table? several IESG types - yes, that's OK - Sam, OK, otherwise we should all be recalled!

- Pekka - vendors don't want to spend time on implementation reports - we have, like, two reports on BGP4.

- Bill - IESG has problems evaluating implementation reports

- Larry - come to consensus on form of reports, don't approve reports?

- Bob - Identify WGs with specs that are excessively feature-rich?

We encourage Larry to move forward with a proposal

5 - next steps with de-cruft  - Eliot

- 56 documents still on the cruft list

- Four alternative ways forward - we need to pick only one.

- Bob - X.400 non-use? FDDI? Harald - No one sees a benefit to the community to leaving these specifications on the standards track.

- ACCEPT RESULTS? Strongest support in the room was for this alternative.

- DON'T ACCEPT RESULTS AND UPDATE 2026 to match what we do? No support in the room

- ACCEPT SOME RESULTS and still update 2026? Light support in the room

- DO NOTHING? Very light support in the room - Bert - I've tried this under Fred Baker, killing one doc was a lot of work, got lots of pushback, suspect that it's going to be harder than you think

- We are lying to people in 2026, but we are lying to them about so many other things...

- David Kessens - this isn't worth the work for the IESG.

- Pekka - it's mostly individual efforts.

- Sam - get the community to chill out about Historic

- Need to do working group last call on this draft. Will publish as working group draft.

4 - next steps in newtrk, how to discuss 1-step/2-step

- Brian - the goal of this group is to develop a revised standards track. Are we really going to do this?

- Larry - remove draft standard approval from IESG?

- Pekka - modifying that part of 2026 won't matter much, IESG doesn't really review implementation reports now, not really a bottleneck

- Spencer - need to keep implementation engineers engaged - two BOF cycles mean up to 11 months before you start on a specification - all the people that are left are standards professionals.

- Dave - if we're going to have more than one stage, we need to think about what they mean - being widely used is helpful to know

- Alex - IESG does review interoperability reports, especially in RTG, please do what the charter says and make the IETF a better place.

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 12 Aug 2005 10:56:16 +0200

Spencer Dawkins wrote:
...
> - Spencer - question Harald's statement that we have to pick between ISDs and SRDs - we COULD do both, if the same people don't have to write both and read both - can't go back to nonexistent worki…

Reading this prompted two thoughts

1. We could indeed do both if we distinguish

   ISDlite, more or less the same as SRDs, and treated as Informative

   ISDheavy, more or less the same as ISDs, and treated as Normative

The idea would be to use ISDlite in most cases, and only use ISDheavy
in relatively few complex cases where the effort is both available
and justified. Obviously, ISDheavy format would be a superset of
ISDlite format.

2. It also struck me that if we went this way, we could start tomorrow.

   A web site to host ISDlite documents "as is" could be set up easily
   (assuming their source form is XML and the active version is HTML)

   An ISDheavy can be processed here and now as a BCP - assuming its
   source form is XML2RFC, it can be approved and published as
   an RFC/BCP and posted in HTML on the same site as ISDlite.

      Brian

.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: John C Klensin <[email-masked]>
Date: Fri, 12 Aug 2005 09:16:11 -0400

Brian,

--On Friday, 12 August, 2005 10:56 +0200 Brian E Carpenter
<[email-masked]> wrote:

> Spencer Dawkins wrote:
> ...
>> 
>> - Spencer - question Harald's statement that we have to pick
>> between ISDs and SRDs - we COULD do both, if the same people
>> don't have to write both and read both - can't go back to
>> nonexistent working groups, so what then? Can't extrapolate
>> from STDs, because we have so few - nothing advances.
> 
> The idea would be to use ISDlite in most cases, and only use
> ISDheavy in relatively few complex cases where the effort is
> both available and justified. Obviously, ISDheavy format would
> be a superset of ISDlite format.

This strikes me as a seriously bad idea.  It does have at least
one advantage, which is that the existence of ISDheavy, if we
did a lot of them, would help to convince people that ISDlite
were not normative.

However...

(i) I think this turns ISDheavy into a conventional
applicability statement, using the usual procedures and document
format (see below).  Those have, of course, been wildly
successful and popular, at least based on the huge number of
them we have generated and continue to generate.

(ii) Unless very significant other changes are made, it gives up
the ability to publish such documents using a process separate
from that of the RFC Editor.  Whether the RFC Editor produces
documents within a month after IESG sign-off or takes a year to
do so, it is not, IMO, desirable to have the documents in the
same queue.

(iii) By virtue of publication as a BCP RFC, ISDheavy documents
would lose the ability to be treated as living documents with
stable snapshots.  The capability is critical as a way to
identify specific problems that have been observed with
particular protocol specifications at a time and under
circumstances in which such notifications would do anyone any
good.   But maybe the community does not care about that one.

(iv) The grouping decisions, which will be treated as normative
if they have IETF's name attached to them in any way, will still
be associated with ISDlite, which makes them either a danger or
not-so-lite.

(v) It doesn't do anything to solve the piece of this problem
that is closest to newtrk's core agenda, i.e., moving us from a
three-step process with categories whose boundaries cannot be
good enough to do the job and that discourages things from
advancing past Proposed by encouraging a high entry barrier for
Proposed.  By permitting the use of explicit prose to
differentiate all of the points along the orthogonal spectrum
from, e.g., "looks like a good idea worth trying" and "the IETF
is convinced that this proposal is fully- or mostly-baked even
if the specification is not and has significant value" and from
"implementations exist" to "interoperability has been claimed
for multiple independent implementations and including all
stated features", ISDs-with-prose potentially permit real
differentiation in ways that do implementers and those deciding
whether to develop, market, deploy, or procure products some
good.   The present three-category model (consisting of
Proposed, Advanced Proposed, and Almost Never) doesn't do that
job, nor would either a "no prose beyond abstract" ISDlite/SRD
or anything else that sustains the same, or a small set of
other, categories that were used without qualification.

and...

>    an RFC/BCP and posted in HTML on the same site as ISDlite.

(vi) Based on the experiences Scott and I had trying to
construct examples, XML2RFC would need to be extended to include
references into numbered lists and, probably, references into
the text of other documents with live links.  
  <xref target="RFC9999">RFC 9999, Section 3.2.1</xref>
works, but is not really wonderful.  I think there were some
other issues as well, but I don't remember whether there were
any that didn't involve the issues of one document embedded in
another, which would not apply for a "real" document.

    john

.

---

From: Spencer Dawkins <[email-masked]>
Date: Fri, 12 Aug 2005 08:39:58 -0500

>good.   But maybe the community does not care about that one.

Please be patient, while I try to think through this...

One elephant in the living room I've been tip-toeing around is the 
"living document" part of this discussion.

Based only on the ISD samples I've seen to date, I REALLY like the idea 
of John Klensin saying, "this is SMTP", or Scott Bradner saying, "this 
is the IETF standards process" ...

... but each of these is one (smart and experienced) person's opinion ...

... so it seems like they should have at least minimal review ... 
(notice the life force draining)

... one reviewing body could be (a? the?) working group ...

... if the working group still existed ... (drain, drain)

... or an area review team ...

... if area review teams existed for all areas ... (drain, drain)

... or an AD could review changes to living text ...

... and if text for updated ISDs was forwarded together with the drafts 
that will be referenced in the updated ISDs, that could work ...

... but if we get to the "IESG review/IETF Last Call" place, all we have 
managed to do, as John points out, is shift applicability statements 
into XML, and that is all (game over)

I really think "living text" matters, but the "living text" needs to 
follow updates to the specifications that make up an ISD pretty closely, 
or it won't be living at all..

Where should we be coming down on this?

Spencer

.

---

From: Douglas Otis <[email-masked]>
Date: Fri, 12 Aug 2005 07:46:27 -0700

> is the IETF standards process" ...

I see the SRD as a companion document to the I-D explaining the related
endeavours affected by the I-D being submitted for IESG action.  Once
that companion review process is in place, these SRD documents should be
considered normative. Although the information in the SRD is brief, the
assertions made in the SRD should be verified.  This verification effort
should actually help with the I-D review process.

The SRD document was structured to be used by scripts that will then
generate the appropriate human readable forms, in HTML, RSS, or plain
text.  One important byproduct of this structural overlay of I-Ds and
RFCs, is a stable and enduring reference related to the endeavor, SMTP
for example.

Within the HTML version of the SRD converted outputs, there should be a
link to a database that includes the "living" informative information.
The information in the database could be broken down into three
partitions with three separate (lightweight) vetting processes.  

IESG, IETF, and External links.

This would be defined in a yet to be written Status database draft.

-Doug

.

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 12 Aug 2005 17:32:39 +0200

> them we have generated and continue to generate.

And indeed, every time I reread RFC 2026 section 3.2,
I do instantly think of the hundreds of such documents.

(For those who didn't work it out: John was being
justifiably sarcastic.)

But then the obvious question is: *why* have so few applicability
statements been written, and why would the definition of
ISDs cause them to be written?

> do so, it is not, IMO, desirable to have the documents in the

My answer is that we could stick the HTML version of an ISDheavy
on the web site immediately, and let the ASCII version appear
when it will.

> good.   But maybe the community does not care about that one.

Well, BCPs can be reissued. I'm envisaging a more modern structure
than we have today - HTML versions updated in place, and ASCII versions
with RFC numbers appearing in due course for archival.

> be associated with ISDlite, which makes them either a danger or

At least they will come from the IETF, whereas today's STD
groupings appear to come from the RFC Editor. But yes, this would
need to be fleshed out a bit for the lite model.

> other, categories that were used without qualification.

I agree; which is why I still think newtrk needs to decide
on a recommendation for the standards track.
> another, which would not apply for a "real" document.

Sure - but I think the xml2rfc crew could be talked into
useful extensions quite easily.

    Brian

.

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 12 Aug 2005 17:35:51 +0200

> Where should we be coming down on this?

How about

   lite = WG review, WG consensus, informative value
   heavy = IETF review, IESG approval, normative value

and an ISD could exist either as lite only, heavy only, or heavy
plus lite-in-progress?

      Brian

.

---

From: John C Klensin <[email-masked]>
Date: Fri, 12 Aug 2005 14:32:41 -0400

--On Friday, 12 August, 2005 17:32 +0200 Brian E Carpenter
<[email-masked]> wrote:

>> applicability statement, using the usual procedures and
>> document format (see below).  Those have, of course, been
>> wildly successful and popular, at least based on the huge
>> number of them we have generated and continue to generate.
> 
> But then the obvious question is: *why* have so few
> applicability statements been written, and why would the
> definition of ISDs cause them to be written?

IMO, so few have been written because it is a singularly
unrewarding activity.  Writing new specifications involving new
design work is lots more fun than the tidying and descriptive
work of AS-writing.  Anyone who attempts one gets a good deal of
abuse no matter what choices are made (or not made) and
virtually no positive reinforcement.   To the extent to which
"just documenting" things gets little respect, ASs tend to get
even less.    The approval process for them requires all of the
hoop-jumping required for technical specifications plus the
extra trouble associated with actually making recommendations
about uses.  And those aren't the only reasons.

I have more hope for ISDs precisely because they are intended to
be lighter-weight pieces, with incremental approval and the
concept that changing one section doesn't automatically open the
whole thing up to review, rehashing of old issues, and so on.
Whether it would work out that way would depend ultimately on
how the IESG chose to use them.  And that, in turn, is why I
have suggested that they are dead unless either the IESG is
sufficiently enthused about the idea to consider them a tool to
be supported and used or the community is willing to replace the
IESG with people who will feel that way.  Even if I thought the
latter was desirable, I don't see nearly the level of consensus
needed to accomplish it.

As I tried to say in Paris, I see the SRD (or ISDlite) concepts
as being useful.  What scares me is that, as they incrementally
converge on the ur-ISD concept, we may find ourselves backing
into a state that, I believe, should be gotten into only with
our eyes open.  Notions about things being claimed to be
non-normative while carrying most of the relevant
characteristics of normative specifications are, to me, just one
example of the "backing in" process.

>> (ii) Unless very significant other changes are made, it gives
>> up the ability to publish such documents using a process
>> separate from that of the RFC Editor.  Whether the RFC Editor
>> produces documents within a month after IESG sign-off or
>> takes a year to do so, it is not, IMO, desirable to have the
>> documents in the same queue.
> 

That would work if we adopted very strict rules about change
control, URL (or other URI) formats, and clearly understood the
implications of an ISDheavy document changing two or three times
between when the first version in a set what submitted for
publication and when something was actually published.  All of
those conditions are feasible: we just need to understand the
reasons and requirements for stable documents whose status at a
given time in the past is easy to determine, and then act
accordingly.

>> (iii) By virtue of publication as a BCP RFC, ISDheavy
>> documents would lose the ability to be treated as living
>> documents with stable snapshots.  The capability is critical
>> as a way to identify specific problems that have been
>> observed with particular protocol specifications at a time
>> and under circumstances in which such notifications would do
>> anyone any good.   But maybe the community does not care
>> about that one.
> 

See above.  I believe that the requirements for stability and
reference-ability of a particular version puts very significant
constraints on "updated in place".   That is not to say "can't
be done", only that this is an area where handwaving is
insufficient.

>> (iv) The grouping decisions, which will be treated as
>> normative if they have IETF's name attached to them in any
>> way, will still be associated with ISDlite, which makes them
>> either a danger or not-so-lite.
> 
> groupings appear to come from the RFC Editor. But yes, this
> would need to be fleshed out a bit for the lite model.

Presumably, if the only problem to be solved was a numbered
grouping mechanism, and the IESG decided it wanted that
responsibility, responsibility for STD numbering (and for
deciding at what point an STD number is assigned) could be taken
over by some IETF entity rather easily.   

My own view is that the STD numbers exhibit the categorization
problem I discussed in an earlier note (and that many others
have described in other ways) -- setting the boundaries of what
is, or is not, part of a particular standard is ultimately a
little bit arbitrary and a great deal a matter of taste.  If one
can group, distinguish between normative groupings and
cross-references, not get hung up about overlapping groups,
etc., a great deal can be accomplished, IMO, without extended
battles about categorization.   If one needs to have groupings
like STD numbers provide, with no explanation, no cross
references as part of the group, and distinct groups, then
either the category turns out to be low-utility or we walk into
endless arguments about it.  

    john

.