Subject: [newtrk] Status of this memo section

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 30 Mar 2004 06:38:13 -0800

Reading the thread on WGS, I run across a common theme:

A WG needs to tell the world what it thinks the status of a document is - 
what has consensus, what's worth reviewing, what's been reviewed, what's 
left to do.

The WGS mechanism has been proposed as one way of setting a "stake in the 
ground" - but we have no consensus on what the "stake" signifies, or when 
it should be set.

So this note proposes another mechanism for communication.
Let's talk.

MECHANISM:

In an Internet-Draft, just after the Abstract, and just before the 
Introduction, create a section called "STATUS OF THIS MEMO".
This section will always begin with the following words:

   This section will be removed before RFC publication.

The rest of the section is a set of <component> <change> <status> tuples - 
identifying a component of the document, its "change status" since the last 
version, and the status as percieved by the document editor at the time of 
drafting.

<component> may be one of the following:
This draft
Section <x.y>

<change> may be one of the following:
is unchanged
has had minor modification
has been significantly modified
is new

<status> may be one of the following:
has WG consensus, has had external review, and is thought stable
has WG consensus, and needs external review
has known deficencies, and will change
is being proposed for evaluation by the WG
doesn't matter

So such a section may say:

STATUS OF THIS MEMO

This section will be removed before RFC publication.

This draft has been significantly modified.
Section 1 to 3 are unchanged, have had WG consensus, have had external 
review, and are thought stable.
Section 4 has had minor modification, has WG consensus, and needs external 
review.
Section 5 has been significantly modified, has known deficencies, and will 
change.
Section 6 is new, and is being proposed for evaluation by the WG.
Section 7 is unchanged, and doesn't matter.

This would augument, but would not replace, the "changes since version -xx" 
sections that are commonly found (and recommended!) in I-Ds today.

And nobody but the I-D editors need to do anything before starting to 
experiment with it.

Offered as a thought only - and submitted under the IPR rules of RFC 3667 
for use within the standards process, so anyone who wants can take the idea 
and spin it into a draft if they feel like it.

Thoughts?

.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 30 Mar 2004 10:12:11 -0700 (MST)

On Tue, 30 Mar 2004, Harald Tveit Alvestrand wrote:

> A WG needs to tell the world what it thinks the status of a document
> is - what has consensus, what's worth reviewing, what's been
> reviewed, what's left to do.

Agreed.

> Introduction, create a section called "STATUS OF THIS MEMO".

Please note that most specs (all IDs and RFCs generated with default
xml2rfc settings?)  already have a "Status of this Memo" section. We
can extend that or have a "Component States" subsection (or somesuch).

>    This section will be removed before RFC publication.

Note that RFCs keep their "Status of this Memo" section.

> The rest of the section is a set of <component> <change> <status>
> tuples - identifying a component of the document, its "change
> status" since the last version, and the status as percieved by the
> document editor at the time of drafting.

I like the intent but I believe this specific approach is too rigid
and costly. Specifically,

> may be one of the following:

What about things like "message syntax", "2nd bit semantics", or
"proxy requirements"? Those things may not be confined to specific
section(s) or their sections may have other things with different
state/status. Yet, they may be stand-alone items of paramount
importance.

I would suggest allowing non-standard descriptions for <component>.
Did you use "may" in a MAY sense?

> has been significantly modified

IMO, maintaining an accurate (i.e., true) state using these change
tokens is too much overhead given the lack of change/issue maintenance
tools for IETF community. Also, a "minor" change may have
"significant" security implications, unknown to the draft authors. We
already have a set of diff tools to produce usable differences between
drafts. Let's provide and improve those instead.

I would suggest removing this formalism.

> is being proposed for evaluation by the WG

I agree that the above formalism is REQUIRED. We should polish the set
of standard status/state terms and allow but discourage non-standard
terms (a SHOULD).

"State" may be a more appropriate term, especially since "status" is
already used with a somewhat different semantics for Status of this
Memo sections.

We should also say what the default status is or require that the
default is defined by the WG. That is, it should be known what the
states of all undocumented components are.

HTH,

Alex.

.

---

From: "James M. Polk" <[email-masked]>
Date: Tue, 30 Mar 2004 15:10:20 -0600

Harald

Silly question or two:

What if the WG doesn't agree with an author's assessment of Status?

Specifically, what if the WG (or a subset of the members of a WG) don't 
believe they agree with an author's claim of WG consensus for a section, 
sections or ID?

There doesn't seem to be an appropriate amount of time (or a mechanism 
other than to force WG chairs to monitor and read/verify each ID - which is 
asking too much of them IMO) to change the text before publication, 
especially if a deadline just past. Yet other readers will not know the 
difference and read these statements as if true - and perhaps not voice an 
opinion or question to the contrary because of the general rule of not 
rehashing covered topics.

Although I agree in general with what you want, I believe it adds 
significant "process" to a process that's already overburdened.

At 06:38 AM 3/30/2004 -0800, Harald Tveit Alvestrand wrote:
>identifying a component of the document, its "change status" since the 
>last version, and the status as percieved by the document editor at the 
>time of drafting.
>
>This would augument, but would not replace, the "changes since version 
>-xx" sections that are commonly found (and recommended!) in I-Ds today.
>
>for use within the standards process, so anyone who wants can take the 
>idea and spin it into a draft if they feel like it.
>
>Thoughts?
>
>
>.

cheers,
James

                                *******************
                 Truth is not to be argued... it is to be presented

.

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 30 Mar 2004 14:47:41 -0700 (MST)

> What if the WG doesn't agree with an author's assessment of Status?

IMO, the section should not be the author' assessment! It should be a
direct recording of the WG consensus, as gauged by the Chair, using
normal rules. If an author makes a mistake, the usual rules apply. If
there is no consensus, there is no real content in the section.

> Specifically, what if the WG (or a subset of the members of a WG)
> don't believe they agree with an author's claim of WG consensus for
> a section, sections or ID?

It is the chair that gauges consensus, not the author. The authors
should be explicitly forbidden to publish real content in this section
without WG approval (i.e., without WG consensus as gauged by the Chair
using normal rules, including appeals, etc.).

> There doesn't seem to be an appropriate amount of time (or a
> mechanism other than to force WG chairs to monitor and read/verify
> each ID - which is asking too much of them IMO) to change the text
> before publication, especially if a deadline just past.
>
> Yet other readers will not know the difference and read these
> statements as if true - and perhaps not voice an opinion or question
> to the contrary because of the general rule of not rehashing covered
> topics.

That's a very good point! By default, the section will be published
with a standard "this draft contains no stable components, is not
ready for implementation, and the WG does not explicitly solicit any
reviews" content.

Furthermore, we can probably add some simple rule (e.g., explicitly
include a draft version number if real content is added) that will
allow for checks whether the section has been brough in sync since the
last publication. A standard boilerplate with a place to insert
then-current version number will help.

Some stale information is unavoidable, but we can probably design the
rules so that it is highly unlikely that stale info is not detected,
and, fortunately, stale information in a few cases is highly unlikely
to cause serious long-term damage.

As for overheads, please note that this section will have real content
only when WG makes an important decision or reaches a major milestone.
Those events do not happen often, and 99% of the overheads will not be
in writing the section up.

Alex.
.

---

From: "James M. Polk" <[email-masked]>
Date: Tue, 30 Mar 2004 18:02:33 -0600

At 02:47 PM 3/30/2004 -0700, Alex Rousskov wrote:
>IMO, the section should not be the author' assessment!

Ok - I'm going with you here. Though I think you have not addressed my 
comment about an author submitting an ID on the deadline and including some 
text in this new section that the Chairs, the WG  and/or some member of the 
WG don't agree with *before* it gets published to ID-Announce.

>Some stale information is unavoidable,

I'm actually not worried about Stale information. I'm worried about 
too-optimistic inaccurate information about when something has reached 
consensus.

>only when WG makes an important decision or reaches a major milestone.

Ok, so I can't help but remember the 2nd PANA meeting (a lot of people 
still remember that session and still talk about it) in which one of the 
chairs (I don't remember which) got up and presented his requirements ID to 
the WG. The lines at the (3 I believe) microphones were 4-5 deep with 
complaints throughout the preso, some quite specific and I thought quite 
valid. The presenter kept saying over and over again: "you obviously don't 
understand what I'm saying"; yet this person stated at the end of the preso 
that he feels the WG believes this effort is ready for WGLC (actually had 
it on the last slide). The room erupted and many literally walked out of 
the room. I've been told since that one of the ADs was in the room and I 
didn't see him speak out either way.

I'm not evaluating the AD here, just pointing out a situation that didn't 
happen too long ago in which the chairs obviously didn't have a sense of 
the room (or their WG) and had their own agenda to move a document/effort 
forward. This type of possibility should not happen in the future - but I 
believe what was originally suggested here could promote that outcome in 
many more future efforts (IDs and WGs in general).

>in writing the section up.

No they won't - but the overhead will be in the chairs reading all of a 
document (each WG document) to know which sections have not changed, which 
ones did, which sections have not reached WG consensus and which ones have 
reached WG consensus; each of which was suggested.

I see much more work being proposed here.

cheers,
James

                                *******************

.

---

From: Alex Rousskov <[email-masked]>
Date: Tue, 30 Mar 2004 17:32:29 -0700 (MST)

On Tue, 30 Mar 2004, James M. Polk wrote:

> Ok - I'm going with you here. Though I think you have not addressed
> my comment about an author submitting an ID on the deadline and
> including some text in this new section that the Chairs, the WG
> and/or some member of the WG don't agree with *before* it gets
> published to ID-Announce.

True, I have not. The case of rogue or too-optimistic authors is
mostly out of scope -- the problem does exist, but is applicable to
publication of any WG document. As an author, I can publish _any_ crap
starting with version 01 of the draft and nobody will catch me until
that crap is published. I am sure that someday, somebody will publish
something really bad/illegal/whatever this way, but addressing core
IETF authentication and WG synchronization problems is out of scope
here.

> too-optimistic inaccurate information about when something has
> reached consensus.

Yes, that's a subcase we cannot fully address. We can only require, in
plain English, that authors do not interpret consensus but only record
explicit WG decisions (as judged by WG Chair, etc.). We could ask WG
chairs to double check, but I do not think it is worth the trouble
because it will create extra synchronization work in 99% of cases to
catch 1% of errors and not all errors will be caught anyway.

Furthermore, since Chair's decisions are appealable, it is possible
that the author published the right thing at the time, but it is no
longer valid! A smart reader will check more recent drafts (and the
section boiler plate can encourage that) for updates. Web publishing
system can help with that. But we can never catch all errors _and_
have a near-real-time publication system.

Having fast, low-overhead publication system, IMO, is more important
than catching additional errors, because the overall volume of errors
should be low and because these kind of errors will usually be
non-fatal.

> >As for overheads, please note that this section will have real
> >content only when WG makes an important decision or reaches a major
> >milestone.
>
> Ok, so I can't help but remember the 2nd PANA meeting
...
> on the last slide). The room erupted and many literally walked out
> of the room.

This problem is real, but out of scope. Chair decisions are
appealable, but appeals are costly, and sometimes an appeal will be
too late to fix all side-effects of the wrong decision. We cannot
afford a 100% fail-proof system, IMO.

> >Those events do not happen often, and 99% of the overheads will not
> >be in writing the section up.
>
> No they won't - but the overhead will be in the chairs reading all
> of a document (each WG document) to know which sections have not
> changed, which ones did, which sections have not reached WG
> consensus and which ones have reached WG consensus; each of which
> was suggested.

I am against a system where the above would be required. It should be
sufficient to document current WG decision(s) regarding _this_ draft
parts (if any). Everything else will rely on reasonable defaults.
Change tracking is out of scope.

> I see much more work being proposed here.

Yes. I will try to draft a version without that non-essential work.

Alex.
.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 30 Mar 2004 17:16:19 -0800

> believe they agree with an author's claim of WG consensus for a section,

I'd say that that was an excellent argument that the chair should declare 
the position of editor open.... but this is no different than the case 
where the editor writes something different from what the WG had consensus 
on.

> other than to force WG chairs to monitor and read/verify each ID - which
> is asking too much of them IMO) to change the text before publication,
> difference and read these statements as if true - and perhaps not voice
> an opinion or question to the contrary because of the general rule of not

If they get it wrong - resubmit the I-D. Numbers are cheap, and we haven't 
seen a draft at -99 yet.

.