Subject: Re: Last Call: <draft-yevstifeyev-ion-report-06.txt> (Report on the Experiment with IETF Operational Notes (IONs)) to Informational RFC

From: Mykyta Yevstifeyev <[email-masked]>
Date: Tue, 12 Jul 2011 19:50:16 +0300

Hello,

As Russ agreed to sponsor this document on I-D submission cut-off date, 
I did make only some minor changed proposed by him during AD 
evaluation.  However I thought the document would be incomplete without 
analysis, so after LC I propose to add the following sub-section in the 
draft:

> 3.4. Analysis and Results
>
>    IONs were intended to serve as a means to document issues related to
>    procedures used by IETF or other parties, but to be more stable as a
>    simple web page and to have a more lightweight procedures for
>    approval than Best Current Practice (BCP) or other sort of RFC.  Even
>    though such middle-ground approach might be quite useful, it also
>    brings a number of complexities and negative effects, which are
>    described below.
>
>    First of all, IONs were mainly scoped to IETF procedural questions.
>    A number of IONs were published defining procedures used by IETF
>    community, such as ion-ad-sponsoring.  However, it should be noted
>    that the formal procedure of IONs approval, laid out in RFC 4693
>    [RFC4693] did not imply community involvement, unlike one for BCP or
>    other IETF Stream RFC.  Even though RFC 4693 intended IONs to cover
>    issues not sufficient for documenting in BCP, this regulation was
>    often overlooked.  This might have resulted in community non-
>    acceptance of such procedures, partial or full, if IONs were adopted
>    on the persistent basis.
>
>    Moreover, as IONs were lower in the hierarchy of IETF documents that
>    RFCs, published RFCs may override what mentioned in a particular ION
>    (whereas a published RFC may change already established procedures),
>    which might result in them not being sufficiently followed, creating
>    documentation conflicts.
>
>    Several IONS were published that describe the procedures used by IESG
>    or its members internally, such as ion-discuss-criteria or ion-tsv-
>    alt-cc.  Such material is obviously more appropriate for publication
>    as IESG Statements, which are also meant to be quite stable when
>    published and are approved at IESG's discretion.
>
>    A number of IONs were published covering different IAOC issues.  Such
>    IONs included ion-execd-tasks and ion-subpoena.  However, even though
>    IAOC works tightly with IETF, they have an ability to publish such
>    material on their site - <http://iaoc.ietf.org/>.
>
>    A one ION - ion-procdocs - was a reference guide to the IWTF Process
>    documents.  An other ION - ion-2026-practice - provided the criticism
>       and operational experience on RFC 2026 [RFC2026].  Both this
>    documents are fine as web pages, since the material contained in it
>    might change quickly and often.
>
>    ion-ion-format and ion-ion-store were published for the purpose of
>    the IONs series itself and were discarded upon experiment closure.
>    They are not analyzed here.
>
>    The aforementioned facts claim that IONs were less useful than the
>    equivalent information published in other way, and should be
>    abandoned, as proposed by Section 4 of RFC 4693 [RFC4693].

In order not to request the 2nd LC after this text is included, I'd like 
to seek community feedback on it during this Last Call.

Thanks,
Mykyta Yevstifeyev

12.07.2011 17:39, The IESG wrote:
> The IESG has received a request from an individual submitter to consider
> the following document:
> - 'Report on the Experiment with IETF Operational Notes (IONs)'
>    <draft-yevstifeyev-ion-report-06.txt>  as an Informational RFC
>
> [ . . . ]

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 13 Jul 2011 10:20:14 +1200

Mykyta,

I think the draft is fine without this addition, which contains some statements
that I disagree with. I don't think analysis is needed; this all ancient history
anyway.

Regards
   Brian

On 2011-07-13 04:50, Mykyta Yevstifeyev wrote:
> _______________________________________________
> Ietf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf
>

---

From: Russ Housley <[email-masked]>
Date: Tue, 12 Jul 2011 19:44:32 -0400

+1

I also disagree with some of the statements in the proposed text.  I do not think it is worth the effort to come up with consensus text.

Russ

---

From: Mykyta Yevstifeyev <[email-masked]>
Date: Wed, 13 Jul 2011 05:46:32 +0300

> that I disagree with. I don't think analysis is needed; this all ancient history
Brian,

Let me explain why I'm planning to include this sub-section.  The 
current draft, after giving background, lists all IONs which were 
published.  Then it notifies the reader about de-facto closure of the 
experiment by IESG and provides information on what was done with IONs.  
Finally, there is a conclusion, which states that IONs were a bad idea.  
Such conclusion needs some basis, and will be unjustified without any 
analysis of why were IONs a bad idea.

If you have some objections/proposals, please feel free to express 
them.  I think we'll be able to find some consensus.

Mykyta Yevstifeyev

---

From: Barry Leiba <[email-masked]>
Date: Wed, 13 Jul 2011 10:48:47 -0400

> Let me explain why I'm planning to include this sub-section.

Why?  Your explanation lacks substance, and further effort here is a
waste of time.

The document as it stands is just fine, except for one sentence that
needs rewording to make sense in English:

OLD
All IONs were republished whether as IESG statements, Web Pages, or
were discarded.

NEW
All IONs were republished as IESG statements, republished as Web Pages,
or discarded.

The document says everything it needs to say.  Let's publish it as it
is, and not spend more time trying to argue or explain anything.

Barry

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 13 Jul 2011 08:52:10 -0600

+1

---

From: Mykyta Yevstifeyev <[email-masked]>
Date: Wed, 13 Jul 2011 20:00:20 +0300

> All IONs were republished as IESG statements, republished as Web Pages,
I 'll make this correction.
> is, and not spend more time trying to argue or explain anything.
Based on the feedback received, I won't include this subsection in the 
draft.

Mykyta

---

From: John C Klensin <[email-masked]>
Date: Sat, 16 Jul 2011 10:39:23 -0400

--On Wednesday, July 13, 2011 10:48 -0400 Barry Leiba
<[email-masked]> wrote:

> Why?  Your explanation lacks substance, and further effort
> here is a waste of time.
> 
> The document as it stands is just fine, except for one
> sentence that needs rewording to make sense in English:
>...

Barry,

I note that we rarely publish analyses of experiments (either
process or technical ones).  I think we should do that a lot
more often but, if there is energy to do so, this wouldn't be my
first priority... or even my 20th.

If IESG statements are adequate as replacements for IONs, then
an IESG statement should be sufficient to terminate that
particular experiment.   And we already have such a statement.
While I could see publishing an RFC that, as Harald suggests,
reflects on the original analysis and explains why either the
problem was inappropriately described or unimportant or why the
solutions weren't worth the trouble, a document that says,
approximately, "we decided to not do that" adds no value to the
IESG Statement and no value to the RFC Series.

Presumably, when the IESG decided to terminate the ION
experiment by an IESG Statement, they made an explicit decision
that an RFC was not needed and that a detailed explanation and
analysis was not worth the trouble.   What do you think has
changed?

So, if only to increase my understanding, why do you think
reviewing this type of document (presumably through multiple
cycles as we quibble about language, etc.) is worth a Last Call
and the community's time?

    john

---

From: John C Klensin <[email-masked]>
Date: Sat, 16 Jul 2011 10:57:56 -0400

--On Wednesday, July 13, 2011 10:20 +1200 Brian E Carpenter
<[email-masked]> wrote:

> Mykyta,
> 
> I think the draft is fine without this addition, which
> contains some statements that I disagree with. I don't think
> analysis is needed; this is all ancient history anyway.

Brian and others,

I think your note, when combined with this document, identifies
a much broader issue that I think we need to start considering
more carefully.

The number of people in the community who actually do work and
who, in particular, feel obligated to review every Last Call
about which they might have knowledge, to make constructive
comments, etc., is not very large.  By definition, they have
finite time.  Relatively few of us have the luxury historically
accorded academics (students and faculty alike) of being able to
set priorities independent of intrusive external complaints
(many academics don't have that luxury any more either).

If we want the specifications and documents that really do need
careful attention, review from different perspectives, etc., to
get that time, then, IMO, we have to start becoming more
disciplined about identifying what is really important...
especially in times when available resources seem to be
shrinking rather than increasing.  Our questions need to include
not only "is this correct" and "does it address an issue that is
either known or that it identifies" but "is it actually
important".

I would hope that much of that discipline could be applied by
authors exercising judgment and maturity.  However, if it is
not, I would hope that IESG members considering whether or not
to sponsor individual contributions would assume that an hour
that goes into reviewing one document (even one that is
unimportant in the grand scheme of things) is an hour that isn't
going to go into reviewing another one, especially one that
might, through the week of the draw, be announced right after
someone has used up all of her available IETF time for a while.  

So, my conclusion about this particular document is ultimately
identical to yours: this is all ancient history anyway (and, by
the way, the IESG has already issued a statement on the
subject).  But that conclusion leads me to disagree with you:
"ancient history" doesn't imply that we should publish the
document, it implies that we should stop wasting time and drop
it.

When I express agreement with Harald's call for more analysis
that might lead us to avoid similar attempts in the future (if,
indeed, IONs were a mistake), it is not because I think that
work would be worth publishing.  I would be much more likely to
still come down on the side of "ancient history; let's not waste
time having a Last Call, discussing it, and publishing".   But
at least in Harald's model we'd be publishing some serious
analysis and new information that would add to the record.  I
don't see this document, freed of all or most analysis, adding a
thing to the existing IESG Statement.  And that makes it a
bigger use of community time, RFC Editing resources, etc.

In case it isn't clear, while this particular document inspired
this comment, the comment applies with equal force to, e.g.,
reclassifying RFCs that are even more ancient history and about
which no one is paying attention any more.  We can do it, the
reclassifications are usually correct technically and
procedurally, but processing them is justified only if we assume
that the community has infinite time and resources and/or that
time spent on something like those efforts isn't time taken away
from other things.

    john

---

From: Barry Leiba <[email-masked]>
Date: Sat, 16 Jul 2011 14:41:56 -0400

...
> and the community's time?

John, you've mis-addressed this.  It's Mykyta who wants to do it ,not
I.  I think it's at least as much a waste of time as you do.  The only
reason I reviewed it and commented on the language at all was that
Mykyta said Russ had agreed to shepherd it.  Either it needs to be
published as it is, without further waste of time, or it needs to go
away.

Barry

---

From: John C Klensin <[email-masked]>
Date: Sat, 16 Jul 2011 19:19:48 -0400

--On Saturday, July 16, 2011 14:41 -0400 Barry Leiba
<[email-masked]> wrote:

>> cycles as we quibble about language, etc.) is worth a Last
>> Call and the community's time?
> 
> John, you've mis-addressed this.  It's Mykyta who wants to do
> it ,not I.  I think it's at least as much a waste of time as
> you do.  The only reason I reviewed it and commented on the
> language at all was that Mykyta said Russ had agreed to
> shepherd it.  Either it needs to be published as it is,
> without further waste of time, or it needs to go away.

In case it isn't clear, I vote "go away" rather than wasting
even the amount of resources that would be required to publish
it substantially as is.  The community has to be able to say
"this isn't worth the energy" and establish consensus around
that.  

I suspect we agree about that; I'm just surprised and chagrined
that you, Harald, myself, and others are needing to spend energy
reading and understanding this and similar things before we can
reach that conclusion.

   john