Subject: Review of draft-yevstifeyev-ion-report-06

From: Harald Alvestrand <[email-masked]>
Date: Fri, 15 Jul 2011 17:18:53 +0200

My apologies for the lateness of this review.

I am not happy with this document.

I was unhappy with the IESG's decision to close the ION experiment, 
since I believe the mechanisms that were chosen to replace it failed to 
fulfil several of the requirements that were driving forces in the 
design of the ION mechanism (as an example, try to find out who, if 
anyone, approved http://iaoc.ietf.org/network_requirements.html, what 
the previous version was, and when this version was approved).

The document does not refer back to the aims of the experiment, which I 
tried to make explicit in section 5 of RFC 4693, which include:

- Easy updating
- Explicit approval
- Accessible history

The sum total of analysis in this document is two sentences:
The cited IESG statement

      It is clear that the IESG, IAB, and IAOC need the ability to
      publish documents that do not expire and are easily updated.
      Information published as web pages, including IESG Statements, are
      sufficient for this purpose.

The draft's statement

    Taking everything into account, it was considered that IONs added
    complications to the maintenance of documents but did not give a
    corresponding benefit to the IETF.

I would at least expect those three points to be explicitly addressed by 
analysis, such as:

- The IESG concluded that publication of IONs was more complex than 
publishing Web pages and IESG statements
- The IESG concluded that the IESG statement mechanism, which has no 
formal definition, was enough documentation of the IESG's decisions 
where decision documentation was reasonable, and that Web pages needed 
no explicit approval
- The IESG concluded that there was no need to provide an accessible 
history of versions of the documents for which the ION mechanism was 
intended

The document also needs a language check, but I feel that the lack of 
*any* explicit analysis with respect to the aims of the experiment, even 
an explicit statement that the issues involved were considered not 
important, is the most important shortcoming of the document.

                         Harald

---

From: SM <[email-masked]>
Date: Fri, 15 Jul 2011 10:19:43 -0700

Hi Harald,
At 08:18 15-07-2011, Harald Alvestrand wrote:
>The document does not refer back to the aims of the experiment, 
>which I tried to make explicit in section 5 of RFC 4693, which include:
>
>- Easy updating
>- Explicit approval
>- Accessible history
>
>I would at least expect those three points to be explicitly 
>addressed by analysis, such as:

I'll quote some text from Section 5 of RFC 4693:

   "The IETF is an open organization, which means (among other things)
    that there are always newcomers coming in to learn how to perform
    work; this places a requirement on the organization to document its
    processes and procedures in an accessible manner."

There is a steep learning curve for newcomers.  One of the ways for 
some people to learn is by reading the RFCs.  Sometimes these 
documents do not reflect current practice.  That is not obvious to 
newcomers who are not aware of the open secrets which seems to be 
common knowledge within the IETF.

There are times when an experiment fails.  When there isn't any 
documentation, one has to rely on those with "institutional" memory 
to pass down the history.

   "The IETF is also a large organization, which means that when
    procedures change, there are a number of people who will like to know
    of the change, to figure out what has changed, and possibly to
    protest or appeal the change if they disagree with it."

It can also be important to understand why it may not be feasible to 
adopt a procedure that has been tried previously.

Commenting on Section 3.2 of draft-yevstifeyev-ion-report-06, "URIs 
don't change: people change them".  The IETF makes an effort to 
preserve old URIs so that its web pages does not expire.  Sometimes a 
document falls through the cracks [1].

The ION experiment is a failure if only because the IETF has yet to 
come to terms with the notion of "lessons learnt".

Regards,
-sm

1. http://www.ietf.org/IESG/STATEMENTS/tsv-alt-cc.txt
    http://www.ietf.org/iesg/STATEMENTS/ad-sponsoring.html
    http://www.ietf.org/iesg/content/agenda-minutes-procedures.html

---

From: Mykyta Yevstifeyev <[email-masked]>
Date: Sat, 16 Jul 2011 07:12:11 +0300

Hello Harald,

As you could see in one of my previous messages, I did intend to include 
some analysis in the draft 
(http://www.ietf.org/mail-archive/web/ietf/current/msg67491.html).  
However, numerous responses which discouraged me from doing this were 
received, and including that section will create problems with 
consensus.  However, I share your opinion that the document won't be 
full, so I think the following analysis section, which is different from 
the previous proposed one, can satisfy you and everybody else:

> 3.4. Analysis
>
>    There were a number of reasons which forced IESG to close the IONs
>    experiment.  One of them is a complexity of their approval and
>    publication, compared with ones for IESG Statements and simple Web
>    Pages.  As IONs were intended to represent operational experience,
>    they might have needed to be updated quickly.  Even though these
>    documents were meant to have a very lightweight approval procedure,
>    it could sometimes be inappropriate for that material which was
>    contained therein.
>
>    Secondly, due to the nature of the scope of IONs, there was no need
>    to allow the access to the revision history (which is unavailable in
>    simple Web Pages as well).
>
>    Thirdly, IONs on procedural question could step into the conflict
>    with Best Current Practice (BCP) RFCs; moreover, as IONs approval
>    procedure did not imply any formal community review, unlike BCPs,
>    they could easily fall in the community non-acceptance.
>
>    Correspondingly, it was concluded that the mandate of IONs can fully
>    be fulfilled by RFCs, when documenting IETF procedures, IESG
>    Statements, when clearing up other issues for which RFC publication
>    process is not appropriate, and Web Pages, when dealing with other
>    IETF-related issues. 

Please see other response in-line.

15.07.2011 18:18, Harald Alvestrand wrote:
> since I believe the mechanisms that were chosen to replace it failed 
> to fulfil several of the requirements that were driving forces in the 
> The document does not refer back to the aims of the experiment, which 
> I tried to make explicit in section 5 of RFC 4693, which include:
>
> - Easy updating
> - Explicit approval
> - Accessible history
It's very easy to add.  I think the 2nd paragraph of the Section 2 may 
be changed to:

>    IONs were meant to be easily updated but to have some explicit IESG
>    approval mechanism; IESG might also have granted the approval right
>    to other parties.  The strict format rules for RFCs were not applied
>    to IONs.  Moreover, the history of revisions was intended to be
>    easily accessed.  For more background information on IONs, please see
>    Section 5 of RFC 4693 [RFC4693] .

> The sum total of analysis in this document is two sentences:
See above.
> [ . . . ]

---

From: John C Klensin <[email-masked]>
Date: Sat, 16 Jul 2011 10:21:06 -0400

FWIW, +1

--On Friday, July 15, 2011 17:18 +0200 Harald Alvestrand
<[email-masked]> wrote:

> I was unhappy with the IESG's decision to close the ION
> experiment, since I believe the mechanisms that were chosen to
> replace it failed to fulfil several of the requirements that
> were driving forces in the design of the ION mechanism (as an
> example, try to find out who, if anyone, approved
> http://iaoc.ietf.org/network_requirements.html, what the
> previous version was, and when this version was approved).
> 
> The document does not refer back to the aims of the
> experiment, which I tried to make explicit in section 5 of RFC
> 4693, which include:
> 
> - Easy updating
> - Explicit approval
> - Accessible history
> 
> - The IESG concluded that publication of IONs was more complex
> than publishing Web pages and IESG statements
> - The IESG concluded that the IESG statement mechanism, which
> has no formal definition, was enough documentation of the
> IESG's decisions where decision documentation was reasonable,
> and that Web pages needed no explicit approval
> - The IESG concluded that there was no need to provide an
> accessible history of versions of the documents for which the
> ION mechanism was intended
> 
> The document also needs a language check, but I feel that the
> lack of *any* explicit analysis with respect to the aims of
> the experiment, even an explicit statement that the issues
> involved were considered not important, is the most important
> shortcoming of the document.
> 
>                          Harald
> 
> 
> 
> 
> 
> 
> _______________________________________________
> Ietf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf

---

From: Harald Alvestrand <[email-masked]>
Date: Mon, 18 Jul 2011 10:30:25 +0200

On 07/16/11 06:12, Mykyta Yevstifeyev wrote:
> Hello Harald,
>
> As you could see in one of my previous messages, I did intend to 
> include some analysis in the draft 
> received, and including that section will create problems with consensus. 
I saw the messages. It is very hard to write an analysis on behalf of 
someone else - to my mind, the analysis of why the IESG decided what 
they did has to be based on information from the IESG; it's impossible 
for any outsider, including you and me, to divine what their thought 
processes on this matter were.

There's no IETF consensus to be documented here; the analysis and 
decision was done by the IESG, and the IESG (as it was composed at that 
time) is the body from which the information has to come. All any 
outsider (like you and me) can do is to help with the editing.
> However, I share your opinion that the document won't be full, so I 
> think the following analysis section, which is different from the 
> previous proposed one, can satisfy you and everybody else:
Now, if someone were to speak for the then-present IESG and say the same 
thing, I would make the following comments:
>>    experiment. 
Nit: "forced IESG" -> "caused the IESG to decide". There's no forcing 
function.
>> One of them is a complexity of their approval and
In a protocol, I'd ask for a definition of "quickly". Days, weeks or 
months? Examples of documents where "days" is the correct timescale?
>>    simple Web Pages as well).
I disagree with this statement, obviously. If the IESG thought so, it is 
correct to include it.
>>    they could easily fall in the community non-acceptance.
I believe this is a red herring. However, I can fully believe that the 
argument was raised, even though I don't believe it, so if the IESG 
indeed thought that, I'm fine with having that documented.
>>    process is not appropriate, and Web Pages, when dealing with other
Nit: "it was concluded" -> "the IESG concluded".