Subject: [Icar] Review types and associated issues

From: Alex Zinin <[email-masked]>
Date: Wed, 17 Dec 2003 12:46:24 -0800

Tuesday, December 16, 2003, 3:57:02 PM, Melinda Shore wrote:
> To be honest, I'm not completely clear on what the subject is, or
> at least not clear enough to be able to answer the questions you
> ask.  We've actually got several review problems.  One is early
> review and another is cross-area review (there are others, but
> I think these are the most pressing).

We might disagree on specific terms while agreeing on the substance,
but here's the dimensions along which the reviews are separated
within the IETF, I believe:

 1. Whom it is performed by and why:

    I think we have two types here:

    a) individuals, reviewing documents not because they have an
       obligation within the IETF structure to do so, but because
       they are interested in this. In other words, this is a
       good-will type of review, which is what I called
       "unstructured" or "peer-to-peer".

    b) ADs (with directorates), IESG reviewing documents because they
       are held responsible within the IETF to review the documents
       in a timely manner. This review happens as a result of a formal
       IETF standards procedure and there is a structured process
       around it. This is what I call "structured" review.

 2. When it is performed:

    a) Late--happening at the end of the life-cycle of the document,
       essentially what we have as the IESG review today.

    b) Early--happening before the ideas are crystallized or at least
       before the WG is finished with the doc. This is what we have
       happening in the WGs today.

 3. Technology coverage:

    a) Functional--covering the primary technology, without paying
       a lot of attention to affects on other pieces of the system
       or other aspects such as security. Unfortunately, this is
       often what happens within the WG.

    b) Cross-functional--covering a wide spectrum of technologies
       and keeping in mind their interactions. IESG review would
       be an example.
       
 So we have at least 3 dimensions here. Currently, the IETF review
 is focused around two spots:

   1. {unstructured, early, functional}
   2. {structured,   late,  x-func}.

 My thinking is that we need to improve both such that we have:

   1. {unstructured, early,        func | x-func}
   2. {structured,   early | late, x-func}

 in other words, we should encourage early cross-functional review by
 individuals, plus we should improve structured review so that
 requesting it early in the process scales well.
   
 I am open to other models, but this seems to reflect the reality as I
 see it today.

> Failures in these areas have
> overlapping but somewhat different results, in that early review
> (i.e. review against the charter, or review for quality within the
> charter) failures tend to allow the working group to produce bad results
> while cross-area review failures can lead to work that's incoherent
> in the broader context, work that's substantially the same as
> work in another area, and so on.
>
> Some of what's been discussed or experimented with already, like
> SIRS, has tended to focus on the early review problem, but I'm inclined
> to think that can be solved with a bigger hammer (making better use of 
> WG
> technical advisors, for example).

It seems that SIRS falls under {unstruct, early, x-func} category,
while use of TAs is a more structured process. I.e., these are
different types of review, and it seems that one should not preclude
the other.

-- 
Alex
http://www.psg.com/~zinin/

_______________________________________________
Icar mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/icar

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 17 Dec 2003 14:01:53 -0700 (MST)

On Wed, 17 Dec 2003, Alex Zinin wrote:

> It seems that SIRS falls under {unstruct, early, x-func} category

Since SIRs are elected with a specific mandate and responsibility to
review documents, would not they be categorized as "structured"? Or do
we need a third category?

Thanks for the definitions! Please copy them to the charter, if any.

Alex.

--

---

From: Alex Zinin <[email-masked]>
Date: Wed, 17 Dec 2003 13:22:53 -0800

Alex,

> we need a third category?

I knew that was coming ;)

I guess the exact definition would be recruiting individuals at large
who perform unstruct review into a new structure that gives them
responsibility and credits. I.e. SIRS changes the struct/unstruct
property, which is fine as long as reviews happen.

> Thanks for the definitions! Please copy them to the charter, if any.

Or maybe someone with English better than mine could come up with
terms that are intuitively understood, so we don't have to inflate
the charter to the size of a tutorial.

Alex