Subject: [Icar] ICAR draft charter

From: Alex Zinin <[email-masked]>
Date: Wed, 7 Jan 2004 12:25:03 -0800

Folks-

Draft charter below. Please read. I'd like to hear from as many people
as possible.

Thanks.

-- 
Alex

   WG name: improved cross-area review (icar)

   Chairs: <TBD>

   General Area Director(s):
    Harald Alvestrand <[email-masked]>

   General Area Advisor:
    Harald Alvestrand <[email-masked]>
   
   Mailing list: [email-masked]
   Subscription: [email-masked]
   Archives : 
   https://www1.ietf.org/mail-archive/working-groups/icar/current/maillist.html

   WG Description:

     Work out mechanisms for improved cross-functional review within
     the IETF. This includes a better community review, as well as
     more structured (formal and role-based) pre-IESG review that may
     be used to improve scalability of the IESG review function. It is
     an explicit goal of the WG to come up with mechanisms encouraging
     earlier review of the documents. An early review is best for
     catching architectural problems while they're still relatively
     easy to solve. In particular, many cross-area interactions can be
     spotted and dealt with, thus avoiding many "late surprises". A
     final review can catch remaining cross-area interactions, as well
     as deal with overall quality issues.
     
     The WG will cooperate with others in starting and evaluating
     experiments with both early reviews and structured reviews. The
     evaluation of such experiments may be published as Informational
     RFCs if the group so desires.

     The WG will also coordinate with other WGs on proposed changes to
     the IETF Working Group operations and Standards process if those
     are considered necessary.

   WG milestones:

     FEB 2004: Submit -00 draft on improved community review
     FEB 2004: Submit -00 draft on improved structured review
     SEP 2004: Submit draft on improved community review to
               the IESG for publication as BCP
     SEP 2004: Submit draft on improved structured community review to
               the IESG for publication as BCP
     SEP 2005: Evaluate WG progress and potential; close or recharter

_______________________________________________
Icar mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/icar

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 7 Jan 2004 14:03:26 -0700 (MST)

Alex,

	The revised charter does not address many comments about the
first version of the charter. What do you want us to do:

	- assume that all unaddressed comments were heard
	  and left unaddressed intentionally by the people
	  who know better;

	- assume that all unaddressed comments were missed
	  by mistake and should be repeated again by people
	  who have nothing better to do than to dig through
	  old postings and repeat old comments;

	- something else?

Please advice.

Also, unrelated to the above, do the proposed milestones meet Dave
Crocker's "usefulness" criteria? Should they?

Thanks,

Alex.

---

From: Melinda Shore <[email-masked]>
Date: Wed, 7 Jan 2004 16:41:02 -0500

I prefer the new, terser version.  There's still some material
in it that could be clearer, though.  You use a lot of different
terms to describe various kinds of reviews and it's not always
obvious what you mean by them.  I think it would be useful to
consolidate where possible and then define each of the terms and
what their relationship is (I assume that some are classes of
cross-area review, but I don't know that).  Similarly, you say
that the wg will cooperate with "others," but I don't know who
"others" means.  Also, have we decided to use a wg process to
make broader changes to IETF structure and processes?  The last
paragraph may need to change as the superstructure is defined
more clearly by the IESG.

So, I like the new version but think it could benefit from more
terminological clarity.

Melinda

---

From: Alex Zinin <[email-masked]>
Date: Wed, 7 Jan 2004 13:32:22 -0800

Alex,

>         - something else?

I did read your comments and some of them were addressed, some
weren't. See below:

> a) The scope of the WG is unclear or wrong. "Cross-area" scope is too
>    narrow (assuming formal IETF areas are implied). "Cross-functional"
>    scope is undefined in the proposed charter. It is not clear
>    whether "cross-functional" predicate eliminates something useful.

"cross-functional" seems to be quite self-explanatory to me and
adding a vocabulary to the charter would be too much, I think.
If you have a wording suggestion, please send the text.

> b) The playing space for the WG is undefined: it is not clear whether
>    the WG is allowed to propose changes to core IETF processes, rules,
>    and roles and, if yes, which processes/rules/roles are in change
>    scope.

The charter says:

> c) The charter assumes that "IESG review function" remains mostly
>    unchanged. There is currently no IETF consensus whether that
>    function needs to change, IMO. Even some of the quoted drafts
>    seem to imply significant changes to IESG review function.

I don't see how I could address this comment. Since there's no
consensus on this issue, we can't say the WG will change the IESG
review function. However, in the process of discussion, the WG may
very well reach consensus that such change is a good idea and it may
recommend a certain process change.

> d) The implied difference between "peer" and "structural" is unclear
>    to me. Define and contrast both terms better since your milestones
>    depend on the difference. Furthermore, an a priory assumption of
>    separation between "peer" and "structural"  review may hinder WG
>    creativity.

I changed the text to say:

              This includes a better community review, as well as
                                     ^^^^^^^^^^^^^^^^
     more structured (formal and role-based) pre-IESG review that may
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     be used to improve scalability of the IESG review function.

If you have a suggestion that would improve the text, please
send your wording.

> Crocker's "usefulness" criteria? Should they?

If you have a problem with the milestones, please explain what it is
and how you believe it should be fixed.

Thank you.

Alex

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 7 Jan 2004 15:15:51 -0700 (MST)

On Wed, 7 Jan 2004, Alex Zinin wrote:

> adding a vocabulary to the charter would be too much, I think. If
> you have a wording suggestion, please send the text.

Cross-functional is not self-explanatory to me. I cannot send the text
since I do not know what you mean by cross-functional. I do not see
why adding a definition of the primary scoping term to the charter is
"too much", especially if the definition is clear/obvious to you.

>      are considered necessary.

The above does not clarify the scope. It is not clear whether the WG
is allowed to propose changes that alter core IETF processes,
especially if those processes affect things other than just
cross-functional review. We must be explicit about what changes, of
any are "allowed" or are in the work scope. Coordination with other
WGs is fine, but does not define the scope.

For example, is it in scope of this WG to discuss enforcement of
review rules, such as that all comments must be recorded and addressed
or that coverage criteria must be satisfied? Is it in scope of this WG
to discuss IETF document management system and propose changes that
affect areas other than review (e.g., registration of IETF
participants to access the system)? Is it in scope of this WG to
discuss whether WG chairs can and should say "no" to already reviewed
drafts if their own review contradicts others opinions? Etc.

These are just random examples to illustrate the scoping problem.
Addressing each example in isolation does not address the problem.

> recommend a certain process change.

This needs to be documented explicitly. For example, "The WG is to
evaluate IESG review function and propose IESG role and process
changes as and if necessary". Or, alternatively, "The WG must operate
under the assumption that the current IESG review function and role
are not going to change much. Any changes to IESG functions and roles
are, hence, out of the WG scope."

>               This includes a better community review, as well as

I still do not know exactly what the difference is and why the two
are a priori separated from each other.

> If you have a suggestion that would improve the text, please

Since I am unsure of the intent, it is difficult for me to
suggest specific improvements. Said that, you can consider
replacing
     Submit -00 draft on improved community review
     Submit -00 draft on improved structured review
with
     Submit -00 draft(s) on improved IETF review

> and how you believe it should be fixed.

The above questions are not an implication of a problem, but an
invitation to other IETFers to discuss whether we are eating our own
dog food with this charter. I hope that, for example, Dave can review
the deadlines and indicate whether they satisfy his usefulness
criteria.

Alex.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Wed, 07 Jan 2004 17:16:01 -0500

Looking at the milestones, I actually hope that we will have several drafts 
on review for consideration by the working group in February.  I would not 
expect to see working group consensus on a single starting point.  I also 
would not be surprised if some drafts cover both community and structured 
review.

Hence, I would suggest that the first two milestones be combined into:
     FEB 2004: at least two drafts on improved review covering community 
and structured reviews.

Yours,
Joel

---

From: Alex Zinin <[email-masked]>
Date: Thu, 8 Jan 2004 17:14:53 -0800

Melinda, Alex, Joel, Robert-

I will post an updated charter in the next message, inline
answers below.

Wednesday, January 7, 2004, 1:41:02 PM, Melinda Shore wrote:
> cross-area review, but I don't know that).

I changed the text to say "cross-functional" everywhere, leaving
cross-area only in the title, mainly because of the acronym.
My personal interpretation of the difference between the two is
that cross-functional includes cross-area and cross-WG within
an area.

I also included definitons in the beginning of the charter.

> Similarly, you say
> "others" means.

I think we should leave this open, since those "others" may be any WG
that are willing to try and experiment with proposed mechanisms, seems
we shouldn't specify them now.

> Also, have we decided to use a wg process to
> make broader changes to IETF structure and processes?

I don't know any other one, and, as far as I know, a WG process is how
we did it the past (Scott will correct me if I'm wrong--I'm admittedly
new to the whole "process changing" saga).

> The last

Sure.

> "too much", especially if the definition is clear/obvious to you.

I changed the text to include definitions. Please check if it is any
better this way and propose improvements if you think they are needed.

> WGs is fine, but does not define the scope.

I think we will need changes in the IETF review process anyways.
However, it is hard to tell now how major they will be, whether they
will affect the core IETF processes, and how those core processes
should be defined. Proposing reasonable changes should be fine as long
as we keep in mind the primary goal of this WG--to improve the review
process.

[...]

> are, hence, out of the WG scope."

The approach should be, I believe, to improve the pre-IESG review
process in such a way that the IESG can make the decision quickly. On
the other hand, I could see how some details of IESG review process
may change to accommodate the new review process.

So, for this question and the question above, I don't think we should
put an explicit guideline in the charter.

> are a priori separated from each other.

I tried to explain this in my earlier message to the list:
http://www1.ietf.org/mail-archive/working-groups/icar/current/msg00020.html

The new text also contains some info in the definitions.

>      Submit -00 draft(s) on improved IETF review

I did something like this to address this and Joel's comments.

>      FEB 2004: at least two drafts on improved review covering community 

I changed the milestones.

Wednesday, January 7, 2004, 2:24:16 PM, Robert Snively wrote:
[...]
> I like this approach.

> I think you might be able to clarify the first paragraph a little
> bit.  As I understand it, you are actually focusing on two
> separate (and probably separable) issues with separate drafts.
> I had a bit of a problem understanding what kinds of things might
> be included in each of the drafts. [...]

Please check the new version, it may explain things better now.
Also, please see my earlier message:
http://www1.ietf.org/mail-archive/working-groups/icar/current/msg00020.html

Regarding guidelines for specific drafts, I think the charter should
not have those as different people may have different ideas in mind.

I also addressed your second comment. Please see the 1st para in the
new charter.

Alex

---

From: Eric Rosen <[email-masked]>
Date: Tue, 13 Jan 2004 10:38:12 -0500

I think the charter's fine.  

Whether  the group  will  actually achieve  something  is another  question.
Getting early cross-functional  review is a great idea,  at least in theory,
but in practice I have the following concerns: 

- While  I would welcome an APPs  guy's perspective on how  a routing change
  might impact APPs,  I don't necessarily want the APPs  guys telling me how
  to do routing.  For cross-functional review to be effective, the reviewers
  and  reviewees must  respect each  others' knowledge  in  their respective
  areas of expertise. 

  In the absence of this mutual respect, the whole thing is just an exercise
  in politics.

- If one really  wants to  know, e.g.,  how a  security change  will impact
  routing, it might  take a security guy and a  routing guy working together
  to figure  out all the  inter-relationships.  That's a bit  different than
  throwing the spec over the wall for someone in another field to review. 

- Everybody will  want to  see the  trade-offs  made in  such a  way as  to
  create the simplest result their own area.  This will lead it irresolvable
  conflicts (as we  frequently see on the main IETF list).   It is also easy
  to demand that other areas do things  that are beyond the state of the art
  in that area. 

I  look   forward  to  seeing   what  can  be   done  to  ensure   that  the
cross-functional review doesn't become counter-productive.

---

From: Mark Allman <[email-masked]>
Date: Tue, 13 Jan 2004 13:29:48 -0500

the latest version of the draft charter looks quite reasonable to me

thanks,
allman

--
Mark Allman -- ICIR -- http://www.icir.org/mallman/