Subject: [Icar] ICAR draft charter rev 2

From: Alex Zinin <[email-masked]>
Date: Thu, 8 Jan 2004 17:17:16 -0800

Revision 2 below, please.

-- 
Alex

   WG name: improved cross-area review (icar)

   Chairs: <TBD>

   General Area Director(s):
    Harald Alvestrand <[email-masked]>

   General Area Advisor:
    Harald Alvestrand <[email-masked]>
   
   Mailing list: [email-masked]
   Subscription: [email-masked]
   Archives : https://www1.ietf.org/mail-archive/working-groups/icar/current/maillist.html

   WG Description:

     The WG will work out mechanisms for improved cross-functional
     review within the IETF, addressing the issues identified by the
     PROBLEM WG. This includes a better community review, as well as
     more structured pre-IESG review that may be used to improve
     scalability of the IESG review function.

     Definitions:

      o Cross-functional review: document review covering different
      aspects of Internet technology, including those represented by
      WGs within different IETF areas.

      o Community review: document review performed by individual IETF
      participants and not caused by their responsibilities within
      the IETF management structure.

      o Structured review: more formal and role-based document review
      performed by individuals assuming certain responsibilities (such
      as by WG chairs, directorate and IESG members.)

     It is an explicit goal of the WG to come up with mechanisms
     encouraging earlier review of the documents. An early review is
     best for catching architectural problems while they're still
     relatively easy to solve. In particular, many cross-functional
     interactions can be spotted and dealt with, thus avoiding many
     "late surprises". A final review (currently done by the IESG) can
     catch remaining cross-functional interactions, as well as deal
     with overall quality issues.
     
     The WG will cooperate with others in starting and evaluating
     experiments with early community and structured reviews. The
     evaluation of such experiments may be published as Informational
     RFCs if the group so desires.

     The WG will also coordinate with other WGs involved in the IETF
     reform process on proposed changes to the IETF Working Group
     operations and Standards process if those are considered
     necessary.

   WG milestones:

     FEB 2004: Submit drafts on improved community and structured reviews
     SEP 2004: Submit draft on improved community review to
               the IESG for publication as BCP
     SEP 2004: Submit draft on improved structured community review to
               the IESG for publication as BCP
     SEP 2005: Evaluate WG progress and potential; close or recharter

_______________________________________________
Icar mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/icar

---

From: Pekka Savola <[email-masked]>
Date: Fri, 9 Jan 2004 17:00:55 +0200 (EET)

> Revision 2 below, please.

Yes, this seems to be good.  

Even though I'm not sure whether forming the WG is necessarily the
required condition for progressing Improved Cross-Area Review, it is
probably a sufficient, and the most logical one (unless the IESG just
decided to adopt a model, but as there was all kinds of weird pushback
in Minneapolis, perhaps this is required :-).

As for the milestones:

==> what does "Submit" mean here?  Submit as personal I-Ds?  Publish 
as WG I-D's (if so, say so)?

For what it's worth, I'm not 100% sure we can decide, with this 
aggressive timescale, which of the current individual submissions 
would be best adopted as WG items (or combined as new proposals to be 
submitted as WG items)?  Maybe you could clarify what kind of process 
you had in mind here?

==> Was the last one meant to be SEP 200_4_?

For what it's worth, it may be very difficult to find ways to improve
community review as it is -- I don't believe there have been many
proposals on that yet.  

Luckily, finding a solution for structured community review (e.g., by
adding those responsibilities for reviewing documents) has already 
sparked at least a couple of good proposals, and progressing with that 
should be much easier.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: David Meyer <[email-masked]>
Date: Fri, 9 Jan 2004 10:53:50 -0800

>> as WG I-D's (if so, say so)?

	As we just had out over on the MBONED list, we might
	consider using language such as  "Publish I-D"  to mean
	what I think you mean by "Submit drafts" (as WG
	document), and perhaps use "Submit to IESG..." to mean
	that an existing draft with be ready to go to the IESG?

	Dave

---

From: Alex Zinin <[email-masked]>
Date: Fri, 9 Jan 2004 18:22:44 -0800

Pekka, Robert, Dimitri-

Thanks for the comments, see below, please.

Friday, January 9, 2004, 7:00:55 AM, Pekka Savola wrote:
[...]

>      FEB 2004: Submit drafts on improved community and structured reviews

==>> what does "Submit" mean here?  Submit as personal I-Ds?  Publish 
> as WG I-D's (if so, say so)?

Since this is a WG milestone, a WG document is implied. Strictly
speaking, we don't publish, the secretariat does, after the
authors/editors submit.

> submitted as WG items)?  Maybe you could clarify what kind of process 

I am not sure either, but we should try to get something ready by
Seoul for a substantial discussion. The way I see it working (though
I'm open to other ideas) is we do the analysis of problems with each
type of review at hand, decide which we problem we can solve, which we
can't, then go back to the features of the proposals on the table,
compare and discuss those and finally assemble solutions that are
documented in WG documents edited by appointed editors.

>      SEP 2005: Evaluate WG progress and potential; close or recharter

==>> Was the last one meant to be SEP 200_4_?

I think it was supposed to be JAN 2005--we need to give some time for
the WG follow up on the possible IETF LC and IESG comments. Thanks for
catching this, I'll update the text.

> community review as it is -- I don't believe there have been many

Yes, I think the community review problem is not 100% solvable, mainly
because we are dealing with sociological aspects, which we are not
good at. On the other hand, if there's something we can improve, we
should do it.

> sparked at least a couple of good proposals, and progressing with that 

Indeed. BTW, the difference between these types of review and how they
can potentially be addressed was my motivation behind splitting the
work items.

Friday, January 9, 2004, 10:01:23 AM, Robert Snively wrote:
> I like this much better.

> I am assuming that "structured review" will include some work on the
> review process itself, including perhaps things like the addressing
> of problems identified by the review, a formal response to the
> reviewers, solicitation of both "yes this is perfect" and "no this
> needs the following change" responses, etc.

> Is that correct?  Should we make it explicit?

Yes, I would expect the final product to at least make suggestions on
these aspects. However, I don't think details of this sort should be
in the charter--I consider these part of the "mechanisms" called out
in it.

Friday, January 9, 2004, 3:14:39 PM, [email-masked] wrote:
> alex, this revision of the charter proposal looks
> pretty good

> in the last sentence, the word "potential" in the
> last sentence might be further detailed

I suppose you mean in the last milestone. If so, this is just a
standard alias to "do we have useful stuff to work on, or do we
close?"

Alex

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 12 Jan 2004 10:19:43 -0700 (MST)

Alex,

	I am sorry, but I still do not like the proposed WG
charter/direction/milestones. Your Version2 edits were very useful in
that they made old problems with scope and authority more explicit and
visible. Perhaps charters are not perceived as important WG documents
in IETF, but I think they should be reviewed and polished as
rigorously as protocols.

	My specific comments and wording suggestions are inlined
below, but I am waiting for somebody to say that "rough consensus does
not require unanimity" and keep going in the direction you want.

On Thu, 8 Jan 2004, Alex Zinin wrote:

>      PROBLEM WG.

What is the rationale behind trying to limit the scope of the WG to
cross-functional review? Given your definition of "cross-functional"
as "document review covering different aspects of Internet
technology", the above scope definition includes virtually _any_
review within the IETF.

IMO, "any review" should be the explicit scope. We should not try to
limit the scope using unbounded concepts like cross-functional review
(as currently defined). I cannot think of any significant review
activity that the above scope statement (along with the
cross-functional definition) would clearly _exclude_.

Also, let's not imply that we will solve all PROBLEM WG problems and
let's not mix scope and an incomplete problem list together.

Suggested wording:

	The WG will improve IETF review mechanisms by adjusting
	existing mechanisms and proposing new ones. The improvements
	will address at least the following known issues:
		- provide a list of issues
		- quoting PROBLEM WG output as needed

>      This includes a better community review, as well as

I do not understand why this WG must stop at the IESG border. Are we
saying that IESG review mechanisms are already good enough and just do
not scale well? What if the best solution will require some IESG-level
changes? How about IAB involvement? Is it necessarily post-IESG?

Suggested wording:

	Review mechanisms under consideration cover both community and
	structured review, may span all levels of IETF structure, and
	may require changes to IETF entity roles or structure.

>       WGs within different IETF areas.

If my changes are accepted, the above definition becomes unnecessary.

>       as by WG chairs, directorate and IESG members.)

Good.

>      encouraging earlier review of the documents.

It is trivial to come up with encouraging mechanisms. We should aim
higher and also be more specific about what "earlier" means.  Earlier
than what? Suggested wording:

	It is an explicit goal of the WG to design "early review"
	mechanisms. These mechanisms should ensure that most IETF
	documents are reviewed and corrected before the estimated
	cost of correction becomes comparable with the cost of
	leaving the review-identified defect unfixed.

The above suggested wording needs further polishing, but I hope it
gives enough idea on how I think we should introduce early review into
the charter.

>      An early review is

I would suggest to remove the above paragraph as unnecessary and even
too suggestive about a stage-like approach and final review role.

>      RFCs if the group so desires.

Does the above wording change anything for this WG? In other words,
what would change if we remove the above paragraph? To me, it sounds
lile a non-statement that is true for every WG.

>      operations and Standards process if those are considered

Same comment: What would change if we remove the above paragraph? It
is so vague that it seems useless to me. If we can identify specific
WGs and liaisons, then we should do so. If we cannot, we are not
really providing any useful information here. All IETF WGs should
cooperate, of course.

>      FEB 2004: Submit drafts on improved community and structured reviews

40 days sounds like an unnecessary aggressive deadline, especially if
the WG has to submit all drafts by then. It would take at least 20-30
days to carefully review and polish existing drafts, and we do not
have any of decent quality to start with. I would suggest to add at
least one more month.

Also, "submitting a draft" is trivial because one can submit a
placeholder. See specific wording suggestions below.

>                the IESG for publication as BCP

Again, I am against separating the two drafts in the charter. Perhaps
we will have one draft addressing both areas with a single, unified
solution. Or maybe we will have three drafts that split the work
horizontally and not vertically (whatever that means). Also, it is not
clear whether we can plan that well in advance.

How about replacing all currently proposed deadlines with:

  FEB 2004: Decide on a comprehensive set of review mechanisms.
  MAR 2004: Document alpha versions of all review mechanisms.
  APR 2004: Evaluate WG progress and potential; close or recharter.

Thank you,

Alex.

---

From: Melinda Shore <[email-masked]>
Date: Mon, 12 Jan 2004 15:51:26 -0500

I think the proposed charter is in very good shape, in terms
of providing the basis for a "contract" between the IETF and
a new working group.  It's clear, it's crisp, it's focused,
it's unambiguous, and most importantly it describes work that
we know needs to be done.  Because chartering is an iterative
(sometimes highly iterative) process I think it's time to try
to get some feedback on this version from the probable area
director (Harald) and the rest of the IESG.

Melinda

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 12 Jan 2004 15:51:42 -0500

Having seen the discussion since this was posted, I thought it sensible to 
comment that I thought this revision provided a good frame for the working 
group.  It defines the goals, some of the approaches, and a set of 
observable milestones in reasonable time frames.

Yours,
Joel M. Halpern

At 05:17 PM 1/8/2004 -0800, Alex Zinin wrote:

>    Archives : 
> https://www1.ietf.org/mail-archive/working-groups/icar/current/maillist.html
>
>    WG Description:
>

---

From: "Spencer Dawkins" <[email-masked]>
Date: Wed, 14 Jan 2004 20:22:46 -0600

Yeah, it's getting there - short note below.

>      encouraging earlier review of the documents. An early review is

I would like an explicit discussion of timing scope. Maybe words like
"It is an
explicit goal of the WG to come up with mechanisms encouraging review
of the documents at every stage of development - from drafts under
consideration for WG adoption (do I have this end right?) to drafts
being forwarded for publication"?

---

From: Alex Zinin <[email-masked]>
Date: Thu, 15 Jan 2004 12:05:20 -0800

Spencer,

  Rev 3 I posted last night expands a bit more on this:

      It is an explicit goal of the WG to come up with mechanisms
      encouraging early review of the documents while ideas are still
      in the formation stage...

  I know it's not exactly what you suggest, but it does answer the
  question of "how early" that was asked several times.

-- 
Alex
http://www.psg.com/~zinin/