Subject: [Icar] Thoughts on ICAR direction

From: Margaret Wasserman <[email-masked]>
Date: Fri, 5 Nov 2004 15:00:26 -0500

Hi All,

(Writing just as an interested participant...)

I've been watching the evolution of ICAR, and it seems like we aren't 
succeeding in building a centralized structure for early review.  I 
think that there are a number of reasons for this, the most pertinent 
of which (IMO) is that a WG is not the right vehicle to develop and 
manage a program.

So, I have some ideas for directions that ICAR could take that don't 
involve building a centralized review infrastructure, so they might 
be more suitable to

(1) Document cross-area review criteria.  This would help to educate 
authors and ad hoc early reviewers in what to look for.

(2) Develop a process for WG chairs to use existing area review 
resources (MIB Doctors, GEN-ART, Security directorate, Ops 
directorate, etc.) to get early review for their documents?  Maybe 
this could be a proposed "July 14" experiment?

Thoughts?

Margaret

_______________________________________________
Icar mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/icar

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 06 Nov 2004 23:19:54 +0100

Margaret Wasserman wrote:
> think that there are a number of reasons for this, the most pertinent of 
> which (IMO) is that a WG is not the right vehicle to develop and manage 
> a program.
> 
> involve building a centralized review infrastructure, so they might be 
> more suitable to
> 
> resources (MIB Doctors, GEN-ART, Security directorate, Ops directorate, 
> etc.) to get early review for their documents?  Maybe this could be a 
> proposed "July 14" experiment?

These are both good suggestions. I wonder if the charter of the TOOLs
effort can be twisted a bit to host this work. It's interesting that
we've had two damp squibs (SIRs and now apparently ICAR) but MIB
doctoring, GEN-ART etc seem to be working nicely.

    Brian

---

From: "Spencer Dawkins" <[email-masked]>
Date: Sun, 7 Nov 2004 22:42:04 -0600

> I've been watching the evolution of ICAR, and it seems like we 
> aren't succeeding in building a centralized structure for early 
> review.  I think that there are a number of reasons for this, the 
> most pertinent of which (IMO) is that a WG is not the right vehicle 
> to develop and manage a program.

I could not, of course, agree more with this statement. I think I know 
the reasons, but they are off-topic for ICAR.

> authors and ad hoc early reviewers in what to look for.

Somewhere between ICAR, NEWTRK/ISD, and Gen-ART there's an idea trying 
to get out - please be patient while I try to explain this...

As a Gen-ART reviewer, I'm finding protocols and enhancements that are 
split across multiple documents that really don't hang together in a 
way that makes sense. Most recently, I was asked to review a protocol 
specification and an associated packet format specification - 
together, these documents define ONE protocol, but both documents had 
independent security sections, and these sections were different. 
Imagine the excitement of an implementor trying to figure out if one, 
or the other, or both, or neither are right in ten years.

It's really not fair to point stuff like this out on the Tuesday 
before a document is on an IESG telechat agenda, but when/where SHOULD 
we point it out?

> this could be a proposed "July 14" experiment?

In all fairness, ICAR was chartered pre-July14, so the only "legal" 
way forward was to develop a full BCP before implementing the BCP 
procedures to find out if they actually work! July14 - now approved as 
BCP 93, RFC 3933 <draft-klensin-process-july14-02.txt>, and coming 
soon to an RFC mirror near you - has as its goal gathering 
experimental experience with process changes before publishing the 
usual BCP document - and specifically calls out opportunities for 
trials in one or two corners of the IETF, rather than mandating 
deployment to the entire IETF community at once, as a normal BCP 
document would do. So, I agree completely.

---

From: Pekka Savola <[email-masked]>
Date: Mon, 8 Nov 2004 14:15:06 +0200 (EET)

On Fri, 5 Nov 2004, Margaret Wasserman wrote:
> So, I have some ideas for directions that ICAR could take that don't involve 
> building a centralized review infrastructure, so they might be more suitable 
> to
>
> (1) Document cross-area review criteria.  This would help to educate authors 
> and ad hoc early reviewers in what to look for.

Good idea.

> (2) Develop a process for WG chairs to use existing area review resources 
> (MIB Doctors, GEN-ART, Security directorate, Ops directorate, etc.) to get 
> early review for their documents?  Maybe this could be a proposed "July 14" 
> experiment?

Also agree here.  People actually already *try* this kind of thing, 
but the directorate addresses and other contact information is not 
well maintained (in http://www.ietf.org/u/ietfchair/directorates.html 
the "broadcast addresses" are not mentioned).

FWIW, the ICAR process is IMHO really critical for getting the quality 
better and taking some sweat off the IESG's back.  Let's see if we can 
still find ways to motivate (or do some arm-twisting) of people to get 
on board.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 08 Nov 2004 09:57:46 -0500

Note that we have not waited for the working group to even fully approve 
the document before trying to make the experiment happen.

My concern, speaking personally, is that the community does not seem very 
interested in actually seeing a well defined pool of early reviewers happen.

Yours,
Joel

At 11:42 PM 11/7/2004, Spencer Dawkins wrote:
>In all fairness, ICAR was chartered pre-July14, so the only "legal" way 
>forward was to develop a full BCP before implementing the BCP procedures 
>to find out if they actually work! July14 - now approved as BCP 93, RFC 
>3933 <draft-klensin-process-july14-02.txt>, and coming soon to an RFC 
>mirror near you - has as its goal gathering experimental experience with 
>process changes before publishing the usual BCP document - and 
>specifically calls out opportunities for trials in one or two corners of 
>the IETF, rather than mandating deployment to the entire IETF community at 
>once, as a normal BCP document would do. So, I agree completely.

---

From: "Spencer Dawkins" <[email-masked]>
Date: Mon, 8 Nov 2004 10:30:43 -0500

Joel,

> Note that we have not waited for the working group to even fully 
> approve the document before trying to make the experiment happen.

Thanks to you and Mark for not letting process get in the way of 
progress :-)

Spencer

---

From: Alex Rousskov <[email-masked]>
Date: Sun, 14 Nov 2004 10:27:56 -0700 (MST)

On Sat, 6 Nov 2004, Brian E Carpenter wrote:

>> (1) Document cross-area review criteria.  This would help to 
>> educate authors and ad hoc early reviewers in what to look for.
>> 
> These are both good suggestions. I wonder if the charter of the 
> TOOLs effort can be twisted a bit to host this work. It's 
> interesting that we've had two damp squibs (SIRs and now apparently 
> ICAR) but MIB doctoring, GEN-ART etc seem to be working nicely.

I agree that the Tools team can document interfaces and/or suggest 
tools to support review processes (e.g., solicitation, archival, and 
linking of reviews). However, it looks like Mark is doing a "good 
enough" job already, and we would need more data/experience to design 
something more permanent.

On the other hand, I would not task the Tools team to document review 
criteria or to develop solicitation process for Chairs. Doing that 
seems too far from the current Tools charter.

Alex.

P.S. I am speaking as a Tools team member, but do not claim to 
represent team's position.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Mon, 15 Nov 2004 13:53:28 +0100

--On søndag, november 14, 2004 10:27:56 -0700 Alex Rousskov 
<[email-masked]> wrote:

> I agree that the Tools team can document interfaces and/or suggest tools
> to support review processes (e.g., solicitation, archival, and linking of
> reviews). However, it looks like Mark is doing a "good enough" job
> already, and we would need more data/experience to design something more
> permanent.

Just checking.....

Mark - do you feel that your "prototype" infrastructure can support a 
serious "freewheeling" review effort, or would you have to invest more 
effort in it before it's "routinely usable"?

                       Harald

---

From: Mark Allman <[email-masked]>
Date: Mon, 15 Nov 2004 08:10:09 -0500

> effort in it before it's "routinely usable"?

Let's use what I have.

If past indications are anything to go by, "freewheeling review effort"
won't be so tough on the tool.

(More of my thoughts later in the day.)

allman