Subject: Document Review: Volunteers Needed

From: Margaret Wasserman <[email-masked]>
Date: Wed, 05 Mar 2003 22:33:55 -0500

Hi All,

The recent poor response to our document last calls (2
comments for one document, 4 comments for the other) is
insufficient to justify advancing these documents to the
IESG.

Before we advance these document, or any documents, to the
IESG, we want to be certain that they have been reviewed by
a significant number of people representing a cross-section
of areas of expertise.

In an effort to fix this problem, Bob, Itojun and I are
considering forming a semi-official v6ops document review team.

This team would consist of people who are willing to spend a
couple of hours reviewing and commenting on v6ops documents at
each stage of the process -- WG acceptance, major intermediate
revisions, WG last call(s), revisions to address IESG comments,
etc.

The review team would be chosen from a pool of volunteers to
represent a broad range of knowledge and expertise, and members
would be removed if they are too frequently non-responsive.  All
review comments would be sent to the list, and treated like any
individual comments.

Like most new processes, we don't know exactly how this
would work, and there will probably be bugs to work out
over time...  But, we strongly feel that we have to do
something to ensure that our documents are well reviewed
before they are accepted by the WG and/or advanced to the
IESG.  We also hope that our efforts may serve as a useful
model for improvements in this area throughout the IETF.

What does the WG think about this idea?

If you think that this is a good idea, how could we give
credit to this team?  Perhaps list them on the v6ops
alternate web-site, and list them as "technical reviewers"
in the acknowledgements section of our published
documents?

Who would be willing to volunteer for this type of role?

Thanks,
Margaret

---

From: "JORDI PALET MARTINEZ" <[email-masked]>
Date: Thu, 6 Mar 2003 07:38:54 +0100

Yes, it could work, and I will be happy to volunteer and participate.

Regards,
Jordi

----- Original Message ----- 
From: "Margaret Wasserman" <[email-masked]>
To: <[email-masked]>
Cc: "Bob Fink" <[email-masked]>; "Itojun" <[email-masked]>
Sent: Thursday, March 06, 2003 4:33 AM
Subject: Document Review: Volunteers Needed

> Who would be willing to volunteer for this type of role?

*********************************
Madrid 2003 Global IPv6 Summit
12-14 May 2003 - Pre-register at:
http://www.ipv6-es.com
Interested in participating or sponsoring ?
Contact us at [email-masked]

---

From: "Marcin Michalak" <[email-masked]>
Date: Thu, 06 Mar 2003 09:29:41 +0100

> Yes, it could work, and I will be happy to volunteer and participate.
Me too :-) I've been working with some IPv6 projects already (NGN-
LAB, 6NET, Eurov6, now Euro6IX) and would be happy to contribute.
 Marcin
----------------------------------------------------------
Marcin Michalak		Research Engineer
Mobile: +41 79 330 83 51	Telscom AG

---

From: César Olvera Morales <[email-masked]>
Date: Thu, 6 Mar 2003 10:01:31 +0100

Hi,

I'm working with Jordi Palet, and I would like to participate in this
effort.

Regards,

César Olvera
Consulintel

---

From: Pekka Savola <[email-masked]>
Date: Thu, 6 Mar 2003 12:24:49 +0200 (EET)

See a few comments below.

> insufficient to justify advancing these documents to the

Good.

> a significant number of people representing a cross-section

This doesn't, I think, guarantee a representation of a cross-section of 
areas of expertise, unless you're very focused when gathering the team to 
see which kind of expertise they represent (more or less).

Just gathering up folks is likely to equal "some" expertise, with 
purposely vague definition of "some".
 
> in the acknowledgements section of our published

Credit is the only "reward" (in addition to Getting Better Internet) to
those folks who do work at the IETF based on its original values (and not
because they're paid to do it, get bonuses for pushing through drafts,
advocate their employer's ideas etc.), so I think it is a must in one way
or another.

Depending on the size of "review board", it could be listed on the web 
page and the drafts.

The size of a board like this should be 5-10 (10 at most) persons.

Remember, the point (at least to me, as I understood it) is to do expert
review on drafts, not substituting "WG review" (by including everyone who
occasionally reads the drafts).

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: "EricLKlein" <[email-masked]>
Date: Thu, 6 Mar 2003 14:33:48 +0200

First, Sorry I missed sending this to everyone.
>> is my comments
> are Margret's responses

----- Original Message -----
From: "Margaret Wasserman" <[email-masked]>
To: "EricLKlein" <[email-masked]>
Sent: Thursday, March 06, 2003 2:21 PM
Subject: Re: Document Review: Volunteers Needed

>
> Hi Eric,
>
> At 08:56 AM 3/6/2003 +0200, EricLKlein wrote:
> >Although I think this idea is good, I think that they should still be
> >published to the whole list so that others can still offer their input
where
> >appropriate.
>
> Absolutely!  I think we have to keep several goals in mind:
>
>          1.  Improve the quality and timeliness of review
>                  that documents are getting.
>          2.  Don't reduce openness and/or the amount of
>                  information sent to the list.  That's why
>                  I specified that all review comments (and
>                  I should have said public document revisions)
>                  will continue to be sent to the WG mailing
>                  list.  It is not our intention to create
>                  a private sub-working group, just a group of
>                  people who have agreed to be accountable for
>                  doing good, timely reviews of our documents.
>          3.  Treat comments from all WG participants with
>                  the same weight as comments from semi-official
>                  reviewers.  (although frequent non-official
>                  reviewers may get arm-twisted to volunteer :-)).
>
> The whole point is to get more consistent, timely and credible
> review of these documents -- not to do anything that will cut
> down on WG openness, or the review feedback that we're already
> getting.
>
> >So I would make it a two part review:
> >Part 1 would be the review team and part 2 would be general comments.
>
> Yes, but I'd like to see both parts happen in parallel, not in
> sequence.  (I realize that you didn't imply a sequencing).
>
> Margaret

With all taht Margaret sid in response to my comments, I agree and would be
willing to review.
Eric

---

From: Margaret Wasserman <[email-masked]>
Date: Thu, 06 Mar 2003 07:59:08 -0500

Hi Pekka,

>see which kind of expertise they represent (more or less).

I agree.  We will need to ask people what their areas of expertise
actually are, and we may need to recruit for other areas.  Of
course, I'm not even sure what the "areas" are that we need
covered...  This will probably take some successive approximation
over time.

>purposely vague definition of "some".

True.  Once we have a list of volunteers, have determine their
areas of expertise, and have selected a subset (based on past
review responsiveness, expertise, etc.), we will probably have
to actively recruit for some uncovered areas.

>occasionally reads the drafts).

I agree.  The exact number may depend on what areas of expertise
we determine we need, and how many folks it takes to cover them.
I'd like to have a small enough group that it is easy for me, Bob
and Itojun to keep track of who has/hasn't returned comments, but a
large enough group to represent a solid, cross-functional review.

Margaret

---

From: Alain Durand <[email-masked]>
Date: Thu, 06 Mar 2003 15:50:37 -0800

Hi Margaret,

I share some of Pekka's concern. Having more people reviewing documents
is good, but not enough. The wg need to make sure that the review 
process
cover a broad enough base, with expertise coming from outside of the wg
and probably also outside of the IETF.

Note: this is not a new problem, we experiment the same lack of
interest back in the NGtrans days

That said, I'm willing to help.

	- Alain.

---

From: Ronald van der Pol <[email-masked]>
Date: Fri, 7 Mar 2003 13:25:53 +0100

> insufficient to justify advancing these documents to the

I think we all agree.

> considering forming a semi-official v6ops document review team.

That's a big step at once. Correct me if I am wrong, but I think
the recent Last Call was the first that said: "Silence does NOT
indicate consent during this phase." Apparently, that was not
enough or people did not read it. Maybe you could first try
to send reminders like: "People, we need more feedback on this
draft otherwise we are not going to advance it". I know that
would work for me :-) With formal review teams I am afraid we
are going to spend more time on procedures than on actual work.

	rvdp

---

From: Margaret Wasserman <[email-masked]>
Date: Fri, 07 Mar 2003 10:45:04 -0500

Good.

>are going to spend more time on procedures than on actual work.

I was hoping that we could do this in a way that is very
light-weight regarding procedures.  It would add a little
overhead for me, Itojun and Bob, but not much to the whole
WG.

What, in particular, are you concerned about?

Margaret

---

From: Marc Blanchet <[email-masked]>
Date: Fri, 07 Mar 2003 11:07:41 -0500

> insufficient to justify advancing these documents to the

My guess on this:
- informational documents, in general in IETF at least from my experience,
usually have much less interest (exceptions exists but...) in reviewing
from people, because they are informational (no protocol bits to take care
of).
- by no means, this is a judgement call on my part, just an observation.

these scenarios documents are good to do, but we should be careful in
having them laying around for a long period. We might end up having a
focus/interest problem in the end.

I would suggest the following:
- during SFO ietf (maybe sunday), we (wg) or you (chairs) call a review
meeting by inviting people to come all at the same time, same room and go
through them, printing/reading on the screen and have the editors ready for
discussion. The wg does not have to be all there, and in fact, would be
counterproductive. But have few people that commit to come and then we all
read it together.
- then in 2 hours, we would have done: many reviewers, reviewed fast, etc...
- speaking of myself, I would commit to a 2hour review session sunday
10h00-12h00am.

A suggestion.

Marc.

---

From: Ronald van der Pol <[email-masked]>
Date: Fri, 7 Mar 2003 21:47:05 +0100

> overhead for me, Itojun and Bob, but not much to the whole

That sounds good.

> What, in particular, are you concerned about?

Several procedural issues have already been raised. I just think a
review team is an overkill at this moment. I think with a little
pressure we can get enough Last Call replies. Jim made a important
observation about the number of volunteers already for a review team.

	rvdp

---

From: Pekka Savola <[email-masked]>
Date: Sat, 8 Mar 2003 00:31:05 +0200 (EET)

On Fri, 7 Mar 2003, Ronald van der Pol wrote:
[...]
> Jim made a important

Note that not everyone necessarily "spammed" the list with their "yes, 
please" :-).  For all we know, there might be a dozen or two volunteers 
already.

---

From: Tim Chown <[email-masked]>
Date: Sun, 9 Mar 2003 21:27:12 +0000

Hi Margaret,

So long as comments are sent to the list, I think this is fine.  Any carrot
to get more comments has to be a good thing.   I would have reservations
if discussions were held off-list though, even if it meant improved quality,
because openness is also important :)

Tim

---

From: Margaret Wasserman <[email-masked]>
Date: Sun, 09 Mar 2003 16:36:11 -0500

Hi Tim,

>because openness is also important :)

I definitely agree.

Margaret