Subject: [Icar] icar futures?

From: Mark Allman <[email-masked]>
Date: Wed, 13 Oct 2004 13:21:54 -0400

Folks-

From my seat, this is what I have observed:

  * there has been no list traffic

  * there has not been any updating of the early review i-d

  * there has been no clamoring for late review guidance

  * the call for reviewers for the ICAR experiment netted 7 reviewers

     + plus 1 who said "ok", but never got back to me with his
       information; plus another who said "as soon as I qualify"
     + and, even some of the most vocal advocates of early review didn't
       belly up when the time to do so came around

  * the IESG did not enthusiastically offer assistance with choosing WGs
    to help seed the process

This doesn't look like a lovely picture to me.  This really smells like
ICAR is something the community does not want.  Or, at least does not
want to put their energy into, in practice.  Joel and I would appreciate
any thoughts on moving forward (and, saying "ok, it didn't work" **is**
a path forward).

Thoughts?

Thanks,
allman

--
Mark Allman -- ICIR -- http://www.icir.org/mallman/

_______________________________________________
Icar mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/icar

---

From: Scott W Brim <[email-masked]>
Date: Wed, 13 Oct 2004 14:41:30 -0400

On Wed, Oct 13, 2004 01:21:54PM -0400, Mark Allman allegedly wrote:
> This doesn't look like a lovely picture to me.  This really smells
> like ICAR is something the community does not want.  Or, at least does
> not want to put their energy into, in practice.  Joel and I would
> appreciate any thoughts on moving forward (and, saying "ok, it didn't
> work" **is** a path forward).

I still think it's the right idea, I think people are just busy.  Let's
poll the wg-chairs and see where they stand.  We don't need the IESG, we
can find an initial set of WGs ourselves, eh?

swb

---

From: Mark Allman <[email-masked]>
Date: Wed, 13 Oct 2004 14:50:40 -0400

Scott-

(our first volunteer reviewer, btw; thanks!)

> I still think it's the right idea, I think people are just busy.
> Let's poll the wg-chairs and see where they stand.  We don't need the
> IESG, we can find an initial set of WGs ourselves, eh?

Let me play the devil's advocate here...

  * "people are just busy"

    I agree.  I are such a people.

    But, if we have to twist people's arms to get them to volunteer to
    be reviewers, what indication do we have that they will actually do
    reviews?  I know a lot of people whom I could likely cajole onto the
    list of reviewers, howwver, it is not clear to me that such a path
    is really the right model.

  * "we don't need the IESG"

    I agree that we do not need this bit of help from the IESG to get
    things started.  I think the broader issue here is that the IESG is
    not enthusiastic about ICAR early review.  Is it reasonable to hope
    to see this sort of thing widely used without some support from the
    IESG?  (Maybe it is... but, I'd like to hear someone argue the
    point.)

On the "poll the wg-chairs" part, I am not sure what you mean.  I did
attempt to solicit reviewers from the WG chair mailing list.  

allman

---

From: Pekka Savola <[email-masked]>
Date: Thu, 14 Oct 2004 07:47:51 +0300 (EEST)

>     IESG?  (Maybe it is... but, I'd like to hear someone argue the

The IESG is crucial for this to work.  I'd suggest you ping IESG, 
through Harald if necessary about this.

One could ask that each AD tries to list 3-4 persons capable of ICAR
reviews they know of, and either the AD or you approach these persons 
1-on-1, soliciting help.

Remember, the fundamental problem is that broadcast don't work... 
you'll have to approach the people the right way ("you were 
recommended as a good reviewer, and we need YOUR help...")..

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 14 Oct 2004 12:16:40 +0200

Let me explain why I didn't volunteer.

1. Too busy in the last few weeks to think seriously about it.

2. Additionally, as a member of Harald's General Area review team,
I get a few reviews to do every two weeks, plus I spontaneously
review a couple of IDs a week in my areas of interest.
So (if I had found time to think about it), I'm pretty sure I
would have concluded I have no spare capacity.

Am I unusual among the people on this list? I don't think so.

Rather than giving up, I think this WG should orchestrate a
recruiting drive during the November IETF. Set up a desk in
the registration area, hang a poster, arrange for volunteers
to staff the desk and sign people on. (And of course, send
another solicitation to [email-masked] just before the meeting,
telling people they will have a final chance to help.)

If that fails, go to end of file.

     Brian
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Brian E Carpenter
Distinguished Engineer, Internet Standards & Technology, IBM

Mark Allman wrote:
> ------------------------------------------------------------------------
>

---

From: Mark Allman <[email-masked]>
Date: Thu, 14 Oct 2004 07:37:58 -0400

> reviews they know of, and either the AD or you approach these persons 

We pitched this idea to Harald before the last IETF meeting as a way to
seed the pool.  His response in the meeting, as I recall, was that he
didn't think the IESG was up for this.  Rather, he thought they'd help
by picking some WGs.  That did not happen.  Maybe Harald has a comment
to insert here or would like to try again.

> recommended as a good reviewer, and we need YOUR help...")..

I agree with that fundemental problem.  On the flip side, I worry that
while leaning on people to "volunteer" may produce a longer list of
reviewers, that does not necessarily yield a useful pool at the end of
the day.  It doesn't even take the IESG to get names.  I bet I could add
a bunch of names to the list without too much effort.  And, I bet there
are others who could, as well.  The fundemental question in my mind is
whether this is a useful approach or not.

allman

---

From: "Alex Rousskov" <[email-masked]>
Date: Thu, 14 Oct 2004 10:42:00 -0600

>   * the call for reviewers for the ICAR experiment netted 7 reviewers

This is not surprising to me, given the audience the call covered (less
than 10% of IETFers?), the incentives offered for reviewers
(virtually none) combined with IETF-wide level of excitement about the  
project
(marginal) and no easy way to join for most IETF participants with time
on their hands. IIRC, this outcome is even worse than I predicted when we
were arguing about pool models and solicitation tactics on this list.

I can only repeat what I have suggested before: provide some nice  
incentives
for reviewers, ask prominent IETFers to plug the experiment every time they
address an audience, praise existing reviewers on plenaries and web site,
think about getting commercial sponsors involved, etc. Good marketing is
required to sell something that the buyer does not need.

We may still fail to attract enough reviewers long-term, but we may at  
least get
enough of them for the initial experiment. If we still do not get enough,
we should close the WG or change the pool model.

Most of the above suggestions would require strong support from IESG, I  
guess.

Alex.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 14 Oct 2004 10:19:17 -0700

>  * the call for reviewers for the ICAR experiment netted 7
....
>   * the IESG did not enthusiastically offer assistance with
>  choosing WGs     to help seed the process

Folks,

There was quite a bit of quick signup for sirs.  So a poor 
showing now is likely to be more than just 'people are busy'.  
People are always busy.  I fear that the explanation could be 
broad-based burnout/frustration/distraction. However, analyzing 
this is probably only distracting.  

The only question that seems to me to be important is:  how do we 
get useful activity?

Placing the IESG into our critical path is unfortunately well 
demonstrated to be a mistake.  We could explore that assessment 
more fully, too, but I'm more interested in this activity doing 
useful things.  Bottom line is that we are on our own, if we want 
to make any progress anytime soon, and frankly I think that's ok.

To me, the critical path to achieve utility depends upon willing 
reviewers and willing reviewees.  Everyone else is overhead.

Brian's comment is interesting because it points out that reviews 
are happening, albeit not under the icar umbrella.

Is there a way to bring those activities within this effort, 
without having the exercise be more than political overhead?  
That is, is there value-add to provide? I think the community 
value-add is the bookkeeping benefit of being able to look in a 
single place and see what pattern of reviews we are getting.

Reviewers:

I suggest that we wholesale copy the sirs list over to the icar 
list.  The sirs signups were of enthusiastic, constructive folk. 
I really do not think that changing the name of the list they are 
on will be viewed negatively by them.  

As a nicety, I suppose we could first send a note on sirs 
declaring that everyone will be copied over in 1 week and that 
anyone wanting to opt-out -- and I hope folks appreciate the spam 
irony of choosing an opt-out model -- needs to do so within that 
week.

Reviewees:

1. Make sure that the icar service is announced broadly.  
wgchair, ietf list, wg chair training, and select working groups 
that we target.

2. Go after those select groups directly.  Long ago I said I had 
one or two in mind.  I still do, but they are taking awhile to 
get chartered.  Hmmm.  Maybe I don't have to wait for that?

I have 2 sets of specifications that need reviews 

	1. CSV
	2. domainkeys and identified mailer

d/
--
Dave Crocker
Brandenburg InternetWorking
+1.408.246.8253
dcrocker  a t ...
www.brandenburg.com

---

From: "Spencer Dawkins" <[email-masked]>
Date: Thu, 14 Oct 2004 22:34:28 -0500

> this is probably only distracting.

Yeah - it IS worth knowing that a bunch of the SIRS were already 
Gen-ART volunteers (for instance, me), and (like Brian) didn't 
volunteer AGAIN...

http://www.alvestrand.no/ietf/gen/reviewer-list.html

Spencer

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 15 Oct 2004 14:00:28 +0200

Dave,

Dave Crocker wrote:
...
> irony of choosing an opt-out model -- needs to do so within that 

I think this is a good idea. Let's do it.

     Brian

---

From: "Spencer Dawkins" <[email-masked]>
Date: Sat, 23 Oct 2004 22:25:33 -0500

Unfortunately, I'll need to be one of the SIRs who opts out of ICAR. 
Just a few thoughts, though ...

I happen to agree with Alex Rousskov that ICAR doesn't have an obvious 
incentive for participation. My sponsoring organization really liked 
the marketing aspects of supporting "Senior Internet Reviewers". I can 
explain to my sponsoring organization why I participate in 
directorates, and I can explain why I participate in an Area Review 
Team. Given a finite number of hours in the day...

I happen to agree with Brian that setting up a recruiting drive at 
IETF 61 is a good thing to do - asking people in person will almost 
always get better responses than a broadcast appeal (see previous 
discussions in a variety of venues for examples).

I happen to agree with Pekka that IESG support for ICAR is crucial. A 
year ago, I had the hope that the IESG would be willing, at some 
point, to review reviews instead of reviewing documents. That 
willingness has not arrived, in the general case. If ICAR reviewers 
are no different than other reviewers, there's no reason for a working 
group to involve involve ICAR reviewers - and, as a side effect, all 
the infinite tuning that we did on qualifications for ICAR reviewers 
was wasted.

I happen to agree with Dave that we got people signed up more rapidly 
for SIR in 2003 than we're getting people signed up for ICAR in 2004. 
That's unfortunate - ICAR really should matter, but we've managed to 
take nine months to design an experiment, losing momentum in the 
meantime. For one IETF participant's view of why this happens, see 
sections 1, 2, and 3 of 
http://bgp.potaroo.net/ietf/all-ids/draft-klensin-process-july14-01.txt, 
which we cut out so we could get this draft published as RFC 3933.

Since I'm agreeing with almost everyone who's posted in this list in 
October, why do I feel so disagreeable?

I have to disagree with one posting. Saying "The IESG thinks that the 
ICAR early review is important to the IETF, and wants to support the 
experiment. But we haven't been able to do much follow-up work for it" 
since ICAR was chartered in February seems to mean that the IESG 
thinking something is important doesn't matter much, or at least 
doesn't translate to any actual movement. That's unfortunate.

But best of luck to all who stay in/opt in. It's been real.

Spencer

----- Original Message ----- 
From: "Brian E Carpenter" <[email-masked]>
To: "Dave Crocker" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Friday, October 15, 2004 7:00 AM
Subject: Re: [Icar] icar futures?

>> list.  The sirs signups were of enthusiastic, constructive folk. I 
>> really do not think that changing the name of the list they are on 
>> will be viewed negatively by them.  As a nicety, I suppose we could 
>> first send a note on sirs declaring that everyone will be copied 
>> over in 1 week and that anyone wanting to opt-out -- and I hope 
>> folks appreciate the spam irony of choosing an opt-out model --  
>> needs to do so within that week.
>