Subject: RE: [Isms] Evaluation process?

From: "Fleischman, Eric" <[email-masked]>
Date: Mon, 29 Nov 2004 10:43:12 -0800

What criteria (and requirements) are the proposals be evaluated against?

-----Original Message-----
From: Ken Hornstein [mailto:[email-masked]] 
Sent: Monday, November 29, 2004 10:22 AM
To: [email-masked]
Subject: Re: [Isms] Evaluation process? 

>Funny, I was going to ask the same question but wait till Nov 1!

I sure hope you mean December 1 :-)

>So, an update to the WG would certainly be nice.  If a decision is 
>reached, I know that the plans are to write up a draft to summarize it,

>but a quick summary earlier on the list would be helpful.

The process has started, but not gotten very far; I was on another trip
after IETF, and was off most of last week, and I suspect many of the
evaluation team members were as will (I have a bunch of email to reply
to about this).

I was thinking of pushing back the deadline (since I doubt we're going
to meet it) for at least a couple of weeks, if no one objects (I haven't
yet consulted Juergen on this, so this isn't definite).

--Ken

_______________________________________________
Isms mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/isms

_______________________________________________
Isms mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/isms

---

From: Ken Hornstein <[email-masked]>
Date: Mon, 29 Nov 2004 15:02:13 -0500

>What criteria (and requirements) are the proposals be evaluated against?

The requirements have been proposed on the mailing list (see the archives
for more detail).  As for the criteria ... "we're working on it".  A
I-D detailing these things should be forthcoming.

--Ken

---

From: Robert Story <[email-masked]>
Date: Mon, 29 Nov 2004 15:04:29 -0500

On Mon, 29 Nov 2004 10:43:12 -0800 Fleischman, wrote:
FE> What criteria (and requirements) are the proposals be evaluated against?

I was wondering about that too. On the one hand, I can see the benefits of a
'behind-closed-doors' evaluation, in that the evaluators wouldn't have to deal
with constantly defending their position, as they would if the whole process
took place on the mailing lists.

On the other hand, I'd hate to see an recommendation made based on a
misunderstanding of one of the proposals.

Hopefully at least the draft authors are involved, answering questions, and
that communication will be made available (or at least summarized) with the
recommendation.

On Mon, 29 Nov 2004 13:22:00 -0500 Ken wrote:
KH> The process has started, but not gotten very far
KH> 
KH> I was thinking of pushing back the deadline (since I doubt we're going to
KH> meet it) for at least a couple of weeks, if no one objects

I'd rather have a slightly late but more informed evaluation over an on-time
but rushed one...

-- 
Robert Story; NET-SNMP Junkie
Support: <http://www.net-snmp.org/> <irc://irc.freenode.net/#net-snmp>

You are lost in a twisty maze of little standards, all different.

---

From: Wes Hardaker <[email-masked]>
Date: Mon, 29 Nov 2004 12:39:35 -0800

>>>>> On Mon, 29 Nov 2004 15:04:29 -0500, Robert Story <[email-masked]> said:

Robert> I was wondering about that too. On the one hand, I can see the
Robert> benefits of a 'behind-closed-doors' evaluation, in that the
Robert> evaluators wouldn't have to deal with constantly defending
Robert> their position, as they would if the whole process took place
Robert> on the mailing lists.

Sigh...  The IETF has done many different ways of evaluating different
proposals in the past.  Generally they've all had problems of some
sort or another in every case.  In many of them in disastrous ways.
The worst thing about a closed-doors or limited-number-of-people
mechanism is that the industry could just give up if they came up with
a solution no one wanted ;-)

Robert> Hopefully at least the draft authors are involved, answering
Robert> questions, and that communication will be made available (or
Robert> at least summarized) with the recommendation.

The draft authors were requested (on the list a while ago) to select
one person per draft as a point of contact for questions.

KH> I was thinking of pushing back the deadline (since I doubt we're
KH> going to meet it) for at least a couple of weeks, if no one
KH> objects

Robert> I'd rather have a slightly late but more informed evaluation
Robert> over an on-time but rushed one...

The ADs mandated a quick decision in order for the WG to exist.  If a
decision isn't made by March the WG is to be shutdown.  Unfortunately,
that may mean a less than ideal analysis process.  The flip side is
that we can't rathole.  Choose your poison.

-- 
Wes Hardaker
Sparta

---

From: Juergen Quittek <[email-masked]>
Date: Tue, 30 Nov 2004 16:43:14 +0100

Robert,

> took place on the mailing lists.

The evaluators will have to defend their recommendation anyway.
And I never heard of a comparable process where after the recommendation
(or decision) nobody complained heavily about it.  Since these complaints
do not have a tendency to stop soon, the IESG gave us a short deadline
for the overall process.

> misunderstanding of one of the proposals.

Everybody does.

> that communication will be made available (or at least summarized) with the

Since the decision will step on somebody's toes anyway,
the evaluators should be careful with stating progress
before their recommendation is final.

Thanks,

    Juergen
-- 
Juergen Quittek        [email-masked]       Tel: +49 6221 90511-15
NEC Europe Ltd.,       Network Laboratories        Fax: +49 6221 90511-55
Kurfuersten-Anlage 36, 69115 Heidelberg, Germany   http://www.netlab.nec.de