Subject: Re: I-D ACTION:draft-etal-ietf-analysis-00.txt

From: Bob Braden <[email-masked]>
Date: Thu, 28 Mar 2002 16:49:36 GMT

*> 
  *> A New Internet-Draft is available from the on-line Internet-Drafts directories.
  *> 
  *> 
  *> 	Title		: Toward a Quantitative Analysis of IETF Productivity
  *> 	Author(s)	: M. Rose, D. Crocker
  *> 	Filename	: draft-etal-ietf-analysis-00.txt
  *> 	Pages		: 23
  *> 	Date		: 27-Mar-02
  *> 	
  *> This memo presents an initial quantitative analysis of the IETF's
  *> working groups using RFC publication as the primary metric.
  *> 
  *> 

Right.  And presumably one would measure the effectiveness of the
health care industry by counting the number of prescriptions that
doctors have written.

Increasingly, the most important standards RFCs may be the
(ill-advised) ones we DON'T publish.

Bob Braden

---

From: Lloyd Wood <[email-masked]>
Date: Thu, 28 Mar 2002 17:59:21 +0000 (GMT)

> health care industry by counting the number of prescriptions that

from that draft:

   If these sorts of measures are useful, the question is what
   additional measurements and analyses should be pursued?

number of authors listed on each WG item output and trends in same
[1, 2].

Obviously.

L.

[1] J. Touch, private communication.
[2] M. Allman, private communication.

<[email-masked]>PGP<http://www.ee.surrey.ac.uk/Personal/L.Wood/>

---

From: Keith Moore <[email-masked]>
Date: Thu, 28 Mar 2002 13:06:56 -0500

> (ill-advised) ones we DON'T publish.

from my experience on IESG, there are far too few of these.

there's nothing wrong with looking at aggregate data about working groups'
production of RFCs.  of course, one should be careful about conclusions.
for instance, a WG that produces many RFCs isn't necessarily better than
one that produces few RFCs,  two WGs that produce the same # of RFCs
in the same time interval aren't necessarily producing the same quality 
or utility for their output, and a WG that shuts down after one year 
isn't necessarily any less productive than a WG that produces a 
document after one year. * 

OTOH, if we have several WGs that go on for several years without 
producing any documents, that's probably a bad sign.

overall, I don't find anything wrong with the draft (from a quick reading)
other than the title - implying that the tables within are somehow 
indicative of IETF or WG "Performance" in any meaningful sense.

Keith

* though I wish that WGs that shut down without producing a protocol
document would at least write up a brief RFC that explains what it 
tried to do and why it shut down - was the problem "too hard" or 
infeasible, was the subject too politically contentious (and what
were the divisive positions),  were there found to be adequate 
solutions already developed, did people lose interest, or what?  
but sometimes even this much reporting is too difficult.