Subject: More on the Secretariat Statement of Work (SOW)

From: Bernard Aboba <[email-masked]>
Date: Fri, 23 Dec 2005 12:50:23 -0800 (PST)

In thinking through the Statement of Work (SoW), I think that an important 
component is to provide the IETF with sufficient information on how well 
the organization is performing. 

There are many metrics for that, but an important one is the time taken in 
various stages of the IETF process.  

Unfortunately, it is not clear to me that we are currently collecting this 
information in a form that makes it easy to analyze.  We are also not 
analyzing the data on a regular basis, using it in a systematic effort to 
improve IETF performance (or at least to prevent it from deteriorating 
further).  

Researchers such as Tim Simcoe of the University of Toronto have studied 
metrics of IETF performance and have come to some interesting conclusions.  
For example, it appears that time from an initial -00 to RFC publication 
varies considerably by area, as well as by designation (Information, 
Experiemntal, Proposed).  In the process of developing this research, Tim 
has also had to do significant work to adjust the data to make it suitable 
for analysis. 

My suggestion is that the IAOC needs to start thinking about what data 
and reports are needed to enable the IETF to measure and improve its 
performance.  

References
----------

Simcoe, T., "Delays and de Jure Standards: What Caused the Slowdown in  
Internet Standards Development?", UC Berkeley Haas  School of Business, 
April 2004.  

Simcoe, T., "Standards Setting Committees", J.L Rotman School of 
Management, University of Toronto, Decmeber 2005. 

Available at: http://www.drizzle.com/~aboba/IAB/simcoe/

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: "Spencer Dawkins" <[email-masked]>
Date: Fri, 23 Dec 2005 18:02:49 -0600

Bernard/All,

Ack on Bernard's note.

I know that speed isn't the only thing that matters, but if we move slowly 
enough, the other stuff that matters won't matter.

I'm remembering from previous discussions (sometime around the time of 
http://www3.ietf.org/proceedings/03mar/134.htm? or 
http://www3.ietf.org/proceedings/04mar/981.htm?) that the states we track in 
the ID tracker are sometimes overloaded, so it's hard to tell who has the 
token and exactly what is happening with the draft, and there are limits on 
what we've been able to do with metrics in the past.

It's definitely worth thinking about this from a metrics perspective.

Spencer