Subject: Weekly posting summary for [email-masked]

From: HarringtonDavid 73653 <[email-masked]>
Date: Thu, 26 Jan 2006 21:17:06 -0500

Hi,

I strongly encourage the collection of data for understanding mailing list behaviors. However, I have concerns that this information could be interpreted to support specific positions about the benefit or lack thereof associated with mailing list behaviors. There has been no work to develop a reasonable model of mailing list behaviors and the benefits of certain patterns, so making unwarranted assumtions based on these statistics could easily lead to abuse.

I suggest designing an experiment, ala RFC3933, to collect statistical mailing list information for purposes of better understanding how certain patterns positively or negatively impact effectiveness. This collection should be done across a wide variety of mailing lists, e.g., all IETF mailing lists, to ensure an adequately diverse sample universe. Once collected, the data should be correlated to other statistics that are related to WG effectiveness, and be corrrelated to subjective analysis of the effectiveness of a WG. 

This data could then be used to design a better process for identifying and handling mailing list behaviors that might be considered abusive. I do not like adding lots of process; I would rather see us focus on developing technology, but if we need to have process, then let's at least design processes based on the best objective data we can gather.

Some of the analysisI think should be done:
1) is there a correlation between the posting patterns and the timely completion of milestones in a WG?
2) is there a correlation between posting patterns and the time between first publication of an I-D and its subsequent adoption by the WG? 
3) is there a correlation between posting patterns and the time between adoption of an I-D by the WG and the publication as a Proposed Standard?
4) Are there specific points in the provess when the posting pattern behaviors change? (such as after PS apporoval but before RFC publication, or during WGLC, or immediately after and updated I-D is posted?, etc.
5) in the subjective view of the chairs **of many WGs, not just ones that had problem posters**, did posting behaviors help or hurt the forward progress of the WG?
6) maybe a survey should be done for all WG participants to get their subjective view of whether posting behaviors helped or hurt WG process

I would be willing to help design and document a data collection experiment to contribute to a better understanding of the factors that impact the effectiveness of IETF processes.

David Harrington
[email-masked]

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Sat, 28 Jan 2006 00:12:32 +0100

Dear David,
This could be done in extracting archives and in building a statistic 
program. Then it could be given to some good mathematician to find 
correlations. The utilisation of as much IETF mailing lists as 
possible over the last 10 years could permit to discover some 
personal behavioral pattern. The real issue is that a mailing list is 
polylogue and is something we have to learn about. For example we all 
know that one single mail may trigger hundredth of posts. The problem 
is not to impeach the fight, but the match use to start it.
jfc

>timely completion of milestones in a WG?

if the Charter is considered by the WG or not.
if the final report to the IESG states that the Charter was fulfilled.
if the authors of the I-D are or not preselected before the creation of the WG?

>2) is there a correlation between posting patterns and the time 
>between first publication of an I-D and its subsequent adoption by the WG?

the delay between the start of a WG and the publication of the I-D.
The number of proposed changes to the I-D. Ratio of considered, 
denied, adopted changes. Number of mails per results  (global, per individual).
Number of never answered relevant questions.

>3) is there a correlation between posting patterns and the time 
>between adoption of an I-D by the WG and the publication as a 
>Proposed Standard?

This is a certain number of periods which should be detailed.

>4) Are there specific points in the provess when the posting pattern 
>behaviors change? (such as after PS apporoval but before RFC 
>publication, or during WGLC, or immediately after and updated I-D is 
>posted?, etc.

I am not sure all that correspond to a real debate? Also it does not 
take into consideration external aspects, appeals, etc.

>5) in the subjective view of the chairs **of many WGs, not just ones 
>that had problem posters**, did posting behaviors help or hurt the 
>forward progress of the WG?

Yes. But the problem is to assess if help/hurt. I would propose a 
simpler criteria - for each I-D Change, the delay since the 
proposition started. The last change. The pattern after the last 
change. The last change date vs. transmission to IESG.

>6) maybe a survey should be done for all WG participants to get 
>their subjective view of whether posting behaviors helped or hurt WG process

This is subjective. The best criteria is the difference between the 
first and last I-D and the way the changes came in (as a proposition 
or to block a proposition. Was that proposition to eventually get the 
resulting change).

The real issue I think is the consensus by exhaustion. This should be 
traced through the participation pattern and the number of mails to 
approve/disapprove a point the Chair deem consensual and the number 
of average participants at that time.

Another important point is the number of participants, silent or 
active during the life of the WG.

>I would be willing to help design and document a data collection 
>experiment to contribute to a better understanding of the factors 
>that impact the effectiveness of IETF processes.

I would suggest a simple thing: to take a few typical mailing lists 
the Chairs would release the number of registered participants (at 
which date), to download them as a file, and to build a tool to 
extact statistics. From previous work in that area for @large list, 
the size of the mails vs. the size of someone input in the mail is an 
important factor, as well as the mail being trimed or not.

I am interested too in working on that.
jfc