Subject: Data on internet drafts/year?

From: Melinda Shore <[email-masked]>
Date: Thu, 25 Sep 2014 18:34:39 -0800

Do we have somewhere data on the number of drafts submitted per year?

Melinda

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Fri, 26 Sep 2014 03:02:01 +0000

> Do we have somewhere data on the number of drafts submitted per year?

First place I’d ask is the RFC Editor. But...

Using the rsync archive at rsync.ietf.org...

I did an ls -lrt on the file of “all internet drafts since” someone started recording them, and all RFCs. I’m not convince that the year they wound of in the archive is the year in the index file, but I don’t feel like trying to parse the index file. But doing that and then gripping for year numbers (and noting that digit-digit-:-digit-digit is at the moment equivalent to “2014")

year    drafts  RFCs
1968		0
1969		0
1970		0
1971		0
1972		0
1973		0
1974		2
1975		0
1976		0
1977		0
1978		0
1979		0
1980		0
1981		1
1982		0
1983		0
1984		0
1985		0
1986		22
1987		43
1988		43
1989		51
1990	0	59
1991	0	125
1992	140	368
1993	8	175
1994	210	183
1995	721	126
1996	1409	174
1997	1688	388
1998	2896	272
1999	2708	269
2000	3369	426
2001	4355	320
2002	4758	233
2003	4969	312
2004	4763	281
2005	4933	327
2006	5066	469
2007	4954	324
2008	5115	309
2009	5431	291
2010	6242	403
2011	6495	390
2012	6974	339
2013	6629	275
2014	1934	60
:	3211	216

---

From: Eric Burger <[email-masked]>
Date: Thu, 25 Sep 2014 23:13:58 -0400

Interesting. From 1998 on it looks like we go from 10% of ID’s becoming RFCs to 5%. A combination of having a typical RFC being published after draft-15, as well as a ton of -00’s that go nowhere, perhaps?

---

From: Melinda Shore <[email-masked]>
Date: Thu, 25 Sep 2014 19:19:43 -0800

Thanks, that's extremely helpful (I'm looking at the ascendancy of the
problematic "problem statement" and would like to compare those numbers
to overall totals).  If I'm understanding you correctly, the : on the
last line is also 2014?

Melinda

---

From: Melinda Shore <[email-masked]>
Date: Thu, 25 Sep 2014 20:44:22 -0800

On 9/25/14 7:13 PM, Eric Burger wrote:
> Interesting. From 1998 on it looks like we go from 10% of ID’s
> becoming RFCs to 5%. A combination of having a typical RFC being
> published after draft-15, as well as a ton of -00’s that go nowhere,
> perhaps?

Well, we don't know what the lag time was, or how that may have
changed during that period (but we can guess!).  I found problem
statements by doing a repository search, which means that I only
counted one instance rather than having a per-revision-based count.
Even so, counting only one revision[*], the number of "problem
statement" drafts grew very quickly in the mid-aughties, a time when
the total number of submissions appears to have been relatively stable.

Anyway, the total numbers are small but to the extent that we
no longer seem to be able to undertake new work without a "problem
statement" I'd suspect that this may be contributing to process
delays.  Also, I have a broader problem with "problem statements"
in that 1) I think a charter should be relatively complete to start
with, and 2) often problem statements represent something that
someone thinks would be interesting to work on rather than a
technically mature idea that would otherwise satisfy what used
to be some of our criteria for taking on new work - basically, certainty
that the people involved can actually solve the problem being laid
out and have some concrete, workable ideas about how to do so.  I think
it's not only our slowness to complete documents that suggests that we
may be taking on some less productive aspects of other standards
bodies, but increased process ossification (problem statement->
use-cases->requirements->gap-analysis->architecture->maybe-one-day-
we'll-publish-a-protocol-eventually-perhaps) as well.

Melinda

[*] the exception to the one-revision count was when an individual
submission was resubmitted as a working group draft.

This is a count, by year, of the number of drafts which were submitted
and which contained the phrase "problem statement" in their title:

year	# of problem statement drafts
1997	2
1998	1
1999	0
2000	1
2001	5
2002	10
2003	9
2004	8
2005	15
2006	36
2007	28
2008	32
2009	23
2010	29
2011	32
2012	25
2013	34
2014	40

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Fri, 26 Sep 2014 05:34:27 +0000

> to overall totals).  If I'm understanding you correctly, the : on the

Yes.

You can get my source at http://trac.tools.ietf.org/group/tools/trac/wiki/DataSources; you’re looking for 
	rsync -avz rsync.tools.ietf.org::tools.id ./id

What you probably really want to do is search the actual file for the date it was posted. That may be different than the date that shows up in the ls -l.

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Fri, 26 Sep 2014 05:35:04 +0000

On Sep 25, 2014, at 9:44 PM, Melinda Shore <[email-masked]> wrote:

> delays.

My understanding, possibly incorrect, is that the process delay is intentional. A brief look at the RFC series and the internet drafts will tell you that we often start a project with a solution looking for a problem. Our response has been to ask for a problem statement draft, and then charter a working group that is not authorized to create a solution without a charter change. I don’t know how that doesn’t make the time interval from “I’d like to work on something” to “solution delivered to IESG” longer.

We also have created a work ethic in which dithering is accepted.

---

From: Melinda Shore <[email-masked]>
Date: Thu, 25 Sep 2014 21:43:03 -0800

On 9/25/14 9:35 PM, Fred Baker (fred) wrote:
> My understanding, possibly incorrect, is that the process delay is
> intentional. A brief look at the RFC series and the internet drafts
> will tell you that we often start a project with a solution looking
> for a problem.

That seems to mirror the situation in which we
seem to be seeing an increasing number of drafts describing a
problem for which there's not really any demand for a solution,
but it seems like an interesting problem (plus the question of whatever
incentives may or may not be provided by an employer to have
stuff working its way through the IETF process, regardless of
actual need).

At any rate I do think we're seeing a lot of low-quality problem
statement drafts.  The number of problem statement drafts is
clearly increasing and it may be the case that any time the
number of <whatever> goes up, the percentage of low-quality
<whatevers> increases, but still.  My core concern that we're
creating this process that requires going through a problem
statement/use cases/requirements/gap analysis/etc. ritual and
that actual protocol work doesn't happen for several years after
the work is chartered (then isn't published for several more
years).

Melinda

---

From: Hideki Sunahara <[email-masked]>
Date: Fri, 26 Sep 2014 15:01:45 +0900

> First place I’d ask is the RFC Editor. But...

This is file which is converted to CSV.

-- 
------------------------------------------------------------------------
慶應義塾大学大学院メディアデザイン研究科        |砂原 秀樹
Graduate School of Media Design                 |Hideki Sunahara
Keio University                                 |E-Mail: [email-masked]