Subject: Re: query about ID/RFC statistics

From: Michael Richardson <[email-masked]>
Date: Tue, 26 May 2015 12:03:53 -0400

I had asked Jari:
> I think that your tool does not take any account of people listed as
> Contributors who are otherwise not in the list of authors.
>
> I'm concerned about our 5-author limits, and the difficultly this may
> present to other pushes to have RFCs better recognized by tenure track
> committees.

He answered:
> This is late… i have not had time to maintain the tools. Sorry. But
> going over author statistics issues today.

I then suggested that maybe someone else might want to step forward.

>Actually, my tool *does* take into account people listed in Contributor
>sections. Not ones in Acknowedgement sections. Of course, the recognition is
>based on natural language, so it is far from perfect.

>But yes, if you are listed as a contributor in an RFC you will be listed by
>my tool - it does not differentiate between authors and contributors.

And I thought that this was a useful thing to get out there.

This is wrt to: http://www.arkko.com/tools/docstats.html

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Christopher Morrow <[email-masked]>
Date: Tue, 26 May 2015 14:41:19 -0400

>>based on natural language, so it is far from perfect.

aren't these things listed in the XML and this a 'quick' xml xoath
parse away from win?

ie:
    <!-- Possibly a 'Contributors' section ... -->

    <section title="Contributors">
        <figure align='left'>
        <artwork><![CDATA[
            Eve Varma
            Alcatel-Lucent
            USA
            email [email-masked]

    ]]></artwork>
        </figure>
        <figure align='left'>
    <artwork><![CDATA[
            Dave Hood
            Ericsson
            USA
            email [email-masked]

    ]]></artwork>
      </figure>

    </section>

(not all things are xml I suppose, and not all include contributors)

---

From: Jari Arkko <[email-masked]>
Date: Wed, 27 May 2015 01:44:44 +0300

> aren't these things listed in the XML and this a 'quick' xml xoath

Yes, they are in some cases… the difficulty with getauthors has been the
special cases. The non-XML… the non-numbered sections… the oddly
intended stuff… the misspelled stuff… people’s names in different
formats… the people who misspell their names :-) or at least their
co-authors names :-)

You could of course say that we should ignore all that broken stuff.
I wanted to have a smaller error rate, hence included many special
cases. Almost all of this is data-driven, so once you add a pattern
line the tool will recognise it in the future. 

Anyway, I do have a set of tools that i really have no time to maintain.
One group of tools is getauthors/authorstats, the one that collects
document statistics. It is operational, but if taken over by anyone
else it needs a rewrite. Despite being somewhat data driven, the
rest of the code is a hack upon a hack.

Another group of tools is the IESG statistics tools, which would
be very interesting, but are no longer operational due to interface
changes to how the data tracker presents itself. It too would need
a lot of work.

If anyone is interested in putting time on these tools, let us know!
For instance, we could start with a Code Sprint project in Prague.
The IETF also runs official tools projects that are funded with
IETF funds. So far I have not considered these tools so business
critical that we’d need to have them done commercially. And some
of them have been up and running through community effort,
i.e., me, Lars, and a few people who have sent me edits.
Let me know if these are so critical that they’d need a more
official IETF attention.

Jari

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 27 May 2015 10:31:09 +0200

Den 27. mai 2015 00:44, skrev Jari Arkko:
> 
>> aren't these things listed in the XML and this a 'quick' xml
>> xoath parse away from win?
> 
> Yes, they are in some cases… the difficulty with getauthors has
> been the special cases. The non-XML… the non-numbered sections… the
> oddly intended stuff… the misspelled stuff… people’s names in
> different formats… the people who misspell their names :-) or at
> least their co-authors names :-)
> 
> You could of course say that we should ignore all that broken
> stuff. I wanted to have a smaller error rate, hence included many
> special cases. Almost all of this is data-driven, so once you add a
> pattern line the tool will recognise it in the future.
> 

This seems to be a common pattern among many of us :-)

Would it be a Good Idea to put them up on github under a reasonable
license?
I've found that projects hosted on github seem to have a vastly lower
barrier to entry for people who want to help than private projects, no
matter how much they are made publicly available through other means.

> One group of tools is getauthors/authorstats, the one that
> collects document statistics. It is operational, but if taken over
> by anyone else it needs a rewrite. Despite being somewhat data
> driven, the rest of the code is a hack upon a hack.
> 
> Another group of tools is the IESG statistics tools, which would be
> very interesting, but are no longer operational due to interface 
> If anyone is interested in putting time on these tools, let us
> know! For instance, we could start with a Code Sprint project in
> Prague. The IETF also runs official tools projects that are funded
> with IETF funds. So far I have not considered these tools so
> business critical that we’d need to have them done commercially.
> And some of them have been up and running through community
> effort, i.e., me, Lars, and a few people who have sent me edits.

---

From: Russ Housley <[email-masked]>
Date: Wed, 27 May 2015 10:28:47 -0400

>> From: Jari Arkko <[email-masked]>
>> Subject: Re: query about ID/RFC statistics
>> Date: 27 May 2015 01:44:44 GMT+3
>> To: Christopher Morrow <[email-masked]>
>> Cc: Michael Richardson <[email-masked]>, ietf <[email-masked]>
>> 
>> 

Is the source code for your pages already publicly available?

Russ

---

From: Jari Arkko <[email-masked]>
Date: Wed, 27 May 2015 17:38:44 +0300

> Is the source code for your pages already publicly available?

Yes:

http://www.arkko.com/tools/authorstats.html

Jari

---

From: Warren Kumari <[email-masked]>
Date: Wed, 27 May 2015 10:56:09 -0400

> Anyway, I do have a set of tools that i really have no time to maintain.

Which is very unfortunate, because they are super useful.

> document statistics.

I often use these tools to figure out what drafts I have active, and
which need a pokin'.

> It is operational, but if taken over by anyone

I think that this would be great -- eventually I think that they
should become IETF / tools maintained...

W

-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 27 May 2015 11:11:36 -0400

Hi Jari,

> http://www.arkko.com/tools/authorstats.html

Some tweaks to this are needed for people like me! I use my personal email 
address (daboo.name domain) as the email address in the author section, but 
I identify my company as "Apple Inc" in the <organization> element, and use 
an apple.com URI value. As it stands, the authorstats seem to think I work 
for a company called "Daboo".

I realize that the email domain is likely to be the least ambiguous way of 
detecting company, due to the variations that might appear in the 
<organization> (e.g., "Apple", "Apple Inc.", "Apple Computer, Inc" etc), 
but perhaps an alternative could be found?

-- 
Cyrus Daboo

---

From: Christopher Morrow <[email-masked]>
Date: Wed, 27 May 2015 12:24:07 -0400

On Wed, May 27, 2015 at 11:11 AM, Cyrus Daboo <[email-masked]> wrote:
> <organization> (e.g., "Apple", "Apple Inc.", "Apple Computer, Inc" etc), but
> perhaps an alternative could be found?

sounds, to me, like you want a simple map of: 'if this then that' (for
a slew of possibly customized by program runner) options.

over all it seems like a great first step is:
  "Import jari's code to github, get people cracking on it"

---

From: "Eggert, Lars" <[email-masked]>
Date: Wed, 27 May 2015 16:26:33 +0000

>  "Import jari's code to github, get people cracking on it"

+1

Lars