Subject: RFC production center XML format usage, was: [IAOC] xml2rfc and legal services RFPs

From: Sandy Ginoza <[email-masked]>
Date: Wed, 23 Feb 2011 07:45:53 -0800

>> Can the RFC Production Center remind us about what the actual numbers
>> are, and also whether they ever generate an XML version if the authors
>> didn't supply one?

Julian's numbers are good approximations; 50-60% of docs have XML source files.  

The Production Center does not create XML files.  We process XML files when XML source files are provided to us.  

Thanks,
Sandy
for the RFC Production Center

> From: Julian Reschke <[email-masked]>
> Date: February 23, 2011 5:09:54 AM EST
> To: Russ Housley <[email-masked]>
> Cc: [email-masked], IAOC <[email-masked]>, IETF <[email-masked]>
> Subject: RFC production center XML format usage, was: [IAOC] xml2rfc and legal services RFPs
> 
> On 23.02.2011 10:46, Julian Reschke wrote:
>> On 22.02.2011 17:18, Russ Housley wrote:
>>> 
>>>>> There are many tools used by I-D
>>>>> authors, including nroff/troff/groff, NroffEdit, MS Word templates,
>>>>> LaTex,
>>>>> and a plethora of editors. The IETF cannot take on every tool that
>>>>> seems
>>>>> popular.
>>>> 
>>>> 
>>>> Isn't the statistic of xml2rfc use somewhere around 50-60%?
>>> 
>>> Roughly half. Again, that is why there is a value to the RFC
>>> Production Center.
>>> ...
>> 
>> Can the RFC Production Center remind us about what the actual numbers
>> are, and also whether they ever generate an XML version if the authors
>> didn't supply one?
>> ...
> 
> I just realized that I have an archive of XML versions of RFCs in AUTH48 state; so I *can* report the percentage of XML versions since ~RFC5000.
> 
> Note that these may be inaccurate (for instance, not all RFC numbers get assigned, right?); it just observes how much XML files ended up in AUTH48.
> 
> Here are the numbers:
> 
> RFC5000 - RFC5099: 41
> RFC5100 - RFC5199: 44
> RFC5200 - RFC5299: 47
> RFC5300 - RFC5399: 48
> RFC5400 - RFC5499: 64
> RFC5500 - RFC5599: 59
> RFC5600 - RFC5699: 58
> RFC5700 - RFC5799: 64
> RFC5800 - RFC5899: 63
> RFC5900 - RFC5999: 68
> RFC6000 - RFC6099: 69
> 
> Best regards, Julian

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Wed, 23 Feb 2011 12:29:33 -0500

> From: [email-masked] [[email-masked]] On Behalf Of Sandy Ginoza [[email-masked]]
> 

Rising fairly steadily, and approaching 70% in the most recent RFCs.

Dale

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 23 Feb 2011 13:59:23 -0500

> Rising fairly steadily, and approaching 70% in the most recent RFCs.

Exactly.

I was impressed with just how steady that increase appears to be over a 
reasonably extended period of time, as well as its seeming to be around 70%, not 
50%, now.

Pretty serious 'market' domination...

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 24 Feb 2011 09:42:21 -0500

On Wed, Feb 23, 2011 at 01:59:23PM -0500, Dave CROCKER wrote:
> reasonably extended period of time, as well as its seeming to be around 
> 70%, not 50%, now.
>

I submit that this is a consequence of the submission process.  The
idnits tool is now an effective gatekeeper against people who are
preparing documents with something other than xml2rfc and, maybe, the
*roff gui that's available.  As a WG chair of a WG with a lot of
participants who have older toolchains, I field regular complaints
about how difficult it is to get things by idnits.  My stock advice is
to use xml2rfc, which is always being updated exactly so that it
conforms to the latest physical layout rules.

I personally think this is sort of a shame, because it reminds me of
other (near) monocultures based on tools that are considerably worse
than the state of the art (Word is my favourite example).

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Thu, 24 Feb 2011 10:18:10 -0500

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Andrew Sullivan [[email-masked]]

how difficult it is to get things by idnits.
_______________________________________________

I would expect that the idnits rules change only very slowly.  What is the real story on that?

Dale

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 24 Feb 2011 10:28:28 -0500

> I would expect that the idnits rules change only very slowly.  What is the real story on that?

They seem to change more quickly than people realise.  Also, of
course, a whole bunch of changes can go by between a given
contributor's last I-D, and a current one.  Not all the changes are
intentional; they're often pretty clearly based on heuristics that as
often as not look to me like they're derived from what certain tools
do.  So then if someone submits something that doesn't do exactly what
those tools do, they get a false failure.

Note that none of this is to attack the tools developers.  I've ranted
before about the absurdity of checking the formatting of early drafts
for perfection, so I won't bother again.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 24 Feb 2011 17:11:00 +0100

Hi Andrew,

> for perfection, so I won't bother again.

Still, I feel that the characterization above is less than spot-on.  Idnits
in submission-checking mode only returns errors for a few selected things
out of the myriad of things it is able to check, and those things are
clearly required by http://www.ietf.org/id-info/1id-guidelines.txt .

Only errors will prevent an automatic draft submission from going through;
ignore the warnings and comments to your heart's content for that purpose.

If you have examples of drafts which conform to the submission requirements
of 1id-guidelines.txt, but don't pass the submission check mode of idnits,
*please* tell me about them so I can fix things.

The ratio of gripes against idnits to actual bug reports is getting to
be a bit annoying; and I'd like to suggest that people either submit
bug reports, or direct the complaints against the requirements of
1id-guidelines.txt rather than against the tool which checks the
requirements if the problem is that the requirements are too strict.

	Henrik

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Thu, 24 Feb 2011 11:25:34 -0500

> From: [email-masked] [[email-masked]] On Behalf Of Henrik Levkowetz [[email-masked]]
> 

Are you saying that one can order the draft submission process to
continue even if the submission tool considers the draft to violate
idnits?  I had no idea that was possible; I believe (from memory) that
that is not clear from the error output.

> requirements if the problem is that the requirements are too strict.

Based on the user interface principle that each error message should
make it clear what possible next steps are, perhaps your advice
"either submit a bug report about the tool or direct a complaint
against the requirements of 1id-guidelines.txt [give link]" should be
provided in any message that the idnits-checker has rejected a draft.
It would probably reduce the number of non-useful gripes.

"There is a Toyota in the driveway.  I am holding the keys in my hand.
What do I do next?"

Dale

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Thu, 24 Feb 2011 16:32:14 +0000

> that is not clear from the error output.

idnits reports 3 categories of nits: errors, warnings & comments.

Only errors prevent automatic submission AFAIK.

HTH
Ben

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 24 Feb 2011 11:38:37 -0500

> requirements if the problem is that the requirements are too strict.

You're quite right that I'm using "idnits" as a portmanteau for the
whole "1id-guidelines checking at submission" bundle.  My apologies
for being imprecise.  I am indeed complaining about the latter and not
about the former.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 24 Feb 2011 17:48:44 +0100

> that is not clear from the error output.

This is at the top of the idnits output in submission checking mode:

  "Showing Errors (**), Warnings (==), and Comments (--).
  Errors MUST be fixed before draft submission."

> It would probably reduce the number of non-useful gripes.

It would also add noise (i.e., text which to most people most of the
time would carry no relevant information, and would just have to be
ignored) to each and every idnits report.

If you google for 'idnits', go to the first hit and click on 'feedback'
you'll be able to send feedback to the right place.  Or go to
tools.ietf.org and click on idnits, and do the same.  I think that
should be easy enough, without adding more noise to the world.

	Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 24 Feb 2011 17:59:02 +0100

Hi Andrew,

> for being imprecise.  I am indeed complaining about the latter and not

Ah.  Ok.  I'll go tweak some code, then :-)

Best,

	Henrik

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Thu, 24 Feb 2011 11:58:56 -0500

> From: Henrik Levkowetz [[email-masked]]
> 

Are people complaining about the errors or the warnings?

> should be easy enough, without adding more noise to the world.

Hey, man, you're the one who doesn't like getting pointless
complaints.  I'm just suggesting ways you might be able to reduce
them.

Dale

---

From: John C Klensin <[email-masked]>
Date: Thu, 24 Feb 2011 13:21:56 -0500

--On Thursday, February 24, 2011 09:42 -0500 Andrew Sullivan
<[email-masked]> wrote:

>> I was impressed with just how steady that increase appears to
>> be over a   reasonably extended period of time, as well as
>> its seeming to be around  70%, not 50%, now.
>> 

Dave,

Sure.   But, having started this thread, let me point out that I
don't think anyone has made a serious claim that xml2rfc is
unimportant to the community, that it should not work well, or
even that putting resources into making it more stable, more
maintainable, or better behaved is a bad idea.  Those figures
are interesting but they would be relevant only if one of those
"don't do this at all" arguments were being made.  They haven't
been.

I raised two issues and two issues only:

(i) That the "tell the community work is going to be started by
issuing an RFP" model was suboptimal and maybe inappropriate.  I
think the discussions and conclusions of the last two weeks
indicate that the community agrees with that position and that
the IAOC has recognized that agreement.

(ii) That perhaps the priorities about extensions in the draft
RFP were inappropriate.  I want to stress "perhaps" because,
while I have a view on that subject, I think the correct
priorities can emerge only from community discussion (which
takes us back to (i) above).  Interestingly, while the numbers
you and others cite can be used to argue that improving RFC
Editor Production Center Staff efficiency in dealing with
documents arriving in xml2rfc format, they can equally well be
used to argue that, if the community is producing that much text
in that format, it is at least equally (if not more) important
to improve the efficiency and capabilities of the tool (and
DTD/Schema) for those who are originating/writing the documents.

> I submit that this is a consequence of the submission process.
> The idnits tool is now an effective gatekeeper against people
> who are preparing documents with something other than xml2rfc
> and, maybe, the *roff gui that's available.  As a WG chair of
> a WG with a lot of participants who have older toolchains, I
> field regular complaints about how difficult it is to get
> things by idnits.  My stock advice is to use xml2rfc, which is
> always being updated exactly so that it conforms to the latest
> physical layout rules.
> 
> I personally think this is sort of a shame, because it reminds
> me of other (near) monocultures based on tools that are
> considerably worse than the state of the art (Word is my
> favourite example).

I agree, especially with regard to having the tools
over-constrain early drafts and sometimes thereby making it
harder to expose ideas and get work done.  I think the community
should be pushing back, very hard, on submission tools that
enforce rules that have never been formally announced by the
IESG and/or approved by the community.  But I think it is (or
should be) a separate issue from whether and how we improve the
behavior or robustness of xml2rfc.

   john

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 25 Feb 2011 08:47:19 +1300

> for being imprecise.  I am indeed complaining about the latter and not

For the record, I positively like the facts that the submission tool
carries out basic conformance checks and that 1id-guidelines is
picky.

1. I like this as an author, because it avoids me having to check things
   as a separate step (until the draft is ready for AD review).

2. I like this as a reader and reviewer of drafts, because it leads
   to a very useful degree of uniformity in the way drafts are
   laid out.

And while I'm at it, I like the fact that xml2rfc does a lot of
fiddly stuff for me that I always found a pain in the neck with
other methods.

Yours,
  A Satisfied Customer

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 24 Feb 2011 13:15:11 -0800

> for perfection, so I won't bother again.

Off-list, I asked Henrik which errors the idnits step finds. A summary 
of his answer is "only two non-boilerplate errors (non-ascii characters, 
and lacking an Abstract), and many boilerplate errors (mostly bad IETF 
Trust text, some about the boring first-page boilerplate)". So, I 
suspect that the large percentage of the non-00 drafts getting kicked 
back are due to the IETF Trust requirements changing over time. We can 
argue (and have argued!) about all that, but I think saying "idnits" is 
likely just shooting the messenger.

--Paul Hoffman

---

From: "Worley, Dale R (Dale)" <[email-masked]>
Date: Thu, 24 Feb 2011 17:09:54 -0500

> From: [email-masked] [[email-masked]] On Behalf Of Paul Hoffman [[email-masked]]
> 
> So, I suspect that the large percentage of the non-00 drafts getting
> kicked back are due to the IETF Trust requirements changing over
> time. We can argue (and have argued!) about all that, but I think
> saying "idnits" is likely just shooting the messenger.

I wouldn't be surprised.  And although "idnits" catches the bullets
that should be aimed elsewhere, it is where the problem is discovered.
In a perfect world, we would ensure that the users are educated enough
that they can discern the correct entity to blame and file their
reports correctly.  But in the real world, the target audience turns
over fast enough that (collectively) it cannot learn.  So would be
more effective if the necessary information is presented to the user
at the moment the problem becomes evident.

Dale

---

From: Suzanne Woolf <[email-masked]>
Date: Thu, 24 Feb 2011 18:39:22 +0000

Colleagues,

With respectful apologies to both the posters I'm quoting, I'm going
to reframe this conversation slightly, in hopes of pointing out what I
think we're really talking about, or should be.

> Ah.  Ok.  I'll go tweak some code, then :-)

The whole process needs work. Focusing on a specific piece is useful
on the merits, but also misses an important point.

I'm a long-time IETF participant who's currently editor on a WG draft
for the first time in several years.

The process of submitting the -00 of the WG draft I'm responsible for
was an eye-opener, and not a happy one.

I knew to use xml2rfc, and I'm familiar with the relevant XML, and I
took a lot of the formatting of meta-boilerplate from a previous
generation of the document. However, the document was still rejected
for automatic submission as a -00 for no reason I could immediately
understand.

A big deal? No, it took two people half an hour to muddle through, and
the obvious backup (email submission) was available, and there were
no deadlines immediately at hand.

Harder than it needed to be? I venture to say yes. The mechanics of
submitting the draft, as opposed to the substance of writing it or the
process of getting WG chair pre-approval, was simply opaque even to an
experienced participant who just hadn't attempted this specific task
before.

I can't tell how much of the wrestling with the tools occurred because
it's been a few years and the process had changed so much, how much
that I had to be reminded to use idnits separately (having looked,
perhaps in the wrong places, for a simple "how to"), how much that the
guidelines document is overly-particular, and how much the submission
tool itself. (The problem was a missing version number in the internal
"filename" parameter, which led to a failed match for the pre-approved
name. The submission tool error was "no previous version found". The
idnits error was that the internal filename didn't have a version
number. Putting those together was harder than it had to be,
especially as I was fooled by the fact I was submitting a file named
exactly what we'd agreed it should be.)

As someone trying to make a contribution as editor of a document
needed by a WG to do its work, I don't care exactly where the problem
lies. I don't even care to characterize it as a documentation error, a
tools error, or a UI (confusing output) error. 

I know that the details of making something like this work can be
arcane, particularly as it evolves. My appreciation for Henrik,
Andrew, and everybody else who makes the whole infrastructure function
is sincere.

But wouldn't it be easier for them, and everyone else, as well as for
me and others who don't write several RFCs a year, if the whole
document submission pipeline could be reviewed and perhaps made easier
to work with?

Suzanne