Subject: Re: [Rfc-markdown] [Tools-discuss] Xml2rfc regression?: rfc2629.dtd (emergency fix in 1.5.25)

From: Carsten Bormann <[email-masked]>
Date: Tue, 25 Jan 2022 07:24:03 +0100

Hi Kesara,

thanks for the super-quick response!

> On 2022-01-25, at 03:17, Kesara Rathnayake <[email-masked]> wrote:
> 
> Note that `SYSTEM` inclusions are allowed without the new flag, as long as the included file is in the templates directory.
> The templates directory can be configured by using `--template-dir`.

There is indeed an rfc2629.dtd under
lib/python3.9/site-packages/xml2rfc/templates
(I’m assuming that this is where my xml2rfc thinks its (default) template directory is.)

However, there also is an rfc2629.dtd in the local directory where the draft’s .xml is (this is needed so we can extract elements from the xml without xmlstarlet complaining).  Xml2rfc apparently finds the local one but then decides it shouldn’t allow access to it, instead of using the template one.

So, essentially, almost all builds that use xml tools for extracting components from the XML file are now broken.

I did an emergency fix in kramdown-rfc 1.5.25 that substitutes

<!DOCTYPE rfc [
  <!ENTITY nbsp    "&#160;">
  <!ENTITY zwsp   "&#8203;">
  <!ENTITY nbhy   "&#8209;">
  <!ENTITY wj     "&#8288;">

for

<!DOCTYPE rfc SYSTEM "rfc2629.dtd" [

if --v3 (kdrfc -3) is given.
This leaves v2 high and dry, but those files are more likely to use the predefined character entities and less likely to use additional XML tooling and this a local rfc2629.dtd.
The emergency fix also makes the predefined character entities (except the four that now seem to be canonical for the RFC editor) unusable with v3 — this is a breaking change (I didn’t check how many drafts this breaks), but ultimately possibly a good one.  If you need a workaround, add:

doctype-reference: 'SYSTEM "rfc2629.dtd"'
or
doctype-reference: 'SYSTEM “rfc2629-xhtml.ent"'

to the YAML (and make sure you don’t have a local copy in the directory).

(FYI: When I looked today in /archive/id, 
of about 53451 XML I-Ds, approximately 
49185 XML I-Ds contained matches for
<!DOCTYPE rfc SYSTEM "rfc2629.dtd”
or similar and approximately 1870 contained matches for
<!DOCTYPE rfc SYSTEM "rfc2629-xhtml.ent”>
which probably has the same problem with a local copy.
So this is not a theoretical concern.)

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 25 Jan 2022 08:35:31 +0100

> On 25. Jan 2022, at 08:00, Martin Thomson <[email-masked]> wrote:
> 
> On Tue, Jan 25, 2022, at 17:24, Carsten Bormann wrote:
> Thank you.  Consider making that permanent.  

I think with the “doctype-reference” escape valve the fix can stay in.

> Having to build an XML catalogue just to do simple XML processing (extracting author names, say) is an annoying hurdle to clear.  

I agree; kramdown-rfc just kept chugging along with most of the first ~50000 XML drafts out there, and this is now gone.

> Inline declarations for entities is probably unnecessary for the output of a tool (UTF-8 does just work), but I do find it easier to distinguish various types of space if entities are used when it isn't U+20.

Yes.  I wish Emacs were better in showing these types (of course, I'll just need to configure it right :-).

I have no idea what made the four entities that are now inserted by the v2v3 conversion so special; these of course highlight the inconsistent application of RFC 7997.
(Ceterum censeo: I want my dashes…)

> Can you push your change please?

Sorry, done.

Grüße, Carsten

---

From: Kesara Rathnayake <[email-masked]>
Date: Wed, 26 Jan 2022 14:39:10 +1300

> So, essentially, almost all builds that use xml tools for extracting components from the XML file are now broken.

I have captured this bug here: 
https://trac.ietf.org/trac/xml2rfc/ticket/703 and working on a fix.
I think this was a long-existing bug where xml2rfc looked at the source 
directory first instead of the templates directory and only surfaced 
because of recent local file access restrictions.

   --Kesara

> _______________________________________________
> Rfc-markdown mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rfc-markdown

-- 
Kesara Rathnayake
Senior Software Development Engineer - IETF LLC
[email-masked]