Subject: [xml2rfc] Links to rfc7991bis.rnc may need some fixing

From: Martin J. Dürst <[email-masked]>
Date: Wed, 2 Mar 2022 15:24:35 +0900

Dear XML2RFC experts,

I just started recently with a new ID using xml2rfc, after a long 
hiatus. Things went fairly smooth, but one thing that caused problems 
was that there are links for rfc7991bis.rnc that actually go to
https://github.com/ietf-authors/rfcxml-templates-and-schemas/blob/main/rfc7991bis.rnc. 
I naively tried to just download that with "save link as" from my 
browser, but that didn't give me the actual file, just a github page 
including the actual file. It took me quite a while to figure out what 
the resulting error messages meant, and what had happened.

I no longer remember where I originally found such a link, but I have 
found one on https://authors.ietf.org/en/templates-and-schemas, in the 
Schemas section.

Anyway, if you put a link on a filename, make it so that it works for 
direct downloading. Everything else is highly confusing. If you want to 
link to the relevant github page, please add a separate link making 
clear that this link doesn't link to the actual file. Many thanks in 
advance.

Regards,   Martin.

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Mar 2022 08:16:24 +0100

On 2. Mar 2022, at 07:24, Martin J. Dürst <[email-masked]> wrote:
> 
> Dear XML2RFC experts,
> 
> I just started recently with a new ID using xml2rfc, after a long hiatus. Things went fairly smooth, but one thing that caused problems was that there are links for rfc7991bis.rnc that actually go to
> https://github.com/ietf-authors/rfcxml-templates-and-schemas/blob/main/rfc7991bis.rnc. I naively tried to just download that with "save link as" from my browser, but that didn't give me the actual file, just a github page including the actual file. It took me quite a while to figure out what the resulting error messages meant, and what had happened.
> 
> I no longer remember where I originally found such a link, but I have found one on https://authors.ietf.org/en/templates-and-schemas, in the Schemas section.
> 
> Anyway, if you put a link on a filename, make it so that it works for direct downloading. Everything else is highly confusing. If you want to link to the relevant github page, please add a separate link making clear that this link doesn't link to the actual file. Many thanks in advance.

This is also very confusing for me.

The authoritative RFCXMLv3 grammar is at

https://github.com/ietf-tools/xml2rfc/blob/main/xml2rfc/data/v3.rnc
(https://raw.githubusercontent.com/ietf-tools/xml2rfc/main/xml2rfc/data/v3.rnc is the raw version of that.)

This appears to be a precursor to the one at rfcxml-templates-and-schemas, which has less indentation and more comment text.

I also find a version of the v3.rnc that includes grammar for xi:xinclude, which is needed for some tools if you use that feature.

The name rfc7991bis.rnc also seems to relate to various other activities.
There is an expired draft at:

https://datatracker.ietf.org/doc/html/draft-iab-rfc7991bis-03

… which points to the xml2rfc-dev mailing list as its locus of discussion.

While there is no repo information on that draft or its datatracker entry, there is a GitHub repo at

https://github.com/rfc-format/draft-iab-xml2rfc-v3-bis

… which appears to have recent activity.

This includes a

https://github.com/rfc-format/draft-iab-xml2rfc-v3-bis/blob/master/xml2rfcv3.rng

… which is used in the accompanying Makefile to create an xml2rfcv3.rnc, as well as an annotated RNG from text from the I-D in that directory.

I don’t find documentation as to what the intentions with these various copies and versions is, and whether the naming of the files has any relation to those.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Mar 2022 08:41:06 +0100

> I don’t find documentation as to what the intentions with these various copies and versions is, and whether the naming of the files has any relation to those.

Further investigation shows that iab-xmlml2rfc-v3-bis, apart from some unnecessary reorderings, has a few sorely needed fixes:

allows blockquote in dd
allows aside in li
allows country after postalLine

(but it doesn’t address e.g. the <contact> weirdness [1])

plus a bizarre change:

does not allow list in t (!?), so list is now an orphan

Unfortunately, these fixes stay irrelevant to I-D authoring until they also are reflected in v3.rnc in xml2rfc.

Grüße, Carsten

[1]: as addressed in http://tzi.org/~cabo/contact-hack.sh

---

From: Jay Daley <[email-masked]>
Date: Wed, 2 Mar 2022 21:35:46 +1300

Hi Martin

That should be possible.  I believe it means issuing a release of the templates and then linking to the latest release of each file.  I’ll ask one of the devs to help sort that and get authors.ietf.org updated.

Jay

-- 
Jay Daley
IETF Executive Director 
[email-masked]

> On 2/03/2022, at 7:24 PM, Martin J. Dürst <[email-masked]> wrote:
> 
> Dear XML2RFC experts,
> 
> _______________________________________________
> xml2rfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc
>

---

From: Jay Daley <[email-masked]>
Date: Wed, 2 Mar 2022 21:54:36 +1300

> On 2/03/2022, at 8:16 PM, Carsten Bormann <[email-masked]> wrote:
> 

I believe this was explained previously but I’ve lost track of what list so here’s another explanation - the intent is to separate out the authoritative grammar from xml2rfc to support that grammar being used by other tools.  For example, by schema-aware XML editors.  The change management process should now be that changes are agreed by the xml-sg-cmt (later the RSWG/RSAB) and then the grammar is updated in the templates repo and that files is then used in xml2rfc.
> (https://raw.githubusercontent.com/ietf-tools/xml2rfc/main/xml2rfc/data/v3.rnc is the raw version of that.)

After authors.ietf.org goes live, that should no longer be the authoritative version, it should be a copy of the authoritative version.
> This appears to be a precursor to the one at rfcxml-templates-and-schemas, which has less indentation and more comment text.

Correct - the same but more readable.

> I also find a version of the v3.rnc that includes grammar for xi:xinclude, which is needed for some tools if you use that feature.

You will unfortunately find all sorts of versions of the grammar that do not appear to be documented or under any change control.  If you find some documentation on them then please point it out.

Also, what tools out there cannot work with separate XML namespaces (the normal way to enable use of xi:include)?

> The name rfc7991bis.rnc also seems to relate to various other activities.

This has been done to address the issue of stable vs changing grammar files.  The v3.rnc you identified above has changed multiple times since RFC 7991 with limited visibility of those changes.  In future the grammars are to be named after the RFCs that document them and bis is used here to indicate a grammar that deviates from the standard and is still in a state of flux.

> I don’t find documentation as to what the intentions with these various copies and versions is, and whether the naming of the files has any relation to those.

That repo was created last year (or maybe 2020) in order to address this issue of new releases of xml2rfc including new versions of the grammar all called v3.rnc.  It basically recreates the version history of that file in a repo.  

The activity that you see isthe Temporary RFC Series PM (John Levine) building a new grammar that includes all the changes made since RFC 7991 that the xml-sg-cmt have agreed should stay and some additional changes they have made.  John has kept the community up to date on these and solicited feedback.  That new grammar will of course need to be in a draft and hence rfc7991bis.

-- 
Jay Daley
IETF Executive Director 
[email-masked]

---

From: Julian Reschke <[email-masked]>
Date: Wed, 2 Mar 2022 13:31:47 +0100

Am 02.03.2022 um 09:54 schrieb Jay Daley:
> ... > That repo was created last year (or maybe 2020) in order to address
this issue of new releases of xml2rfc including new versions of the
grammar all called v3.rnc.  It basically recreates the version history
of that file in a repo.

2016.

> ...

Best regards, Julian

---

From: "John Levine" <[email-masked]>
Date: 2 Mar 2022 13:32:49 -0500

>does not allow list in t (!?), so list is now an orphan

Yeah, I took it out because <list> is deprecated and has never
appeared in an RFC. There was an earlier argument in which it was
added to <aside> and then removed.

Surely kramdown doesn't generate it.

R's,
John

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Mar 2022 20:22:55 +0100

On 2. Mar 2022, at 19:32, John Levine <[email-masked]> wrote:
> Yeah, I took it out because <list> is deprecated

So is <spanx.
Deprecated v2 grammar is part of the v3 grammar; this change would be the first one from 7991 to be completely taken out.
(I wouldn’t mind having a v3-native grammar at some point, but this single change is premature.)

> and has never

(Well, yeah.)

> Surely kramdown doesn't generate it.

Actually, it does.  The logic in xml2rfc v2v3 to generate the various v3-native elements to do <list is complicated.
I also figured this logic might change over time (but haven’t followed whether it actually did — which saved me a lot of time by the way).
I didn’t see a need to do a poor replica of that logic; after all xml2rfc is the gold standard for v3, so the right thing was to let the gold standard do its thing.

(It would be trivial to update the one case for which kramdown-rfc still generates <spanx in v3-mode, but <list is more work, and more risk.)

Grüße, Carsten

---

From: "John R Levine" <[email-masked]>
Date: 2 Mar 2022 14:59:40 -0500

>> Surely kramdown doesn't generate <list>
>

I believe it, but I also know they all are gone by publication time, and 
when I look at the <list> elements in I-Ds it's pretty straightforward to 
see what kind of list they are.

I can put it back in, but I don't see the point.  The editors will change 
it to something else, why not figure out how to put in the right kind of 
list in the first place?

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

PS: Worst lists I saw:

       <t>Type
               <list>
                       <t>This field is set to the value (TBD1) which indicates that it's a
 					  SR Candidate Path's Path SID sub-TLV.
                       </t>
               </list>
       </t>

       <t>Length
               <list>
                       <t>This field is set to the length of the sub-TLV's Value field in
 					  octets. If Headend and Endpoint fields are in IPv4 address format which
 					  is 4 octets long, it MUST be set to 40; If Headend and Endpoint fields
 					  are in IPv6 address format which is 16 octets long, it MUST be set to 64.
                       </t>
               </list>
       </t>

---

From: Jay Daley <[email-masked]>
Date: Thu, 3 Mar 2022 09:31:11 +1300

> On 3/03/2022, at 8:22 AM, Carsten Bormann <[email-masked]> wrote:
> 
> On 2. Mar 2022, at 19:32, John Levine <[email-masked] <mailto:[email-masked]>> wrote:
>> 

I think we need to be a lot clearer about our use of "deprecated" and how we use version numbers.  

To make sure we’re all on the same page and without wishing to patronise anyone, in a software context, "deprecated" means "A feature that is still available but is targeted for removal and implementors should now be replacing with a different way of dong the same thing.", and pretty much everyone uses semantic versioning (https://semver.org)

From reading RFC 7991 it’s not clear if "deprecated" was used correctly because it reads much more as if the intent was to obsolete those features.  However the deprecated features remain in the grammar, which is as expected.

The inconsistencies are that the only documentation on these features is to list them as deprecated features not to explain how they work, and the tools/processes sometimes allow the deprecated features and sometimes don’t..  This is unusual in a software context where deprecated features are generally still documented and still permitted but labelled as deprecated.

Also, it’s not clear what the plan is for when the deprecated features will be obsoleted.  If we could agree that we are using semantic versioning and that v2, v3 etc are major versions then if the next release of RFCXML is intended to be v3.1 then the deprecated features need to remain but if it is intended to be v4 then they can be removed.

Jay

> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/xml2rfc <https://www.ietf.org/mailman/listinfo/xml2rfc>
-- 
Jay Daley
IETF Executive Director
[email-masked]

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 2 Mar 2022 23:12:24 +0100

Hi Jay,

> I think we need to be a lot clearer about our use of "deprecated" and how we use version numbers.  

This is a very good point.

We are using a transition model that employs the construct of a hybrid v2v3 document (what I have been calling the mongrel format).

Authors can continue to use v2 documents and enhance them with v3 elements and attributes.
xml2rfc —v2v3 converts these transition documents to v3, either with the —v2v3 flag or because a mongrel format document was supplied to —v3.
The v3 grammar therefore describes the mongrel format as well as pure v3.

Nothing about a v3.1 or v4 would change that transition model.  There are still tons of v2 documents out there that will gradually gain v3 functions.
(Given that the transition started on 2020-09-04 with xml2rfc 3.0.0, and I would expect the transition to last some 5 to 10 years, this means we can stop worrying by end-2025 to end-2030.)

Kramdown-rfc is hitchhiking on this by also outputting mongrel documents and using xml2rfc as the source of truth on v3.

Of course, a v3.1 could cut off this transition model.
This would simply be a hostile act to authors who are still using v2/mongrel.

So I think “deprecated” here means: still part of the admissible input (authoring side) but no longer in use on the editing/publishing side.
To better support the latter, it could be useful to have a v3-native grammar in addition to the existing v3 grammar.

Grüße, Carsten

---

From: Jay Daley <[email-masked]>
Date: Thu, 3 Mar 2022 12:11:51 +1300

> On 2/03/2022, at 9:35 PM, Jay Daley <[email-masked]> wrote:
> 
> Hi Martin
> 

This has now been done.  For those interested - the /blob/ was replaced with /raw/ in the path.

Jay

-- 
Jay Daley
IETF Executive Director
[email-masked]

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 3 Mar 2022 00:15:44 +0100

On 3. Mar 2022, at 00:11, Jay Daley <[email-masked]> wrote:
> This has now been done.  

Thanks!

Though not on
Martin Thompson's Markdown Template

> For those interested - the /blob/ was replaced with /raw/ in the path.

Good to know — I’m always using raw.githubusercontent.com and now I’m wondering why.

Grüße, Carsten

---

From: Martin J. Dürst <[email-masked]>
Date: Thu, 3 Mar 2022 08:57:58 +0900

Hello Jay,

Many thanks for your quick action.

Regards,   Martin.

---

From: Jay Daley <[email-masked]>
Date: Thu, 3 Mar 2022 13:18:40 +1300

> On 3/03/2022, at 12:15 PM, Carsten Bormann <[email-masked]> wrote:
> 

Done.

Jay

-- 
Jay Daley
IETF Executive Director
[email-masked]

---

From: "John Levine" <[email-masked]>
Date: 2 Mar 2022 20:02:35 -0500

>Kramdown-rfc is hitchhiking on this by also outputting mongrel documents and using xml2rfc as the source of truth on v3.

The current version of 7991bis matches xml2rfc other than taking out
<list> which I can put back if you really need it, and the optional
<country> after a seqence of <postalline> which should be added
shortly.

Other than that the grammar in xml2rfc hasn't changed since 3.0 two years ago.  Really, it's time to stop using v2.

R's,
John

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 3 Mar 2022 02:23:34 +0100

> On 3. Mar 2022, at 02:02, John Levine <[email-masked]> wrote:
> 
> <list> which I can put back if you really need it,

Yes, please.

> and the optional

That is a very good change.

I hope we can get some of the work on making the grammar more regular and more structured done next, so we can use author names in list items etc.

Besides the grammar work, we also need to work on the less-documented restrictions that are posed outside the grammar, e.g., character set issues.

None of these require terminating the transition prematurely.
Actually, the transition strategy merges nicely into a world where we distinguish format variants less by version number and more by purpose, e.g., authoring, RPC editing, publication.

> Other than that the grammar in xml2rfc hasn't changed since 3.0 two years ago.  Really, it's time to stop using v2.

Yeah, it’s time to stop doing a lot of things…
Or, more precisely to start stopping things.
I did mean it when I said 5-10 years…

Grüße, Carsten

---

From: "John R Levine" <[email-masked]>
Date: 2 Mar 2022 21:25:00 -0500

>> <list> which I can put back if you really need it,

If kramdown puts a <list> into the generated XML, that means that when the 
document is edited, the editor has to manually flag it and try to guess 
which kind of list it is, since published RFCs use the un-deprecated kinds 
of lists.  That doesn't seem like great tooling.

> I hope we can get some of the work on making the grammar more regular 
> and more structured done next, so we can use author names in list items etc.

For v4, perhaps.  As I hope you are aware, v3 has been frozen for two 
years other than a few tiny tweaks like <postalline>.

Regards,

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 3 Mar 2022 03:34:27 +0100

> If kramdown puts a <list> into the generated XML, that means that when the document is edited,

You keep saying that.
The document will have been processed by xml2rfc —v2v3 by then, so the editor will never see the <list.

> the editor has to manually flag it and try to guess which kind of list it is, since published RFCs use the un-deprecated kinds of lists.  That doesn't seem like great tooling.

We already have all the tooling that is needed to generate native v3 for editing and publishing.
It’s just distributed between kramdown-rfc proper, which takes care of the needs of the author, and xml2rfc, which takes care of the needs of the editing of the XML files into the form finally to be published.

> For v4, perhaps.  As I hope you are aware, v3 has been frozen for two years other than a few tiny tweaks like <postalline>.

I honestly don’t know why we are doing this to ourselves.
These are bugs, and bugs need to be fixed.

I’m not sure we have a roadmap for a v4, but then I don’t care at the moment, as v3 is good enough for now ONCE WE FINALLY START FIXING ITS BUGS.

Grüße, Carsten

---

From: "John R Levine" <[email-masked]>
Date: 2 Mar 2022 21:50:05 -0500

> The document will have been processed by xml2rfc —v2v3 by then, so the editor will never see the <list.

RFC 7991bis describes the v3 grammar.   If for some reason you're still 
generating v2, that's still RFC 7749.

> These are bugs, and bugs need to be fixed.

If you believe there are bugs, please submit tickets.

R's,
John

---

From: Jay Daley <[email-masked]>
Date: Thu, 3 Mar 2022 16:23:33 +1300

> On 3/03/2022, at 11:12 AM, Carsten Bormann <[email-masked]> wrote:
> 
> Hi Jay,
> 

If I understand you correctly then your mental model of this is way off mine. The point about the use of deprecated features is about that means for forward compatibility, nothing whatsoever to do with purity.

Here is how I see it (translating your slightly unsavoury language):

pure = a fully v3 compliant document that should be forward compatible with v4 because no deprecated features are used.  In the software world this is normally called ’strict’ format/mode.

mongrel = *also* a fully v3 compliant document but one that uses deprecated v3 features and therefore should not be forward compatible with v4 because those deprecated features should be obsoleted in v4.

For xml2rfc to swap out deprecated features for their intended replacements is just a feature of the tool, nothing more and nothing less.  

For RFCs to be published in strict format (without deprecated features) is simply sensible future-protection and again nothing more and nothing less.  

> Nothing about a v3.1 or v4 would change that transition model.  There are still tons of v2 documents out there that will gradually gain v3 functions.

If v4 obsoleted the v3 deprecated features then yes that would change.  In my view it is poor form to maintain deprecated features into the next major version as it implies that they should not have been deprecated and so v4 should obsolete those features and change the transition model.

v3.1 should not obsolete those features that if the principles of semantic versioning are followed.

> To better support the latter, it could be useful to have a v3-native grammar in addition to the existing v3 grammar.

It makes sense to have a strict version of the grammar (without the deprecated features) for validation by schema validators instead of relying on xml2rfc.  

Jay

-- 
Jay Daley
IETF Executive Director
[email-masked]

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 3 Mar 2022 06:49:13 +0100

On 3. Mar 2022, at 03:50, John R Levine <[email-masked]> wrote:
> RFC 7991bis describes the v3 grammar.   If for some reason you're still generating v2, that's still RFC 7749.

Kramdown-rfc is not setting version=3 on the <rfc element, in order to trigger the v2v3 processing (*) in xml2rfc, so the end result is “pure” v3.
Apart from that, its output should be within the v3 grammar (using v2 compatibility features), maybe sometimes triggering the issues in that grammar.

But kramdown-rfc itself does not directly put out the “pure” v3 grammar, as Jay called it; that is the job of the v2v3 processing in xml2rfc.

> If you believe there are bugs, please submit tickets.

For a while, it wasn’t clear where to submit tickets on v3.rnc that would have advanced it further beyond 7991’s grammar.
But you can start with https://github.com/ietf-tools/xml2rfc/issues/519 and https://github.com/ietf-tools/xml2rfc/issues/570 — I stopped submitting these when it wasn’t clear these would be processed.

(There are a few more non-grammar tickets at https://github.com/ietf-tools/xml2rfc/issues?q=is%3Aissue+is%3Aopen+cabo+ .
Some of these are a bit hard to read because of limitations of the trac-to-github conversion.)

Grüße, Carsten

(*) https://github.com/ietf-tools/xml2rfc/issues/658#issuecomment-1032284086

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 3 Mar 2022 07:10:58 +0100

Hi Jay,

> If I understand you correctly then your mental model of this is way off mine. The point about the use of deprecated features is about that means for forward compatibility, nothing whatsoever to do…

Right.  Not sure if I should switch to “pure” (as below) then.

> pure = a fully v3 compliant document that should be forward compatible with v4 because no deprecated features are used.  In the software world this is normally called ’strict’ format/mode.

The grammar unfortunately does not express the distinction between (1) v3 features that are there for v2 compatibility and will be converted by v2v3, and (2) v3 features that are here to stay and can be published.

It would indeed be good to have a term for the latter — publishable doesn’t quite hit it as that also implies prepped.

(It is somewhat ironic that prepping creates default values for compatibility attributes...)

> mongrel = *also* a fully v3 compliant document but one that uses deprecated v3 features and therefore should not be forward compatible with v4 because those deprecated features should be obsoleted…

Yes.  See my answer to John; I don’t know whether a 7991 grammar compliant document that doesn’t set /rfc/@version to 3 is “fully v3 compliant”.

> For xml2rfc to swap out deprecated features for their intended replacements is just a feature of the tool, nothing more and nothing less.  

Yes.  It is also a brilliant part of the transition strategy.
I can’t trace who came up with this — I suspect Henrik did.

> For RFCs to be published in strict format (without deprecated features) is simply sensible future-protection and again nothing more and nothing less.  

Absolutely (but see the comment about attributes…).

> If v4 obsoleted the v3 deprecated features then yes that would change.  

There is nothing that would stop xml2rfc from having a v2v4 function that translates a mongrel v2..v4 into v4.

> In my view it is poor form to maintain deprecated features into the next major version as it implies that they should not have been deprecated and so v4 should obsolete those features and change the transition model.

There is no need for "v2 but only for compatibility in v3" features to be in the v4 grammar; we just need to ensure that the v2v4 processor can handle these.

> v3.1 should not obsolete those features that if the principles of semantic versioning are followed.

Right.

> It makes sense to have a strict version of the grammar (without the deprecated features) for validation by schema validators instead of relying on xml2rfc.  

v3-strict certainly is a better word than the one I used.
Do you happen to have one for "v2 but only for compatibility in v3” features (list, spanx, t/@hangtext, rfc/@seriesno …)?
(We also need a word for already-deprecated v3 features such as relref; these should not be in v3-strict.)

Grüße, Carsten

---

From: Jay Daley <[email-masked]>
Date: Thu, 3 Mar 2022 20:33:42 +1300

Hi Carsten

> On 3/03/2022, at 7:10 PM, Carsten Bormann <[email-masked]> wrote:
> 
> Hi Jay,
> 

The grammar is never going to do that, it’s only a grammar.  Deprecation is human attribute and so expressed in the documentation

> Yes.  See my answer to John; I don’t know whether a 7991 grammar compliant document that doesn’t set /rfc/@version to 3 is “fully v3 compliant”.

I need to investigate this more but it does seem as if we have two ways to do the same thing in xml2rfc - by setting  /rfc/@version and by the -v3 or -v2v3 switch.  Ideally the version should be set in the document and the switches used to override but it doesn’t sound as if that is the way it works.

> There is nothing that would stop xml2rfc from having a v2v4 function that translates a mongrel v2..v4 into v4.

I don’t see what you mean by a "mongrel v2..v4".  None of the currently deprecated features in v3 should make it into v4 (or if they do then they should not have been deprecated).  

> Do you happen to have one for "v2 but only for compatibility in v3” features (list, spanx, t/@hangtext, rfc/@seriesno …)?

I still think we are using language differently.  Features that were in v2 and are deprecated in v3 are just as much v3 features as the non-deprecated features - deprecation only affects their status for v4 where they should be removed.  Therefore there are only two grammars here

- v3
- v3 strict

> (We also need a word for already-deprecated v3 features such as relref; these should not be in v3-strict.)

Um, "deprecated features"?

Jay

-- 
Jay Daley
IETF Executive Director
[email-masked]

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 3 Mar 2022 08:57:26 +0100

> The grammar is never going to do that, it’s only a grammar.  Deprecation is human attribute and so expressed in the documentation

Relax-NG has the concept of annotations.
I don’t know if there is a conventional way to use this like .feature in Section 4 of RFC 9165, but the basic infrastructure is there.

A grammar without such a mechanism is less helpful in protocol evolution, which is what we are talking about right now.

> I need to investigate this more but it does seem as if we have two ways to do the same thing in xml2rfc - by setting  /rfc/@version and by the -v3 or -v2v3 switch.  Ideally the version should be se…

xml2rfc command line switches:
—v3 switches on v3 processing rules (which has been default since 3.0.0, 2020-09-04), so you get the formatter that has been rewritten for v3.
—v2v3 is something different: it specifies the *output*, i.e. an XML file that is converted to v3-strict; no rendering is done.
The document tells you the format of the *input*.
(A version field has all the typical problems that a version field has, I’m not going to elaborate here.)

> I don’t see what you mean by a "mongrel v2..v4".  None of the currently deprecated features in v3 should make it into v4 (or if they do then they should not have been deprecated).  

As I said, I don’t care.  As long as somebody can take a v2 file, sprinkle in some v3 and v4 features, and get this processed by xml2rfc --v2v4, I’m fine.

Semantics one might attach to words such as “deprecated” and “server” are not relevant for this; this is just a user requirement.

(Again, we should think less in versions and more in processes and actors.)

> Um, "deprecated features"?

Yes, but I’m specifically talking about new RFC 7991 features that did not make it into what we call v3-strict now.

Grüße, Carsten

---

From: Jay Daley <[email-masked]>
Date: Thu, 3 Mar 2022 22:38:43 +1300

> On 3/03/2022, at 8:57 PM, Carsten Bormann <[email-masked]> wrote:
> 
>>> 

I’ve learnt something new - thank you.  My XML editor appears have multiple ways of displaying annotations to provide grammar assistance.

> Semantics one might attach to words such as “deprecated” and “server” are not relevant for this; this is just a user requirement.

AFAICT from reading RFC 7991 there is nothing in v2 that was obsoleted for v3, it was all deprecated (except the PIs but they are ignored) and if that’s correct there is no such thing as a "v2 document with a sprinkle of v3" that is simply a v3 document (even if it claims to be v2 in /rfc/@version).

It sounds as if you want a v4 grammar and a v4-with-v3-deprecated-features included, which defeats the point of deprecation.  When we get to v6, will be have a v6-with-v5-deprecated-and-v3-deprecated-and-v3-deprecated grammar?

> Yes, but I’m specifically talking about new RFC 7991 features that did not make it into what we call v3-strict now.

Now I’m confused.  What features are new to RFC 7991 that would not be in v3-strict given that v3-strict is v3 without any deprecated features?  Was a feature added as a new deprecated feature?

Jay

-- 
Jay Daley
IETF Executive Director
[email-masked]

---

From: Julian Reschke <[email-masked]>
Date: Thu, 3 Mar 2022 12:53:44 +0100

> I’ve learnt something new - thank you.  My XML editor appears have multiple ways of displaying annotations to provide grammar assistance.

Those maybe?

<https://github.com/rfc-format/draft-iab-xml2rfc-v3-bis/blob/a15c9b47ac259d5da6e4b7baaa3f3061ba11e95b/xml2rfcv3-annotated.rng#L24>

:-)

---

From: "John R Levine" <[email-masked]>
Date: 3 Mar 2022 12:35:17 -0500

> The grammar unfortunately does not express the distinction between (1) 
> v3 features that are there for v2 compatibility and will be converted by 
> v2v3, and (2) v3 features that are here to stay and can be published.

xml2rfc has separate grammars for v2 and v3.  In the github repo, look in 
the data directory for v2.rnc and v2.rng vs v3.rnc and v3.rng.

The v3 grammar is the v3 grammar, and we are not going to change it 
gratiutiously.  As many people including you have pointed out, we need to 
make a pass over it and regularize what elements can appear in similar 
contexts, e.g. text or block, but that is only going to allow things in 
addition to what's there now.

Some items like <list> and <relref> have never been used in a published 
RFC so we might as well take them out of the v3 grammar.  There are also a 
small set of things which are used but I think are ill-advised, like the 
complicated postal addresses and the rfc docName attribute.  If we change 
or remove them, we will need a plan to update the published XML so our 
published RFCs continue to match our published grammar, and we're not 
going o do that without a lot of notice.

Regards,

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 3 Mar 2022 18:55:54 +0100

> The v3 grammar is the v3 grammar, and we are not going to change it gratiutiously.  

That is reassuring.

> As many people including you have pointed out, we need to make a pass over it and regularize what elements can appear in similar contexts, e.g. text or block, but that is only going to allow things in addition to what's there now.

Exactly the right approach.

> Some items like <list> and <relref> have never been used in a published RFC so we might as well take them out of the v3 grammar.  

This mail started so good.

Now why are you proposing to take these out gratuitously?
The v3 grammar is not just for publishing RFCs.
It is a shared resource between authoring, editing, and publishing.

> There are also a small set of things which are used but I think are ill-advised, like the complicated postal addresses and the rfc docName attribute.  If we change or remove them, we will need a plan to update the published XML so our published RFCs continue to match our published grammar, and we're not going o do that without a lot of notice.

I think the grammar is the last thing that needs to change while we are stopping these practices (all these elements and attributes are optional).
More important is that the processing that we use is actually realizing these changes at the right point in the chain.

One thing I just noticed is that the RPC now seems to take out the “produced by kramdown-rfc” comments on the way to prerelease/rfcXXXX.notprepped.xml.  This change doesn’t help with statistics.  I don’t know whether this was an oversight, a feature of a new implementation, or a policy that is being applied manually.  The grammar of course does allow these comments.

Another thing we can’t help noticing is that the RPC is using old versions of xml2rfc.  E.g., the fresh AUTH48 draft I’m looking at has the line:
  <!-- xml2rfc v2v3 conversion 3.7.0 -->
Xml2rfc 3.7.0 was 2021-04-05, about 11 months ago (see [1] for more data).
This old version could be the cause of the spurious default-valued attributes on <section and <xref elements, which do not make our lives easier.
The grammar, of course, does allow these noise attributes, but the editing process obviously should not add them.

So the grammar plays a role in a few cases (including its irregular structure as you noted), but it is probably as important to make sure the processing steps actually work well.

A v3-strict grammar could exclude v2-legacy aspects and deprecated v3 aspects such as relref; it might be helpful to have available for tools that want to process published RFCs.

Grüße, Carsten

[1]: https://github.com/cabo/kramdown-rfc/issues/162#issuecomment-1052859560

---

From: "John R Levine" <[email-masked]>
Date: 3 Mar 2022 13:24:50 -0500

> It is a shared resource between authoring, editing, and publishing.

If people actually use these things, sure, we can leave them in.  But the 
point of rfcxml is to provide an archival series, and the fewer bits of 
useless cruft we have to document and support, the better.

> I think the grammar is the last thing that needs to change while we are stopping these practices (all these elements and attributes are optional).

The <postal> stuff depends on an abandonware library that I know is 
already out of date.  The processing is why we need to take it out of the 
grammar.

> Xml2rfc 3.7.0 was 2021-04-05, about 11 months ago (see [1] for more data).

I know their programmer installs new versions within a few days of them 
being released, so that is surprising.  I can check to be sure their staff 
isn't using old copies. but I'm also wondering at what point the 
conversion happened, perhaps when a draft was submitted.

To point out the obvious, the sooner our tools catch up and produce v3 
directly, the fewer problems we will have.  V2 has been obsolete for years 
as you have seen, trying to nurse it along just causes needless confusion.

Regards,

---

From: Jean Mahoney <[email-masked]>
Date: Thu, 3 Mar 2022 13:45:23 -0600

Hi all,

On 3/3/22 12:24 PM, John R Levine wrote:
>>> Some items like <list> and <relref> have never been used in a 
>>> published RFC so we might as well take them out of the v3 grammar.
>>
> If people actually use these things, sure, we can leave them in. But 
> the point of rfcxml is to provide an archival series, and the fewer 
> bits of useless cruft we have to document and support, the better.
>
>>> There are also a small set of things which are used but I think are 
>>> ill-advised, like the complicated postal addresses and the rfc 
>>> docName attribute. If we change or remove them, we will need a plan 
>>> to update the published XML so our published RFCs continue to match 
>>> our published grammar, and we're not going o do that without a lot 
>>> of notice.
>>
>> I think the grammar is the last thing that needs to change while we 
>> are stopping these practices (all these elements and attributes are 
>> optional).
>
> already out of date.  The processing is why we need to take it out of 
> the grammar.
>
>> Another thing we can’t help noticing is that the RPC is using old 
>> versions of xml2rfc.  E.g., the fresh AUTH48 draft I’m looking at has 
>> the line:
> I know their programmer installs new versions within a few days of 
> them being released, so that is surprising.  I can check to be sure 
> their staff isn't using old copies. but I'm also wondering at what 
> point the conversion happened, perhaps when a draft was submitted.

[JM]  The comment line is added when "xml2rfc --v2v3" is run on a 
document and is otherwise not updated.

All comments are stripped before a document is published.

Best regards,

Jean

> directly, the fewer problems we will have.  V2 has been obsolete for 
> years as you have seen, trying to nurse it along just causes needless 
> confusion.
>
> Regards,

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 3 Mar 2022 21:40:03 +0100

Hi Jean,

thanks for giving us some more details.

On 2022-03-03, at 20:45, Jean Mahoney <[email-masked]> wrote:
> 
> [JM]  The comment line is added when "xml2rfc --v2v3" is run on a document and is otherwise not updated.

In the specific case I was looking at, the v3 conversion was actually done by the authors in mid-2021 — there just was a lot of waiting in the RFC editor queue for other documents before it could be edited.  So that explains the 3.7.0.  (Maybe a separate cleaning run is needed when editing is actually begun.)

> All comments are stripped before a document is published.

Which makes them even more useful as a transparency/diagnostic tool, including during the final phases of development (“editing”).

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 3 Mar 2022 21:51:57 +0100

On 2022-03-03, at 19:24, John R Levine <[email-masked]> wrote:
> If people actually use these things, sure, we can leave them in.  But the point of rfcxml is to provide an archival series,

… and supports all the steps towards that.
(Sorry, broken record mode.)

> and the fewer bits of useless cruft we have to document and support, the better.
> 
> The <postal> stuff depends on an abandonware library that I know is already out of date.  The processing is why we need to take it out of the grammar.

Makes sense - this is just a very punctual part of the grammar, so there is less pain than something that affects all text.

> I know their programmer installs new versions within a few days of them being released, so that is surprising.  I can check to be sure their staff isn't using old copies. but I'm also wondering at what point the conversion happened, perhaps when a draft was submitted.

(See my mail to Jean.  So, for this case, the RPC is off the hook, but I’m still shocked by the results of looking at rfc9???.xml.)

> To point out the obvious, the sooner our tools catch up and produce v3 directly, the fewer problems we will have.  V2 has been obsolete for years as you have seen, trying to nurse it along just causes needless confusion.

Yeah.  I hear that.  It’s a bit like IPv4.  And I need to explain occasionally (in about every second email) that kramdown-rfc produces v3 “directly”, with the -c flag, employing xml2rfc as the XML writer.

I would feel much better if I didn’t get the impression that the fact that kramdown-rfc internally uses some v2 is becoming a mental excuse to play down issues that the authors using it do experience.

But above all, we need to get these 133 open issues addressed!

Grüße, Carsten

---

From: "John R Levine" <[email-masked]>
Date: 3 Mar 2022 16:54:45 -0500

> But above all, we need to get these 133 open issues addressed!

We're working on it. got a lot addressed last month.

Regards,