Subject: [Rfc-markdown] New xml2rfc release: v2.25.0

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 26 Aug 2019 05:44:25 -0700

Hi,

This is an automatic notification about a new xml2rfc release, 
v2.25.0, generated when running the mkrelease script.

Release notes:

xml2rfc (2.25.0) ietf; urgency=medium

  This rounds up the remaining known issues raised by the RFC Editor staff.
  Commit log excerpt:

  * Rolled back an earlier requirements change, and added a restriction on 
    pycountry due to a buggy release.

  * Fixed a number of issues with the xml generated for ToC and Index.  This
    makes the ToC output from prepped files the same as from unprepped files,
    which was not the case earlier.

  * Fixed an log() argument error.

  * Modified test input files to silence known issues with legacy rfc xml
    test files, in order to more easily be able to see newly appearing errors.

  * Fixed a string formatting error.  Fixes issue #417.

  * Changed processing progress messages to more consistently obey --quiet, 
    and to be visually distinct from errors and warnings.

  * Modified the PI stripping so as to be able to silence warnings during 
    preptool processing.

  * Added indentation handling for variations of <ol> on request from the
    rfc-editor staff.

  * Moved the check for appropriate <bcp14> content from the text renderer 
    to the preptool, and tweaked it to permit &nbsp;, e.g., 'MUST&nbsp;NOT'.

  * Added a base_url setting to avoid an error message during pdf generation.

  * Added an option --id-reference-base-url to set base url for rendering 
    of <xref> with I-D section references, with a sensible default; and set a 
    default value for --rfc-reference-base-url for <xref> with section= ease of 
    use.

  * Tweaked the conditions for a preptool warning about missing docName to 
    only apply in non-rfc mode, and added generation of any missing <link 
    rel='prev'> element from docName if present.

  * Widened the search for seriesInfo elements when handling the 
    --rfc-reference-base-url option, in order to handle all possible 
    placements, and fixed a bug in the creation of target URL when
    using this option.

  * Added a warning for <vspace> elements without proper v3 alternatives 
    during v2v3 conversion.

  * Fixed a bug introduced in [3201] when stabilizing attribute order,
    which could cause errors when running v2v3 conversion with XInclude
    insertion.

  * Changed the code for --info dump to work for both py27 and py3x.

  * The --legacy-list-symbols option was checked for validity before the 
    version attribute of the input file was seen.  Moved this check (and some 
    similar cases) later, in order to permit it to be used with v3 input 
    without giving the --v3 option.  Fixes issue #414.

 -- Henrik Levkowetz <[email-masked]>  24 Aug 2019 22:02:32 +0000

The preferred way to install xml2rfc is by doing 'pip install xml2rfc',
and 'pip install --upgrade xml2rfc' to upgrade.  If there are system-
installed python modules which pip will not upgrade, you may have to
use 'pip install --upgrade --no-deps xml2rfc' and install dependencies
manually.

The new version is also available through SVN checkout, with
  'svn checkout http://svn.tools.ietf.org/svn/tools/xml2rfc/tags/cli/2.25.0'

Regards,

	Henrik
	(via the mkrelease script)

---

From: Julian Reschke <[email-masked]>
Date: Mon, 26 Aug 2019 16:21:46 +0200

On 26.08.2019 14:44, Henrik Levkowetz wrote:
> ...
> ...

IMHO a bad idea, because now rendering preferences leak into the
content. It means that tools that do something meaningful with <bcp14>
will have to process variations of whitespace characters (just NBSP or
more??).

If the desire is to consistently keep together keywords in output, it
might be a better idea to let the renderer do that automatically.

Whatever the outcome is, this should be precisely described in RFC 7991bis.

Best regards, Julian

---

From: Miek Gieben <[email-masked]>
Date: Mon, 26 Aug 2019 15:45:20 +0100

[ Quoting <[email-masked]> in "[Rfc-markdown] bcp14, was: [xml2rfc..." ]
>Whatever the outcome is, this should be precisely described in RFC 7991bis.

I tend to agree with Julian. This mean I'm better of always outputing "MUST&nbsp;NOT" instead of just
using a normal space.

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 26 Aug 2019 19:14:07 +0200

On 2019-08-26 16:45, Miek Gieben wrote:
> I tend to agree with Julian. This mean I'm better of always outputing
> "MUST&nbsp;NOT" instead of just using a normal space.

It seems to me that the alternative to recognising "MUST&nbsp;NOT" as a
valid keyword, and not emit warnings for it, is to disallow U+00A0 (which
is equivalent to &nbsp; occurring in our XML) in this position.  To me,
that seems worse than the original problem. 

It seems to me that future tools that process our XML should be prepared
to deal with unicode whitespace generally, not only ASCII space.

	Henrik

---

From: Julian Reschke <[email-masked]>
Date: Mon, 26 Aug 2019 19:26:36 +0200

On 26.08.2019 19:14, Henrik Levkowetz wrote:
> . > It seems to me that the alternative to recognising "MUST&nbsp;NOT" as a

Yes. Or warn about it.

> to deal with unicode whitespace generally, not only ASCII space.

Yes, but not necessarily everywhere.

<bcp14> was not added just for rendering, but also so that tools could
extract information about normative requirements.

I just don't see how detecting the keywords will work reliably if it
requires converting an essentially open-ended set of Unicode characters
(given that the set of whitespace characters can be extended).

If the sole reason for this change is to avoid breaking up MUST_NOT (and
friends), why not hard-wire that into the renderers (for HTML, I believe
it could be done with CSS...)? That will at least create consistency,
and avoid yet another task to deal with during AUTH48.

Best regards, Julian

---

From: Fred Baker <[email-masked]>
Date: Mon, 26 Aug 2019 13:36:45 -0700

> On Aug 26, 2019, at 10:14 AM, Henrik Levkowetz <[email-masked]> wrote:
> 

That makes sense to me. The normative language is as seen by the eye and processed by the brain, not as processed by some random computer program. One implication is that it can be broken by a line feed or any other construct that the brain renders as white space.

---

From: Dave Lawrence <[email-masked]>
Date: Mon, 26 Aug 2019 19:06:25 -0400

Julian Reschke writes:
> it could be done with CSS...)?

And I readily admit I might have missed a memo, but if that is the
sole reason then how did we arrive at the conclusion that they needed
to be inseparable?

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 27 Aug 2019 01:46:46 +0200

Hi Dave,

> sole reason then how did we arrive at the conclusion that they needed

They don't.  There's no such rule that I know of, and I don't think there
should be one.

But if an RPC editor decides to make it so in some cases, why should xml2rfc
get in the way and issue an inappropriate warning about unrecognised keywords?

No reason to do so that I can see.  Which is why I made xml2rfc recognize
both "MUST NOT" and "MUST&nbsp;NOT" (and "MUST\tNOT", and "MUST\nNOT", and
some others) as a BCP 14 key word.

	Henrik

---

From: Julian Reschke <[email-masked]>
Date: Tue, 27 Aug 2019 07:44:16 +0200

> sole reason then how did we arrive at the conclusion that they needed

That's a good question as well.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 27 Aug 2019 07:48:33 +0200

> get in the way and issue an inappropriate warning about unrecognised keywords?

It would be good to better understand the requirement. In particular I'd
like to understand why this needs to happen on a case-by-case basis. It
sounds as if historic requirements (line breaking in plain text) leak
back into the canonical source file.

> some others) as a BCP 14 key word.

What about other non-ASCII whitespace characters?

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 27 Aug 2019 08:05:00 +0200

FWIW, opened
<https://github.com/rfc-format/draft-iab-xml2rfc-v3-bis/issues/87>.

Best regards, Julian

---

From: Robert Moskowitz <[email-masked]>
Date: Thu, 29 Aug 2019 15:22:38 -0400

> v2.25.0, generated when running the mkrelease script.

Notes on my experience with the upgrade:

I get the standard warning:

DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 
2020. Please upgrade your Python as Python 2.7 won't be maintained after 
that date. A future version of pip will drop support for Python 2.7.

I have both Python2 and 3 installed on my Fedora 30 system.

What is causing this message?  An old version of pip or xml2rfc?

Is there a way to install xml2rfc v3 to start checking it out without 
loosing v2?

thanks

---

From: Robert Sparks <[email-masked]>
Date: Thu, 29 Aug 2019 14:35:40 -0500

Hi Bob -

The message is coming out of pip, and I suspect your default python is 
currently 2.7.

If you run pip --version, it should tell you which version of python 
it's running on (and consequently which version it is installing 
packages for).

There is only one xml2rfc - it can process both v2 and v3 documents. It 
also runs under either python 2 or python 3.

RjS

On 8/29/19 2:22 PM, Robert Moskowitz wrote:
> 2020. Please upgrade your Python as Python 2.7 won't be maintained 
> after that date. A future version of pip will drop support for Python 
> 2.7.
>
> _______________________________________________
> Rfc-markdown mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rfc-markdown

---

From: Robert Moskowitz <[email-masked]>
Date: Thu, 29 Aug 2019 15:57:03 -0400

> it's running on (and consequently which version it is installing 

Looks like I have a pip, pip2, and pip3 installed.  So a question for 
the Fedora list.

> There is only one xml2rfc - it can process both v2 and v3 documents. 
> It also runs under either python 2 or python 3.

So no surprises?   The big v3 reveal being discussed on the IETF list 
will have no effect on drafts in development?

thanks

>> DEPRECATION: Python 2.7 will reach the end of its life on January 
>> 1st, 2020. Please upgrade your Python as Python 2.7 won't be 
>> maintained after that date. A future version of pip will drop support 
>> for Python 2.7.
>>

---

From: Robert Sparks <[email-masked]>
Date: Thu, 29 Aug 2019 14:59:03 -0500

On 8/29/19 2:57 PM, Robert Moskowitz wrote:
>> The message is coming out of pip, and I suspect your default python 
>> is currently 2.7.
>>
xml2rfc won't try to process your draft as v3 until you change the 
source to say it is v3 source, or you use the --v3 argument.
>>> maintained after that date. A future version of pip will drop 
>>> support for Python 2.7.
>>>
>>> Is there a way to install xml2rfc v3 to start checking it out 
>>> without loosing v2?
>>>
>>> thanks
>>>

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 29 Aug 2019 22:24:19 +0200

Hi Bob,

> I have both Python2 and 3 installed on my Fedora 30 system.

This warning indicates either that the default python is still 2.7, or
that the pip command you are running is symlinked to pip-2.7.

> What is causing this message?  An old version of pip or xml2rfc?

Pip.  The pip that comes with Python 2.7 gives the warning above.

Irrespective of which pip and python you use (2.7, 3.5, 3.6, 3.7),
the installed version of xml2rfc now supports vocabulary v3.

> Is there a way to install xml2rfc v3 to start checking it out without 

xml2rfc v2 and v3 are supported by the same install.  In order to produce
v3 output, use 'xml2rfc --v3 ...'.  In order to convert your v2 xml files
to v3 xml, use 'xml2rfc --v2v3 ...'.

Note that when speaking about 'v3' xml2rfc, there is a certain need to
indicate which aspect of xml2rfc v3 support you are referring to:

There is both the matter of the v3 xml vocabulary, where many v2 constructs
are deprecated; and there are the v3 formatters, which are completely
rewritten to handle the v3 xml.

Now, having made that clear, I'm afraid that the following may cause some
blurring again, but it's been done in the interest of making forward progress
easier:  If you invoke one of the v3 formatters, by saying for instance
'xml2rfc --v3 --text ...', but your input file uses deprecated v2 elements,
those will be quietly converted internally, using the v2-to-v3 converter,
and then handed to the v3 formatter.  This of course adds a bit of magic
to the process, and you will have better control of your output if you work
with clean v3 input files.  Converting your v2 files to v3 with --v2v3 should
give you a good starting point for that.

I hope that helps.

Best regards,

	Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 29 Aug 2019 22:26:03 +0200

I see Robert's responses landed while I was typing my response ,:-)

On 2019-08-29 22:24, Henrik Levkowetz wrote:
> xml2rfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/xml2rfc
>

---

From: Robert Moskowitz <[email-masked]>
Date: Thu, 29 Aug 2019 16:49:03 -0400

> the installed version of xml2rfc now supports vocabulary v3.

I just asked on the Fedora list.  I am suppose to be running pip3, not 
just pip.  There is also a  pip2, but don't use it as it is the same as pip.

The default version is still 2.7:

$ python --version
Python 2.7.16

But Python3 is installed.  I am going to have to dig more.  Just for my 
own 'sanity'.  Ha!

> v3 output, use 'xml2rfc --v3 ...'.

I tried this with both drafts that are 'ready' and both gave me the warning:

draft-moskowitz-hip-hierarchical-hit-00.xml(14): Warning: Setting 
consensus="true" for IETF STD document (this is not the schema default, 
but is the only value permitted for this type of document)

I need to find where 'consensus' is explained.

> In order to convert your v2 xml files

 From what you say below, this is something I should do.  I also foresee 
a possible challenge in that I do a lot of copy and paste from old 
drafts to new, and I could end up dragging something into converted 
draft that is not for v3 from an unconverted old file.

Actually I have a number of drafts in various stages.  Ugh.

---

From: Julian Reschke <[email-masked]>
Date: Fri, 30 Aug 2019 06:21:27 +0200

On 29.08.2019 22:49, Robert Moskowitz wrote:
> ... > I tried this with both drafts that are 'ready' and both gave me the
> warning:
>

<https://greenbytes.de/tech/webdav/rfc7991.html#element.rfc.attribute.consensus>

FWIW, I'm not sure the Warning is correct since this is still an
Internet Draft.

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 30 Aug 2019 06:43:31 +0200

> <https://greenbytes.de/tech/webdav/rfc7991.html#element.rfc.attribute.consensus>

I’m not sure whether you are serious when you say the attribute is “explained” there.
Yes, there is text about this attribute.
No, it is not explained.
The reference to 7841 just points to more text about “consensus” there, but not about the meaning of this attribute.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 30 Aug 2019 06:46:01 +0200

On Aug 29, 2019, at 22:49, Robert Moskowitz <[email-masked]> wrote:
> 
> I also foresee a possible challenge in that I do a lot of copy and paste from old drafts to new, and I could end up dragging something into converted draft that is not for v3 from an unconverted old file.

For the next decade or so, the v2v3 avenue is going to be the one that many drafts will use.
Going for v3 only source gives you more control (maybe; some of this control is wrested away from you in the preptool), but has little practical value to most RFC authors in an organization where documents take 3 to 7 years to gestate.

Grüße, Carsten

---

From: Julian Reschke <[email-masked]>
Date: Fri, 30 Aug 2019 06:53:38 +0200

> The reference to 7841 just points to more text about “consensus” there, but not about the meaning of this attribute.

There's more text at
<https://greenbytes.de/tech/webdav/rfc7991.html#attribute-consensus>;
and the main definition should reference that:

> A.3. The "consensus" Attribute
>
> For some of the publication streams (see Appendix A.2), the "Status of This Memo" section depends on whether there was a consensus to publish (again, see Section 3.4 of [RFC7841]).
>
> The consensus attribute can be used to supply this information. The acceptable values are "true" (the default) and "false"; "yes" and "no" from v2 are deprecated.
>
> The effect of this value for the various streams is:
>
>     "independent": none.
>     "IAB": mention that there was an IAB consensus.
>     "IETF": mention that there was an IETF consensus.
>     "IRTF": mention that there was a research group consensus (where the name of the research group is extracted from the <workgroup> element).

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 30 Aug 2019 06:57:37 +0200

>> A.3. The "consensus" Attribute

That’s more useful, indeed.  Thanks.

Grüße, Carsten

---

From: Robert Moskowitz <[email-masked]>
Date: Fri, 30 Aug 2019 05:51:09 -0400

On 8/30/19 12:53 AM, Julian Reschke wrote:
>> The reference to 7841 just points to more text about “consensus” 
>> there, but not about the meaning of this attribute.
>
> There's more text at
>> For some of the publication streams (see Appendix A.2), the "Status 
>> of This Memo" section depends on whether there was a consensus to 
>> publish (again, see Section 3.4 of [RFC7841]).
>>
>> The consensus attribute can be used to supply this information. The 
>> acceptable values are "true" (the default) and "false"; "yes" and 
>> "no" from v2 are deprecated.
>>
>>     "IRTF": mention that there was a research group consensus (where 
>> the name of the research group is extracted from the <workgroup> 
>> element).
>
> Best regards, Julian

Oh, Dear Lord, this is such a wonderful addition that captures the state 
of work.  It just calls out for an April 1 RFC!!!!!

Is if fine grain enough.  What about design team consensus, or work 
group consensus, to Webster's definition of consensus to Oxford 
definition of consensus (boy was that a fun meeting of US and UK people 
back in '95!: "We have consensus here." "What are you saying, we do not 
consent to this proposal!").

Then if there is a consensus attribute, where is the running_code 
attribute.  How many interoperable versions?

Oh the potential!!!!

Carsten and Julian, are you up for an RFC on this one?    :)

I would love to see your German take on the US vs UK discussions on 
tabling...

Bob

---

From: Julian Reschke <[email-masked]>
Date: Fri, 30 Aug 2019 12:02:45 +0200

> of work.  It just calls out for an April 1 RFC!!!!!

FWIW, this is not new at all. It has been around for at least nine years.

What's new is just the (IMHO confusing) warning message.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Fri, 30 Aug 2019 15:19:03 +0200

> and avoid yet another task to deal with during AUTH48.

FWIW; this could be done with "white-space: nowrap" (see
<https://developer.mozilla.org/en-US/docs/Web/CSS/white-space>).

Best regards, Julian