Subject: Re: [art] New RFCs text formatting

From: John C Klensin <[email-masked]>
Date: Thu, 28 Nov 2019 21:22:48 -0500

--On Thursday, 28 November, 2019 20:05 +0100 Julian Reschke
<[email-masked]> wrote:

> On 28.11.2019 17:33, John C Klensin wrote:
>> ...
>> [1] The concern is about user-developed tools, used by one
>> person or a cluster of people who have shared them, not ones
>> developed or maintained under IETF auspices.
> 
> Do you have a concrete case of a tool that fails with the new
> format?

Julian, I think that is precisely the wrong question to be
asking.  My note indicated that the concern was about private
tools, not ones that the IETF community has necessarily ever
been told about. I'll answer the question anyway, which is that
I have such tools, mostly written for an emacs clone as either
macros or in its proprietary extension language.  Some of them
predate even the first instance of XML2RFC.  A subset of those
include some that have not been used recently but that were used
to write drafts for RFCs either direct-to-text or using private
macros for nroff or similar format markup systems that were then
converted to text using those systems before submission. Others
have been in active use more recently.  One got into trouble
with RFC RFC 8689 because, for some reason, it tried to do a
search on "[Page" or some regex equivalent.  I don't know how
important that search is or how hard it would be to replace (I
haven't looked at the internals of most of those in 20-odd
years) but the point is that this is a more disruptive change
than some of would have predicted.

My response to Paul's comment is similar, again with the
observation that it may be much too late to do anything other
than maybe learn a lesson: it really does not make any
difference whether I can find a specific promise or not.  I
believe the community of people who work with RFCs (not just the
subset who have been designing and implementing v3 or the
overlapping subset who design and implement too for the
community) were led to believe this was going to be a
non-disruptive change.  We were told that we could keep working
in v2 (with whatever v2-prep tools we had) and submitting that
way and the Production Center would sort things out, and, while
v3 would bring many advantage to the Production Center, those
who wanted to use it, and people who preferred reading RFCs in
more "modern" formats, no one needed to make changes in how they
were doing things.  In addition to the positive things that have
been said about v3, to the best of my knowledge at least, the
community was never told "this is going to result in some
incompatible changes that you will need to deal with, are those
changes acceptable for the benefits they will bring".  In
itself, the absence of that question strongly implies only
changes that are non-disruptive except to those who choose to
use new features and to the Production Center.

Well, causing old tools, even non-public tools, to not work as
they did before and without changes to those tools is a
disruptive change.

>> [2] Reading RFC 7994 only after these changes were
>> identified, I find that Sections 4.1ff remove headers and
>> footers and the page numbers from the TOC.  But the
>> introductory paragraph of Section 4 does say
>> 
>> 	"One plain-text output will be created during the
>> 	publication process with basic pagination that includes
>> 	a form feed instruction every 58 lines at most,
>> 	including blank lines."
>> 
>> So perhaps that omission is a bug.
> 
> AFAIR, the intent was to *paginate*, nothing more (yes, no
> headers,
> footers, no page numbers).

That is certainly what 7994 says.   To me, the important
question is whether publication in that (and other) documents is
sufficient to warn the community about changes that may be
incompatible with tools or working procedures used by
contributors.  If the answer is "yes", then every one of us is
responsible for being thoroughly familiar with every RFC or
announcement that might affect IETF procedures, publications,
and other ways in which we do things so there is an opportunity
to raise timely objections.   I don't think we want to go there
but should instead assume that people in the community should be
warned by explicit questions and requests for consensus before
incompatible changes are made.  YMMD, of course,

   best,
    john

---

From: Julian Reschke <[email-masked]>
Date: Fri, 29 Nov 2019 06:01:49 +0100

> they did before and without changes to those tools is a

John, with all due respect, there's a reason I asked, so for me this
question definitively is not the "wrong" question.

When we had these discussions many years ago, the assumption was that
tools that operate on plain text RFCs would strip pagination first (or
ignore it altogether) before doing what they need to do next. I would
like to understand whether this assumption was wrong, and how *exactly*
tools now break.

> incompatible changes are made.  YMMD, of course,

Heather has sent out mails, published FAQs, explained things at the
plenary, and had a poster at the RFC desk for a long time. I would say
this was hard to miss.

Best regards, Julian

PS: I have many other issues with the V3 switch, mainly that we are now
publishing "canonical" versions in a undocumented and un-reviewed
format. In comparison to that, the changes in the produced text format
currently seem to be a minor issue.

---

From: John C Klensin <[email-masked]>
Date: Fri, 29 Nov 2019 09:31:00 -0500

> like to understand whether this assumption was wrong, and how

Ok, that is fair.  As can't speak for others but, as I think I
said in my earlier note, some of my tools do searches on
"[Page".  I don't remember why but suspect it may have had to do
with skipping over boilerplate or the like when those formats
were changing.  I don't know if my tools have other
sensitivities (see below) and won't know until I try to use them
and see what (if anything) breaks or do a careful code search.
I'm also not claiming that it would be hard to change those
tools --I just don't know without looking -- just that making me
(and others) spend time adapting is possibly not in the best
interests of the IETF if the total amount of time someone can
spend on IETF work is limited and that adaptation time takes
away from doing technical work.

>> question is whether publication in that (and other) documents
>> is sufficient to warn the community about changes that may be
>> and other ways in which we do things so there is an
>> opportunity to raise timely objections.   I don't think we
>> want to go there but should instead assume that people in the
>> community should be warned by explicit questions and requests
>> for consensus before incompatible changes are made.  YMMD, of
>> course,
>> ...
> 

Most of those messages have said that v3 is coming, there are
big changes, throughput of the Production Center is likely to
slow down.  And, fwiw, I've seen the "RFC desk" twice in about
three years and, for personal (but IAB/IETF-induced) reasons,
have been somewhat tuned out).  AFAICT, none of those messages
have clearly said "if you are dependent on pagination and/or
per-page headers or footer, get over it", at least without
having that information buried (however unintentionally) in a
lot of material that was basically noise.

> PS: I have many other issues with the V3 switch, mainly that
> we are now publishing "canonical" versions in a undocumented
> and un-reviewed format. In comparison to that, the changes in
> the produced text format currently seem to be a minor issue.

What seemed to be a high likelihood of something like this
happening was one of several reasons why some of us were
concerned about, and opposed to, making the XML format the
canonical/archival one.  In this case, that concern was that,
once actual deployment started, the community would discover a
need to make adjustments to what was basically an untested
specification.   The need to debug something that had previously
been tested only among the developers and other very interested
parties and to make adjustments as the code was developed and
evolved should come as a surprise to no one with any experience
in programming or product deployment.  Having the documentation
be somewhat out of synch with that is no surprise either.

Whether it is a minor detail or not depends on how the community
reacts to the change.  As I understand the announcements that
have been made, people have the option of submitting text and
having the Production Center sort of the details.  So, suppose
that, either because of the interactions with existing tools or
because v3 is a someone-undocumented or under-documented moving
target, some of us drop back to preparing documents using nroff
(or one of its descendants), MS Word, or text directly and then
submit the text files?  You may know things that I don't, but
I'd assume that would either require much more staff in the
Production Center or would slow document throughput to a trickle
for a long time.  I would consider that a much more major issue
that either the tools issue or the undocumented and un-reviewed
format ones that feed into it.

As usual, YMMD.

   john

---

From: Keith Moore <[email-masked]>
Date: Fri, 29 Nov 2019 11:13:28 -0500

> like to understand whether this assumption was wrong, and how *exactly*

Yes, the assumption was wrong.

And it's not possible to enumerate exactly how all of the tools everyone 
is using now break.   But more importantly, that's the wrong question to 
ask.

Many of us realize that when we revise deployed protocols, it's better 
to NOT to make assumptions about which obscure features of deployed 
protocols people depend on.   Instead we try to maintain strict 
compatibility when possible, because we realize that we can't reliably 
know about all of the assumptions that are embedded in existing 
implementations.    Sometimes it's necessary to break strict 
compatibility, but arguments of the form "nobody depends on feature X" 
are always dubious and should be interpreted as red flags.

For better or worse, the legacy text RFC format is a widely deployed 
protocol.   And while most people these days are probably not using this 
feature, there are actually quite a few modern printers out there that 
understand plain text, including form feeds, and also several software 
programs that paginate text files based on form feeds.

(Expecting everyone out there to use Windows is not only incorrect, it's 
also insulting.)

Keith

---

From: Julian Reschke <[email-masked]>
Date: Fri, 29 Nov 2019 18:07:32 +0100

> Yes, the assumption was wrong.

FWIW, I think this is something the RSE should answer.

 From my recollection:

1) We did not require the new implementation to do proper pagination,
because we didn't want people to refer to page numbers in RFCs anymore.
Keeping them would be encouraging that practice, and would be in
conflict with the goal for RFCs to be paginable (is that an English
word?) according to their preference. (Device, paper size, font size etc)

2) We also expected that pagination in the plain text output would
require complexity in the formatter, and we wanted to avoid that. That
point is moot now, as the developer decided to implement it anyway.

So if you actually want a plain text RFC with "proper" pagination, take
the "canonical" XML and run it through xml2rfc with the proper options.

> programs that paginate text files based on form feeds.

For me, printing plain text RFCs never ever made sense, because on DIN
A4, they leave ~25% of the page unused. So please don't assume everybody
uses letter-sized paper.

I also don't quite get why you are mentioning printers at all. Are you
saying you can't print the HTML variant?

> (Expecting everyone out there to use Windows is not only incorrect, it's

Not sure what this has to do with the operating system.

In any case, this really sounds like the comeback of a discussion we had
~7 years ago (with probably thousands of emails), leading to the
publication of RFC 6949.

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 29 Nov 2019 18:29:06 +0100

> ~7 years ago (with probably thousands of emails), leading to the

Yes!

We are now starting to gain actual experience with the brave new world instead of just random guessing how that will feel (with a heavy dose of process confabulation at the time).  In software, obtaining end-user experience generally leads to iterations.

Grüße, Carsten

---

From: Keith Moore <[email-masked]>
Date: Fri, 29 Nov 2019 12:46:08 -0500

> because we didn't want people to refer to page numbers in RFCs anymore.

I can sort of see that, because if different people are referring to 
different versions of the RFC, the page numbers don't work.   I 
definitely prefer using section numbers for reference.   At the same 
time, unpaginated RFCs cause problems other than those caused by 
omitting page numbers.

> the "canonical" XML and run it through xml2rfc with the proper options.
Sure, but now I have to go through extra steps, and I have to treat new 
RFCs differently than old ones.
> A4, they leave ~25% of the page unused. So please don't assume everybody

Oh, I don't.   But even if, say, you print RFCs 2-up sideways on A4 
paper, the pagination is still useful.  (Besides, one person's wasted 
space on the page is another person's ample space for notes.)

Printing RFCs makes a lot of sense to me, in some circumstances.   It's 
much easier to make notes on paper than on an HTML or PDF document.   I 
can read paper more easily in coffee shops or on airplanes than I can on 
a laptop.   If I'm writing code based on an RFC, I can print out the RFC 
and refer to it without the RFC taking up screen space.   On the other 
hand, a great many more documents can be carried on a laptop than in 
paper form.   So, I routinely use different RFC formats depending on the 
circumstances.

> saying you can't print the HTML variant?

I never use the HTML variants produced by the RSE.  The type is too 
small, the sans serif typeface is less legible than Courier. On a normal 
width browser window the small type used makes the lines too long to 
read.   And yes, I can narrow the browser window to make the lines 
shorter, but the result "looks" wrong and then I'm constantly resizing 
the browser if I need to view other pages.   The "HTMLized" RFCs 
produced by the IETF tools are a bit better for my purposes.   It's not 
that I have a great love for fixed-pitch type or paginated text on a 
scrolling display, but the RSE HTML versions aren't really making the 
best use of either HTML or the browser environment.

I also don't want to use a web browser to read RFCs in most 
circumstances, because web browsers are themselves pretty dysfunctional 
for document review and not great even for occasional reference when 
writing specifications or code.   They waste too much screen space, they 
make it cumbersome to arrange one's screen space effectively.   Also 
with a typical desktop, when you click on an editor or word processor to 
write code or make notes, it often buries the very browser window you're 
referring to, unless maybe you have so much screen space available that 
you can completely avoid any overlap of windows.  Of course, it's not 
the fault of the RSE that modern desktops and web browsers are so 
dysfunctional, and web browsers aren't all they're cracked up to be.

> Not sure what this has to do with the operating system.

All of the other modern desktop OSes I've used have had little or no 
problem dealing with ordinary text files, even if the default 
end-of-line convention was different.  Windows is the only one I've seen 
that was gratuitously, and aggressively, incompatible.

Keith

---

From: Julian Reschke <[email-masked]>
Date: Fri, 29 Nov 2019 20:03:02 +0100

> We are now starting to gain actual experience with the brave new world instead of just random guessing how that will feel (with a heavy dose of process confabulation at the time).  In software, obt…

Yes, the transition takes too much time. It should have been done
several years ago, but it's still not finished (despite claims that it
is). I really wish that energy would go into finishing the actual format
conversion, and not bike-shedding the text format :-).

Keep in mind that it's a compromise. At the one extreme there were
people asking for no change at all (thus, no non-ASCII). At the other
end of the spectrum, people proposed no page-breaking and no
line-breaking (seriously).

What's currently generated is really very close to what we had before.

Best regards, Julian