Subject: [rfc-i] RFC Series publishes first RFC with non-ASCII characters

From: "Heather Flanagan (RFC Series Editor)" <[email-masked]>
Date: Thu, 14 Sep 2017 14:07:05 -0700

Hello all,

RFC 8187, "Indicating Character Encoding and Language for HTTP Header
Field Parameters", is the first RFC to be published with UTF-8 encoding
and include characters not in the basic ASCII character set. This
document has been, with the author's consent, patience, and support,
used to test the existing tool chain to produce RFCs to see where the
environment has difficulty in handling non-ASCII characters. The RPC is
continues this testing with the PRECIS document cluster (C326), which is
currently in AUTH48.

The RPC and the Tools Team has identified several areas that need to be
modified to support these characters. Some of those areas will
ultimately be handled with the new format tools; others have been
modified as part of the more general work to prepare for the new RFC
format. For the documents being used to test the toolchain, a
significant amount of manual processing is required to publish the
RFCs with non-ASCII characters in the final text. In order to keep
overall processing times down, leave staff enough time to test
the other tools that are being developed as part of the RFC format
project, and allow the editors time to create and/or update new
procedures as the v3 tools are released, no additional non-ASCII
documents outside of RFC 8187 and the PRECIS cluster will be published
until the new format tools are in production.

Many thanks go out to Julian Reschke and Peter Saint-Andre for their
support, the Tools Team for answering a variety of technical questions,
and to the RPC staff who worked to ensure that the RFC
publication process resulted in readable RFCs.

Heather Flanagan, RSE

_______________________________________________
rfc-interest mailing list
[email-masked]
https://www.rfc-editor.org/mailman/listinfo/rfc-interest

---

From: "HANSEN, TONY L" <[email-masked]>
Date: Fri, 15 Sep 2017 16:18:30 +0000

On 9/14/17, 5:07 PM, "rfc-interest on behalf of Heather Flanagan (RFC Series Editor)" <[email-masked] on behalf of [email-masked]> wrote:

    RFC 8187, "Indicating Character Encoding and Language for HTTP Header
    Field Parameters", is the first RFC to be published with UTF-8 encoding
    and include characters not in the basic ASCII character set. ...
        
    Heather Flanagan, RSE 

Congratulations, Heather, to you, your team, and the authors. This is a big step forward, and I know it took lots of hard work and community involvement to get here.

I think the steps the team took used the right balance of progress and restraint in this important area.

Congratulations again.

	Tony Hansen

---

From: "Patrik Fältström" <[email-masked]>
Date: Fri, 15 Sep 2017 19:39:39 +0200

> I think the steps the team took used the right balance of progress and restraint in this important area.

Indeed! Congratulations!

   Patrik Fältström

---

From: Martin J. Dürst <[email-masked]>
Date: Sat, 16 Sep 2017 09:22:37 +0900

On 2017/09/15 06:07, Heather Flanagan (RFC Series Editor) wrote:
> environment has difficulty in handling non-ASCII characters.

Congratulations to everybody involved! Really great to see this move 
forward!

Regards,   Martin.

P.S.: Looking at the RFC, there were exactly 5 non-ASCII characters (two 
'£'s and one '€' in the examples, and two 'ü's, one in Julian's address 
and one in my name in the acks. Now if I ever need a claim to fame, I 
can claim that my name contributed to the first non-ASCII characters in 
RFCs :-).

P.P.S.: Regarding the content of that RFC, I think may already know, but 
for the record, I'll repeat here that I think it's way outdated. I 
really wish the HTTP WG would take some hints from this RFC work and 
finally get the message that straightforward UTF-8 is the way to go (we 
all knew that ages ago, anyway).

> The RPC is

---

From: "Patrik Fältström" <[email-masked]>
Date: Sat, 16 Sep 2017 06:48:17 +0200

On 16 Sep 2017, at 2:22, Martin J. Dürst wrote:

> P.P.S.: Regarding the content of that RFC, I think may already know, but for the record, I'll repeat here that I think it's way outdated. I really wish the HTTP WG would take some hints from this RFC work and finally get the message that straightforward UTF-8 is the way to go (we all knew that ages ago, anyway).

+1

RFC 2130 from April 1997:

> This report recommends the use of ISO 10646 as the default Coded
> Character Set, and UTF-8 as the default Character Encoding Scheme in
> the creation of new protocols or new version of old protocols which
> transmit text. These defaults do not deprecate the use of other
> character sets when and where they are needed; they are simply
> intended to provide guidance and a specification for
> interoperability.

;-)

   paf

---

From: Tim Bray <[email-masked]>
Date: Sun, 17 Sep 2017 23:09:00 -0700

My hearty thanks to all those who dragged this process into the 21st
century. I might go so far as to say 💓 💓 💓!

Seriously, strong work.

---

From: Jari Arkko <[email-masked]>
Date: Mon, 18 Sep 2017 14:00:27 +0300

Indeed. This was an important and very useful milestone. Thanks, all who helped make it happen. And yes, I’m sure there’s plenty of work ahead too.

Jari

---

From: Julian Reschke <[email-masked]>
Date: Fri, 22 Sep 2017 14:51:46 +0200

On 2017-09-14 23:07, Heather Flanagan (RFC Series Editor) wrote:
> ...
 > ...

Out of curiosity: as the current version of xml2rfc produces the UTF-8 
plain text just fine (*), what additional processing are you referring to?

> ...

Understood.

Now what does this mean for drafts being written right now, and which 
are not expected to be ready for publication in the next, let's say, 12 
months? Can we start submitting I-Ds with non-ASCII characters right now 
(without getting stopped by id-nits...)?

Best regards, Julian

(*) with the possible exception of the BOM :-)

---

From: "Heather Flanagan (RFC Series Editor)" <[email-masked]>
Date: Fri, 22 Sep 2017 09:54:17 -0700

> plain text just fine (*), what additional processing are you referring

There are the manual checks to determine if everything renders exactly
as expected. There are a number of discussions as we learn how to deal
with half- and full-width characters in the same document, and try to
determine the best path forward to make such characters in tables to
line up properly. There are even more discussions as we learn more about
invisible space characters outside of the ASCII realm and try to
understand if those are something we need to write scripts to identify.
xml2rfc is just one component of getting these documents published.

> are not expected to be ready for publication in the next, let's say,
> 12 months? Can we start submitting I-Ds with non-ASCII characters
> right now (without getting stopped by id-nits...)?

That's an excellent question. I think this is something to discuss with
both the IESG and the Tools Team, though I am fairly confident that
"right now" isn't possible. An idnits update is one of the tools
contracted for, and is not yet complete (see the expected timeline here:
https://trac.tools.ietf.org/tools/ietfdb/wiki/FormatToolsPlan). Also,
we've seen that the datatracker isn't rendering non-ASCII characters
correctly in the PDF automatically generated from RFC 8187
(https://trac.tools.ietf.org/tools/ietfdb/ticket/2370). So, there is
still some work to do before this will all work.

-Heather

---

From: Julian Reschke <[email-masked]>
Date: Fri, 22 Sep 2017 19:14:14 +0200

> as expected. There are a number of discussions as we learn how to deal

Ok.

> line up properly. There are even more discussions as we learn more about

Seriously: don't bother.

> understand if those are something we need to write scripts to identify.

If this is understood to be a problem, let's have a script that checks 
for these characters.

> https://trac.tools.ietf.org/tools/ietfdb/wiki/FormatToolsPlan). Also,

Well, idnits s just advisory. All it can block is automatic submission. 
(I wish it did not).

> still some work to do before this will all work.

Looking at the PDF linked from the datatracker 
(<https://www.rfc-editor.org/rfc/pdfrfc/rfc8187.txt.pdf>), I don't see a 
problem.

That said: I don't think that a minor bug in a conversion tool for a 
supplemental output format is sufficient reason to block progress...

Best regards, Julian