Subject: I-D file formats and internationalization

From: Robert Sayre <[email-masked]>
Date: Tue, 29 Nov 2005 20:38:31 -0500

I've noticed that the recent debate on the ASCII text format has often
conflated formatting of artwork and Unicode support. I think finding a
non-text artwork format that has free uniform authoring (including
diffs) and viewer support will be impossible for the next 5-10 years.
An XML equivalent to Postscript may eventually be widely implemented.
The current effort, SVG, is a massive specification, unevenly
implemented, and lacks a thorough test suite.

Unicode support is a different matter. I find the current IETF policy
to be incredibly bigoted. Many RFCs and I-Ds are currently forced to
misspell the names of authors and contributors, which doesn't seem
like correct attribution to me. So, I recommend that the IETF
secretariat and the RFC Editor change their policies to allow UTF-8
text files. That way, older RFCs and I-Ds produced using the current
tools would follow the same encoding.

I'm sure someone has already suggested this approach, but I'll add my
voice to the chorus.

Robert Sayre

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Ole Jacobsen <[email-masked]>
Date: Wed, 30 Nov 2005 08:28:44 -0800 (PST)

Robert,

This is a good point. It even applies to the IETF secretariat. It used to 
be impossible to register with your "real name" if it contained non-ASCII
characters. I think that has changed, I recall having Seen Olafur 
Gudmundson's badge with the real Icelandic "curly d" (or whatever it is 
called in English) at a recent meeting. I have not seen Japanese or 
Chinese or Korean, which I guess would be the next logical step...

Ole

Ole J. Jacobsen 
Editor and Publisher,  The Internet Protocol Journal
Academic Research and Technology Initiatives, Cisco Systems
Tel: +1 408-527-8972   GSM: +1 415-370-4628
E-mail: [email-masked]  URL: http://www.cisco.com/ipj

---

From: Eduardo Mendez <[email-masked]>
Date: Wed, 30 Nov 2005 19:05:13 +0100

It was sad that people can accept this.
To degrade  the name of their friends.
In every country this is insulting.

It is good news you can type better.
But this has not changed in RFC.
If in the "thanks" section you hurt a name.
The "thanked" person, will not be happy.

Eduardo Mendez

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 30 Nov 2005 21:41:44 +0100

Robert Sayre wrote:

> I'm sure someone has already suggested this approach,
> but I'll add my voice to the chorus.

<http://tools.ietf.org/html/draft-hoffman-utf8-rfcs>

I really don't like this approach for various reasons.

                    Bye, Frank

---

From: Douglas Otis <[email-masked]>
Date: Wed, 30 Nov 2005 13:54:49 -0800

> I really don't like this approach for various reasons.

Rather than opening RFCs to text utilizing any character-set  
anywhere, as this draft suggests, there could be alternative UTF  
fields for an author's name and reference titles, and perhaps defined  
characters for simple line and table drawing that invoke automatic  
translation when an ASCII text version is generated.  Being able to  
review the ID as it would appear as an RFC would also seem to be a  
requirement.  It seems problematic for protocol examples to use non- 
ASCII characters owing to there not being ubiquitously displayable  
character-sets.

-Doug

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 30 Nov 2005 14:23:27 -0800

At 1:54 PM -0800 11/30/05, Douglas Otis wrote:
>anywhere, as this draft suggests,

That is not what the RFC suggests at all. The character set is 
Unicode. The encoding is UTF-8. That's it.

>  there could be alternative UTF fields for an author's name and 
>reference titles, and perhaps defined characters for simple line and 
>table drawing that invoke automatic translation when an ASCII text 
>version is generated.

That's a possibility (if you define what an "alternative UTF field" 
is). Why is it better than simply using UTF-8 everywhere?

>Being able to review the ID as it would appear as an RFC would also 
>seem to be a requirement.

That means changing the Internet Drafts process as well. Certainly 
possible, but more daunting that changing one process at a time.

>   It seems problematic for protocol examples to use non-ASCII 
>characters owing to there not being ubiquitously displayable 
>character-sets.

Unicode is universally displayable if you have the right font(s). 
Regardless of that, however, any sane document author would not 
assume that every person reading the document could display it. They 
would put a legend or explanation near the example.

--Paul Hoffman, Director
--VPN Consortium

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 30 Nov 2005 23:54:14 +0100

Douglas Otis wrote:

> there could be alternative UTF fields for an author's name
> and reference titles,

For the new 3066bis language tags registry we adopted the
known &#x12345; notation for u+012345.  As Bob said raw UTF-8
characters won't fly with `cat rfc4567 > /dev/lpt1` and other
simple uses of RFCs.

> perhaps defined characters for simple line and table drawing

Ugh.  That's a case where I'd prefer PDF or something better.
"ASCII art" is one thing, IMHO it's cute.  But "line drawing"
char.s are a PITA:  my local charset pc-multilingual-850+euro
still has this, today it's just crap, let's forget it please,
"codepage 437" etc. and curses ACS_* are ancient history.

Ok., in theory Unicode has these characters, but if we'd really
want this we could as well jump from plain text to MathML.

> characters owing to there not being ubiquitously displayable

Yes, OTOH I vaguely recall one of Martin's (?) texts, where his
attempt to talk about non-ASCII issues in ASCII wasn't straight
forward, to put it mildly, and it certainly wasn't his fault.

Similar texts published by the W3C using UTF-8 are even worse
from my POV (both with a pre-UTF-8 browser and otherwise, for
the latter I just don't have the required fonts).

                             Bye, Frank

---

From: Douglas Otis <[email-masked]>
Date: Wed, 30 Nov 2005 17:59:08 -0800

> Unicode. The encoding is UTF-8. That's it.

Unicode provides a unique number for every possible character within  
a current range of about 97,000 characters.  These characters include  
punctuation marks, diacritics, mathematical and technical symbols,  
arrows, dingbats, etc.  Displaying one of these characters requires a  
character-set (synonymous with a display system's font-set or  
character-repertoire), or using the unicode vernacular, a script.  It  
is not just a matter of which character is displayed, which character- 
repertoire is used, but there are also Middle Eastern right-to-left  
issues as well.

>> reference titles, and perhaps defined characters for simple line  
>> and table drawing that invoke automatic translation when an ASCII  
>> text version is generated.
>

Such alternative field could be an added element to the DTD or Schema  
defining the XML input document.  When the output is other than  
ASCII, the alternative field could be displayed.  To allow  
compatibility with existing tools, the ASCII version would not be  
affected.  Permitting access to _some_ extended characters could  
improve upon the quality of some line-drawing for non-ASCII outputs.

To avoid the "pain-in-the-ass" issue, improved drawings could be  
generated by a simple web based drawing application, where the  
translation back into ASCII artwork would be straight-forward, and  
yet provide comparable results.  Currently, improved graphics are  
limited to the generation of HTML tables.  The drawing application  
could even create the needed XML wrapper for an RFC.

>> Being able to review the ID as it would appear as an RFC would  
>> also seem to be a requirement.
>

As an ID becomes an RFC, it seems expecting last minute changes to  
the document would be even more daunting.

> assume that every person reading the document could display it.  
> They would put a legend or explanation near the example.

Assume such characters can not be displayed, at least not with the  
ASCII version that excludes the extended character-set allowed by  
unicode.  An escape mechanism would be needed to accommodate  
alternative text, where displaying '?' for the unicode characters  
that extends beyond ASCII would not be a very satisfactory solution,  
as this would make the ASCII version less authoritative, to say the  
least, and break the way many use the RFC text files.  I liked the  
idea that Frank suggested, use the HTML escape sequence to declare  
the unicode character.  This allows the ASCII version to remain  
authoritative.

-Doug

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 30 Nov 2005 18:29:46 -0800

At 5:59 PM -0800 11/30/05, Douglas Otis wrote:
>a current range of about 97,000 characters.  These characters 
>include punctuation marks, diacritics, mathematical and technical 
>symbols, arrows, dingbats, etc.  Displaying one of these characters 
>requires a character-set (synonymous with a display system's 
>font-set or character-repertoire), or using the unicode vernacular, 
>a script.  It is not just a matter of which character is displayed, 
>which character-repertoire is used, but there are also Middle 
>Eastern right-to-left issues as well.

It may be better to use a single vocabulary for discussing things 
such as internationalization and character sets. That's the purpose 
of RFC 3536.

>the document would be even more daunting.

Yep, that's the tradeoff. We already make some automatic changes 
after in Internet Draft is approved by the IESG, and we allow others 
without IESG oversight. This would be another class. That scares some 
people, and not others. Having Internet Drafts use Unicode in UTF-8 
instead of ASCII scares some people, and not others.

>least, and break the way many use the RFC text files.

No escape mechanism is needed. Non-displayable characters are still 
in the RFC, they simply can't be displayed by everyone (but they can 
be displayed by many). This is infinitely simpler, and a much better 
long-term solution, than "an escape mechanism". Further, there would 
be no more "ASCII version" to be authoritative. The Internet Draft 
clearly says that there is a single RFC, and it has a single encoding.

>   I liked the idea that Frank suggested, use the HTML escape 
>sequence to declare the unicode character.  This allows the ASCII 
>version to remain authoritative.

... as well as unreadable and unsearchable using normal search 
mechanisms. The purpose of the proposal is to allow RFCs to be 
readable and searchable using the encoding that is common on the 
Internet, without resorting to sorta-kinda-HTML or an "escape 
mechanism". Remaining with plain ASCII would be better than either of 
the latter.

--Paul Hoffman, Director
--VPN Consortium

---

From: Masataka Ohta <[email-masked]>
Date: Thu, 01 Dec 2005 13:15:19 +0900

Robert Sayre wrote:

> like correct attribution to me.

It is your stupidity that you can't recognize peoples' names correctly
represented in ASCII.

See how widely non-ASCII domain names and mail addresses are used.

However, if you insist, you may use ASCII art.

								Masataka Ohta

---

From: Robert Sayre <[email-masked]>
Date: Wed, 30 Nov 2005 23:52:06 -0500

> It is your stupidity that you can't recognize peoples' names correctly

Well, I'm not the sharpest knife in the drawer, but Google is even
duller than I am. Anyway, I'm wondering what all the command line
whinging is about, since I came home from work and tried some command
line tools on <http://www.cl.cam.ac.uk/~mgk25/ucs/examples/quickbrown.txt>.
I tried cat, vi/vim, more/less, and pico/nano on Ubuntu Linux 5.10.

--

Robert Sayre