Subject: Re: Internationalization and the IETF (Re: Will Language Wars Balkanize the Web?)

From: Vernon Schryver <[email-masked]>
Date: Thu, 7 Dec 2000 01:58:04 -0700 (MST)

> From: Harald Alvestrand <[email-masked]>

> >The same thinking that says that MIME Version headers make sense in
> >general IETF list mail also says that localized alphabets and glyphs must
> >be used in absolutely all contexts, including those that everyone must
> >use and so would expect to be limited to the lowest common denominator.
>
> it may have escaped the notice of some that a fair bit of the discussion on 
> diacritcs was carried out using live examples, and while I am sure there 
> were some who did not see the diacritics on screen, at least there was a 
> single definition of how to get from what was sent on the wire to what 
> might have been displayed on the screen, and MANY of the participants 
> actually saw them correctly displayed.

Diacritical marks are no different from Cyrillic, Arabic, Greek, Hebrew,
Sanskrit, and other non-Latin character sets in not being not part of
the international language.  The goal of communicating is to communicate,
not wave flags in support of national languages.  When you are trying
to talk to strangers and have no clue about their languages, you are a
fool to not use the common, international language, no matter how poor
and ugly it is.

> MIME character sets is an example of a battle fought and won.

When MIME is used to pass special forms among people whose common
understandings including more or other than ASCII, MIME is a battle
fought and won.

When MIME is used to send unintelligible garbage, it is a battle fought
and lost.  Whether the garbage is HTML, the latest word processing
format from Redmond or a good representation of the mother tongue of
1,000,000,000 people is irrelevant to whether the use of MIME is wise
or foolish.  If the encoding is not known before hand to be intelligible
to its recipients, then the use of MIME is foolish.

MIME is a good *localization* mechanism, either in geography or culture
or in computer applications (e.g. pictures or sound).

The continuing IETF efforts to extend MIME to include yet more extra or
special forms in the vague hope that the recipient will surely be able to
interpret at least one is probably the best of what we can expect from
"internationalized" domain names in 2 or 3 years.  Unless something like
Vint Cerf's principle of encoding *localized* domain names in ASCII is
followed, the IDN efforts will at best repeat the history of MIME email
exemplified by the many Microsoft MIME formats.

In MIME, except in special cases, the "universal" form of the body is
either sufficient and the fancy versions useless wastes of cycles, storage,
and bandwidth, or the "universal" form can only say "sorry, better upgrade
your system."  Just as in the vast majority of HTML+ASCII email where
there is can be no useful difference and there is rarely a visible
difference between the ASCII plaintext and the HTML encrypted version,
*localized* domain names will either be unusable outside their native
provinces or they will be usable with a 7-bit ASCII keyboard.

Vernon Schryver    [email-masked]

---

From: Dave Crocker <[email-masked]>
Date: Thu, 07 Dec 2000 07:23:11 -0500

At 01:58 AM 12/7/00 -0700, Vernon Schryver wrote:
> > diacritcs was carried out using live examples,
>
>not wave flags in support of national languages.

In a sense, Harald's observation points out a case in which all those other 
sets very much ARE part of the "international" language.

The live examples were a) intended, b) appropriate, and c) successful.

It does not matter whether readers understood the semantics of the strings; 
they needed to be able to see them.

         That is not national flag waving.

         That is global utility.

>and lost.

Technical standards work often gets distracted by trying to deal with 
issues that are outside the scope of reasonable technical standards 
work.  It should not be the task of such work to dictate or constrain users 
to only socially acceptable behavior.  That is a social task, not a 
technical one.

Choosing to send various types of data requires making decisions about the 
context.  No technical standard can be designed to "automatically" 
determine when it is, or is not, appropriate to send that data, whether it 
is diacritical marks, kanji, or an excel spread sheet.  Even when the 
sender has information about recipient capabilities, social factors affect 
the choices.

         So sending such data in MIME inappropriately
         is STILL an example of a battle fought and won.

At least the recipient has the unintelligible data well isolated and 
labeled.  MIME did its job.

At 08:19 AM 12/6/00 -0500, vint cerf wrote:
>Even if we introduce extended character sets, it seems vital
>that there be some form of domain name that can be rendered
>(and entered) as simple IA4 characters to assure continued
>interworking at the most basic levels. This suggests that
>there is need for some correspondence between an IA4 Domain
>Name and any extended characterset counterpart.

The same task is at issue for the DNS as it was for MIME.  We need a 
mechanism for labeling and encoding DNS strings and, I believe, we need it 
to be added to the existing DNS.

         Users of those strings will be all over the world,
         not just in a particular locale.

The need for this capability is massive and immediate.

There WILL be a solution deployed.  In fact there already is.

The question is whether a coherent extension to DNS will be done in a 
fashion which will keep the DNS integrated, or whether this requirement 
produces an independent DNS.

         That's not flag-waving.

         That's multiple DNS namespaces.

We need to be careful to distinguish two different requirements.  One is 
for a mechanism to encode domain names in non-ascii character sets.  The 
second is for an equivalence mapping from non-ascii domain names into ascii 
domain names.  The former is so that the technical and operational aspects 
of the DNS remain coherent.  The latter is so that everyone has a way to 
reach a particular domain, even if they cannot generate the non-ascii form 
of the name.

The extreme form of the latter task involves ascii encodings that are 
"comfortable" for human users; that requirement is not solved in human 
non-technical situations.  I believe that the example of alternate choices 
of "jin" and "gin" as representations for some chinese character(s) was 
used.  Hence this extreme form of the task is not going to be solved by 
lowly IETF protocol designers.

At best, use of ACE-like encodings permits an ascii representation, albeit 
one that is "uncomfortable".  It is as far as the IETF should go in trying 
to permit a "universally accessible" form for all domain names.

Interestingly, we do not need to have all domain names exchange and stored 
in an ACE form, forever.  Just as MIME is able to support pure binary 
encodings, so can the DNS.  The ACE form can be mapped to when needed.

d/

=-=-=-=-=
Dave Crocker  <[email-masked]>
Brandenburg Consulting  <www.brandenburg.com>
Tel: +1.408.246.8253,  Fax: +1.408.273.6464

---

From: Keith Moore <[email-masked]>
Date: Thu, 07 Dec 2000 09:05:20 -0500

The notion that use of languages other than English can or should be 
'localized' strikes me as both shockingly arrogant and hopelessly naive.  

People can and will use their own languages on the Internet - in email, 
on the web, and in domain names, and without regard to their location
in either the physical world, the currently topology of the network,
or the TLD of the host they are using at the moment.  Furthermore, a 
great many people use multiple languages (not necessarily including 
English) is, so that a given person, host, or subnetwork will often 
need to exist in multiple (potentially competing) locales at once.

And while a great many people - who speak only a single langauge, 
and whose travels are confined to a small geographic area where
others speak only that language - might indeed be happy with a 
localized solution, adoption of purely localized solutions would 
impair the vast number of people who do not fall into that category.

The question is not whether people will use non-ASCII characters in
domain names, but whether the various uses of non-ASCII characters
will coexist peacefully with each other and with existing applications,
and whether applications will continue to interoperate with one another.

So while it's quite important that IDNs be able to be represented
in ASCII for compatibility with existing applications and the large
number of protocols that use DNS names as protocol elements, and 
even though we all understand that pure-ASCII, Romanized versions
of non-English names will continue to enjoy wide use -- we still need
to produce an IDN standard as soon as possible.

Fortunately the IDN group is making very good progress, and I'm 
confident that consensus around a concrete proposal will soon emerge.

Keith

---

From: John Stracke <[email-masked]>
Date: Thu, 07 Dec 2000 10:22:33 -0500

Keith Moore wrote:

> Furthermore, a

Sometimes even in the same sentence.  My mother grew up partly in Quebec;
when she's talking to her siblings, they'll often use French words when the
English ones don't come to mind immediately.

--
/==============================================================\
|John Stracke    | http://www.ecal.com |My opinions are my own.|
|Chief Scientist |=============================================|
|eCal Corp.      |How many roads must a man walk down before he|
[email-masked]|admits he is LOST?                           |
\==============================================================/

---

From: "Theodore Y. Ts'o" <[email-masked]>
Date: Thu, 7 Dec 2000 11:04:27 -0500

Date: Thu, 07 Dec 2000 07:23:11 -0500
   From: Dave Crocker <[email-masked]>

Indeed.  If I get a mail message which is in HTML only, 99.97% of the
time it's SPAM-mail.  And I've lost count of how many time I've received
Chinese (or other Asian language) SPAM-mail.  In fact, I'm seriously
thinking about coding up a rule which automatically junks HTML mail
unread. 

I guess MIME is useful for something.  :-)

						- Ted

---

From: Masataka Ohta <[email-masked]>
Date: Fri, 8 Dec 2000 05:05:00 +0859 ()

Vernon;

> understandings including more or other than ASCII, MIME is a battle

FYI, we, Japanese, have, long before MIME, been and still are
exchanging local characters purely within the framework of RFCs
821 and 822. See RFC 1468.

> MIME is a good *localization* mechanism, either in geography or culture

No.

ISO 2022 gives the good localization mechanism.

Unlike MIME, you can use and we are using it in UNIX files without
mail headers, file types, charset tagging nor POSIX locales.

ISO 2022 gives proper localization information. It can be used in
internationalized computer files to store international
characters and on internationalized computer terminals to
display international characters.

However, even with ISO 2022, it is meaningless to "internationalize"
domain names, of course, because ISO 2022 do not "internationalize"
people using domain names.

The only problem of ISO 2022 is that it is too complex having too
much optional features beyond the localization. So, proper profiling,
such as that specified in RFC 1468, is essential.

Then, ISO 10646 *simplified* ISO 2022 by removing the essential
feature of localization keeping all the other complexities,
many of which are now, though ignored, mandated.

MIME charset may be useful for ISO 10646.

MIME charset can supply the localization information to ISO 10646
as I demonstrated, as a silly joke, in RFC 1815.

							Masataka Ohta

PS

Note that MIME charsets of "ISO-8859-*" also removes the essential
but optional feature of ISO 2022 to give localization information
inline, which makes MIME useful for "ISO-8859-*".