Subject: Re: Layer 2 and "idn identities" (was: Re: [idn] what are the IDN identifiers?)

From: "Bruce Thomson" <[email-masked]>
Date: Tue, 4 Dec 2001 11:43:22 +0900

Written by liana Ye

> I sincerely feel, you don't know what you are talking about.
> And you don't understand what is going on with CJK 
> users.  I don't even know how to start to discuss with 
> you about your above comments.  May be after I have a 
> good night of sleep, I can come up with new ideas.

Sounds like you could use some sleep all right. Try not to
immediately question the credentials of those who don't
rush to accept your ideas.

I am belatedly trying to come up to speed on SC/TC, and
I see that it does have a lot of merit. But I think that the
notion of "equivalent characters" is really trying to take
this game too far. There are two words ways to write "egg"
in Japanese, pronounced the same, but one is written
with two characters, and one with only one character. Are
going to call these words "equivalent"? Sounds like you
would like to do that, if I understand your concept.

Bringing the English/French stuff in is a useless analogy too,
which just slows the conversation down. Case folding
is just not the same as SC/TC, and certainly not the
same as "equivalent characters". The more thoughtful
among us would like to be convinced of your arguments
on their merits, and not on the basis of analogies.

It will be a big step for me to agree with SC/TC, but I
may well do it. You may remember that originally I
was totally opposed to nameprep altogether.

If you are concerned about my credentials vis-a-vis
the CJK issues, I admit that I know very little about
China, but I am probably the wg's most vocal (if
admittedly not the most knowledgeable) person on the
subject of Japanese. I have lived in Japan 23 years, and all
my everyday communications with the exception of
this list are in Japanese. I personally wrote the first
Japanese word processor software for a laptop computer
back in 1984, and run an ICANN registrar in Japan.
Perhaps I can give you SOME insights into the
culture of language here, more than you might get
from elsewhere.

Bruce

> 
> On Mon, 3 Dec 2001 12:49:23 +0900 "Bruce Thomson" <[email-masked]>
> writes:
> > Written by liana Ye:
> > 
> > > 
> > > And Chinese TC/SC example:
> > > <wind> has four code points in Chinese:
> > > TC, SC, TC radical, SC radical.
> > > 
> > > I believe there are other code points of <wind>  have 
> > > been allowed in UCS.  Isn't  it a time to use the 
> > > "equivalent character set" term? Or may we use 
> > > "equivalent codepoints" to stay away from the lanugage
> > > connotation?
> > > 
> > 
> > So you would like these four code points to be normalized and
> > indistinguishable? There must be a huge cultural difference
> > between Japan and China in their feelings about the language.
> 
> Bruce:
> 
> Using your arguement above:
> 
> 0048; 0068; Case map
> 
> 210B; 0068; Additional folding
> 210C; 0068; Additional folding
> 210D; 0068; Additional folding
> 
> 1D407; 0068; Additional folding
> 1D43B; 0068; Additional folding
> 1D46F; 0068; Additional folding
> 1D4D7; 0068; Additional folding
> 1D573; 0068; Additional folding
> 
> So you would like these nine code points to be normalized and
> indistinguishable? There must be a huge cultural difference
> between English and French in their feelings about the language.
> 
> > 
> > In Japan, there are two characters for "river" (sorry, I am
> > not going to look up the code points.) But nobody here
> > in Japan would see the logic of treating them as identical.
> > They are different, and these differences between characters
> > are part of the rich culture of the nation.
> > 
> > To be sure, there are some characters that are so old that
> > while they are still used in names (such as in my own home address),
> > they could be normalized into their modern equivalents without
> > complaint. But who is going to sort all this out in a reasonable
> > timeframe?
> > 
> > The CJK unification makes it even worse, of course, as decisions
> > made for China affect other countries.
> > 
> > What is the down side of treating these characters as different,
> > anyway? Worst case, a company with "wind" in its name might
> > have to register two or even four variations of the name.
> > Works for me.
> > 
> > Bruce
> > 
> 
> I sincerely feel, you don't know what you are talking about.
> And you don't understand what is going on with CJK 
> users.  I don't even know how to start to discuss with 
> you about your above comments.  May be after I have a 
> good night of sleep, I can come up with new ideas.
> 
> Liana
> 
> 
>

---

From: "Maynard Kang" <[email-masked]>
Date: Tue, 4 Dec 2001 16:56:14 +0800

> would like to do that, if I understand your concept.

I find the notion of comparing TC/SC equivalence to Japanese
Kanji/Hiragana/Katakana equivalence rather compelling and far-stretched to an
extent.

Having gone through an institutionalized form of Chinese-language education
exclusively in SC with hardly any opportunity (or neccessity) to learn TC, I
must say that if not for the availability of TC/SC conversion programs (e.g.
NJStar), it would be a mammoth task for me to communicate with my Taiwanese
friends over e-mail, since my knowledge of TC characters is almost as sparse
as my knowledge of Arabic (i.e. practically non-existent).

Compare this with Japanese; I do not think you can find two Japanese-speaking
individuals, one having knowledge of "egg" in Kanji ONLY and one having
knowledge of "egg" in Hiragana ONLY.

Chances are most Japanese-speakers know both equivalent forms.
Chinese-speakers may not.

regards

---

From: John C Klensin <[email-masked]>
Date: Tue, 04 Dec 2001 09:36:28 -0500

--On Tuesday, 04 December, 2001 16:56 +0800 Maynard Kang
<[email-masked]> wrote:

> Kanji/Hiragana/Katakana equivalence rather compelling and
> far-stretched to an extent.
> 
> Having gone through an institutionalized form of
> Chinese-language education exclusively in SC with hardly any
> opportunity (or neccessity) to learn TC, I must say that if not
> for the availability of TC/SC conversion programs (e.g.
> NJStar), it would be a mammoth task for me to communicate with
> my Taiwanese friends over e-mail, since my knowledge of TC
> characters is almost as sparse as my knowledge of Arabic (i.e.
> practically non-existent).

Maynard,

There are two separate issues here, and I think part of the
problem is that they keep getting confused.

One is whether you should be able to communicate with your
Taiwanese friends and whether automatic translation
(character-mapping, if you prefer) techniques should be available
to help you.  The answer to that question is certainly "yes", and
I don't think anyone on this list would disagree with that.  If
you had Arabic-using friends, and you spoke Arabic, but didn't
read and write it, the answer would be the same: it would be
wonderful to have automatic translation/ mapping/ writing
systems, they should work well, and you should be able to use
them.

But what we are told in the west is that the original beauty of
Chinese is that a single writing system unifies several different
languages and spoken forms.  If the Chinese you speak is
"Mandarin", then you wouldn't necessarily expect to communicate
in spoken form with a Cantonese-only speaker, either: only the
writing system unifies the two language forms.

For better or worse, the decisions to "simplify" the language
sacrifices some of the universality of the writing system in the
interest of making it easier to teach and write.  Worthwhile, I
presume, but a tradeoff.  In the general case, hard to have it
both ways but, again, I don't think you would find anyone who
would argue that, if computer means can be made to help with the
mappings, people should be prevented from doing so.

The second question is whether that set of mappings/ conversions/
translations ought to be incorporated into IDN or the DNS.   And
it is _there_ that we differ.  Once again, the DNS incorporates
strings of individual characters, not names.  The further we get
away from doing bit-string-level matching, the more trouble we
get ourselves into, and TC-SC mappings are pretty far from
bit-string matching.

The logic "I should be able to communicate with my Taiwanese
friends, even though we don't use the same characters to write
the same words" doesn't work, at least in this DNS context, any
more than "I should be able to communicate with my
Arabic-speaking friends, even though I can't read their language"
does.   If I said that about Arabic, I would be justly criticized
for expecting the DNS to compensate for my ignorance.

> Compare this with Japanese; I do not think you can find two
> Japanese-speaking individuals, one having knowledge of "egg" in
> Kanji ONLY and one having knowledge of "egg" in Hiragana ONLY.
> 

At least in some styles of teaching Japanese outside of Japan,
Hiragana is taught first. So finding someone who cannot recognize
a given Kanji character, even when the Hirigana for the work is
known or can be guessed merely requires finding someone young
enough or new enough to the language.
 
regards,
    john

---

From: "Maynard Kang" <[email-masked]>
Date: Wed, 5 Dec 2001 04:45:22 +0800

Hi John,

(For the record, I think you have got me wrong. The motivation behind my
previous post was to merely point out the inappropriate comparison of TC-SC
equivalence to Japanese character equivalence, and was not intended to justify
(or otherwise) the inclusion of language-dependent canonicalization processes
such as TC-SC into the DNS layer)

> get ourselves into, and TC-SC mappings are pretty far from

No, I don't think we differ much on this (--> my first paragraph in this
mail).

I agree that language-dependent canonicalization may be too incredibly complex
to be included in the DNS layer. Doing it at Layers 2 and above seems like a
more plausible solution since more locale-specific parameters (or facets, as
you call it) can be gathered at that layer, minimizing the risk of incorrectly
interpreting equivalences if only bit-string matching is used.

But the million-dollar question is this:

Given that:

Average Internet users probably do not care where the i18n is done anyway, as
long as they can access resources with names in their own native language,

and,

If i18n of names can be achieved at layers other than the DNS, then:

Why internationalize the DNS in the first place, and why continue with the
work on the IDN WG, if the DNS only serves to provide an identifier-based (and
not names-based) lookup service for Internet hosts and services?

IMHO, if we have decided to go ahead with IDN and we have come this far,
perhaps we should aim to provide a solution as comprehensive as possible, lest
users be bewildered when they cannot resolve their hostnames due to
non-equivalence of characters.

But if the comprehensive solution cannot be done at the DNS layer, then we
should not have to live with a half-baked solution, and resources spent on
this WG should be diverted to IRNSS instead.

> for expecting the DNS to compensate for my ignorance.

Now, I might be justifiably considered ignorant if I were to expect a computer
to be able to help me understand written Arabic when I only understand the
spoken form.

However, the inability to perform mental TC-SC equivalence matching is common
amongst many Chinese speakers/writers, largely due to educational policy
differences. Many Taiwanese do not read Simplified characters. Most mainland
Chinese do not use Traditional characters anymore (although they can read
them). Construing this inability as ignorance is thus almost a form of
culturo-linguistic arrogance (albeit with a geo-political dimension to it).

Perhaps I might not expect the DNS to do it, but that is because I know what
the DNS is and what it was originally constructed to do, having a better
appreciation of its (limited) responsibilities after reading your dns-role
draft.

The average Internet user probably does not, though. And these users,
especially those with a monolithic view of how computer systems work, expect a
certain level of "assistance" (for lack of a better term) to be rendered to
them by the system (which encompasses all layers, not just the DNS layer).

I think there is an explicit requirement from the users for this "assistance"
to be rendered, regardless of what layer it is done at.

> enough or new enough to the language.

Yes, I agree that in this case there might be a possibility that a Japanese
speaker may not be able to recognize a Kanji character but can do so for its
Hiragana equivalent.

(After all it is not uncommon to find Furigana hints alongside Kanji
characters in Japanese textbooks to aid in pronunciation for not-so-adept
Japanese speakers)

However, you are making a comparison here between Japanese speakers based on
their level of language mastery (or fluency); a comparison between young
Japanese speakers and mature ones is akin to comparing unripened apples to
fully-ripe apples.

The difference between TC and SC would be better appreciated as one between
apples and pears though. i.e. The respective trees probably belong to the same
genus/species, but the fruits are distinct enough for us to name them
differently (I would surreptitiously doubt the accuracy of this statement for
I am no botanist, but assume true for the purposes of this analogy).

Let me rephrase my earlier statement for greater clarity: A _fluent_ Japanese
speaker/writer would probably know both forms of "egg" in Japanese, but a
_fluent_ Chinese speaker/writer may not know both TC and SC forms of "egg" in
Chinese.

Thus, it is probably a less compelling argument to include Japanese character
equivalences into naming/identification engines than it is for Chinese.

But one should not construe all forms of CJK equivalences as having the same
level of necessity of being included in a canonicalization scheme, because
they do not.

regards,
maynard

---

From: "Bruce Thomson" <[email-masked]>
Date: Wed, 5 Dec 2001 11:10:46 +0900

> Kanji/Hiragana/Katakana equivalence rather compelling and far-stretched to an

Sorry, I confused you by carrying my conversation with Liana over 2
posts. I was not indending to say that the two forms of egg in Japanese
corresponded to an SC/TC equivalence. I was responding to
Liana's proposal(?) that the 4 forms of "wind" in Chinese (2 SC,
2 TC) be treated as equivalent. If you are going to do that,
logically the same thing should be done for Japanese. There are
some cases with semi-obsolete characters where such equivalences might
be useful, but the whole issue is quite complex, poorly defined, and
might actually be objectionable to many users in my opinion.

SC/TC equivalence itself is far simpler than the "four winds, two eggs"
equivalences, and has quite a bit of merit. I won't express any
real opinion on it until I study it further.

By the way, both forms of egg I was referring to are Kanji. There
are of course additional kana forms. I doubt any Japanese user
would want to treat Kanji and kana as equivalent at the domain
name level. As usual, I am counting on Yoneya-san to keep me
honest when I make statements like this.

Bruce

---

From: "Maynard Kang" <[email-masked]>
Date: Wed, 5 Dec 2001 11:49:52 +0800

> might actually be objectionable to many users in my opinion.

I guess that is the main difference - SC/TC equivalence is not at all
objectionable to Chinese users in my opinion. In fact, it is a feature that
some Chinese expect to be naturally present in a naming system.

If I read you correctly, the same cannot be said for Japanese.

> real opinion on it until I study it further.

It is not so simple as to be able to be done _accurately_ by an code-based1-1
bit-string matching process. There are semantic, syntactic and contextual
considerations that require at the very least a morphological analysis process
in order for TC/SC to be done with a reasonable amount of accuracy (i.e.
orthographically).

A good starting point for further understanding TC/SC is
http://www.ietf.org/internet-drafts/draft-ietf-idn-cjk-01.txt and Jack
Halpern's C2C article (http://www.basistech.com/articles/C2C.html).

> honest when I make statements like this.

I seem to recall Yoneya-san saying something to the effect of "no
canonicalization should be done for Japanese" back at the Adelaide meeting. I
might have remembered wrongly though.

regards,
maynard