Subject: [idn] Re: Chinese Domain Name Consortium (CDNC) Declaration

From: Scott Bradner <[email-masked]>
Date: Sun, 03 Feb 2002 08:08:42 -0800

> Assume Chinese name of 10 glyphs. Each one may take 2 versions,
> TC or SC.

how often are Chinese names written using a mixture of TC & SC?

Scott

---

From: Erin Chen <[email-masked]>
Date: Mon, 04 Feb 2002 18:41:13 +0800

Dear Scott,

Before I describe how often are Chinese Names written using a mixture of 
TC & SC, I explainhow theuser behavior and the current Input Method
Environment ofapplicaiton which user would face to.

1. In the user behavior of writtenChinese in ones daily life,it is 
   VERY often of using mixture of TC & SC. I say VERY often here
   because it is the userwritten custom. It is difficult to have a 
   statistics to describe how often.
2.In the current Input Method Environment ofapplicaiton, user would
  face to a IME of could type mixture TC & SC very easily. And because
   of the custom of using mixture TC & SC, the boundary of TC & SC is 
   becoming indistinct. It is also difficult to distinguish them by user.

Although it is difficult to give a general statics. But I can offer some
staticsof TWNIC idn.tw testbed. In our testbed there are registerd 
27,665 idn.tw Chinese domain namenow. 

In TWNIC idn.tw testbed, we have make a experiment on a TC & SC set
(U+81FA, U+53F0). May be our experimental statics can used as explain
how often. In the experiment we allow registrants to choose one of the
(U+81FA, U+53F0) if they need in their Chinese domain name. In our 
statics, there are2,311 Chinese domain namechoose TC U+81FA and 
3,938 choose SC U+53F0

In A Complete Set of Simplified Chinese Characters, there are well 
defined more then 2,000 set of TC & SC which are frequently used in 
Chinese daily live. 

Erin Chen

---

From: "James Seng/Personal" <[email-masked]>
Date: Mon, 4 Feb 2002 20:49:11 +0800

I have tried to be silent all these while but I feel it is time for me
to speak up. Note that what I said is in my own capacity, and does not
represent the IDN WG, Marc (my co-chair), my company or whatever. It is
solely, and primarly my own opinion.

The problem of Traditional Chinese & Simplified Chinese cannot be
expressed as a bicameral (upper/lower case) problem. To say it is
similar to "A" and "a" is at best misleading. TC/SC is not a simple 1 to
1 mapping.

Neither can it be compared to other problems in some other languages.
Every language is unique, each of them face with problems that its
unique only to itself, as demostrated in some of the mails in this
thread. So lets please do away with any anology.

The complexity of TC/SC is not something can be explained in a single
word, but it is described quite well in the following articles:
    http://playground.i-dns.net/one/onec_sum.htm
    http://www.cjk.org/cjk/c2c/c2cbasis.htm.

In particular, pay attention to the second article and on the section of
"code-point substitution". Quote: "The easiest, but most unreliable, way
to convert SC to TC, or vice versa, is to do so on a
codepoint-to-codepoint basis".

Some subset of TC/SC problem can be solved using using this method,
however unreliable it is. But such solution have to take into
consideration of conflicting languages (such as Japanese & Korean) and
thus striping it down to a even smaller subset. In a JET meeting whereby
we do this investigation, this would bring the simple 2400+ TC/SC pairs,
down to around 300+ TC/SC pairs. (And note, (U+81FA, U+53F0) pairs, an
commonly used characters since it represent "tai" in "taiwan" is not
part of the sub-subset). One would therefore question the usefulness of
such partial solution.

There have been many attempts to solve the "TC/SC problems", and in
fact, some draft have been admitted to the group. Unfortunately, none of
them have managed to stand up to the scuruity of the working group.

In addition, the Unicode Technical Consortium have send the following
recommendation to the working group on TC/SC
    http://www.imc.org/idn/mail-archive/msg04005.html

Nevertheless, the "Chinese language" problem is one which we cannot
deny, altho it may not be as serious as others is painting it. But all
problems are serious, the degree of seriousness depends on the
individual perspective.

But the fact we talk about "language" is a wrong start. Human language
is complex. There are very few language rules that can be encapsulated
within computer software accurately. And we have to ask ourselves if the
expection that DNS is able to do so is fair.

This is why the group have steer away from "How do I put a Chinese words
into the domain name" problem and work on "How do I put a Han Ideograph
into the domain name". In short, we deal with scripts, and not
languages.

This is not to say we close our eyes and ears away from the "language"
problem. Once again, lets me remind everyone that every language have
its own problems so it is not just Chinese. Instead, we need a more
complex naming system, one that John have already explained in his mail,
for *all* language problems:
    http://www.imc.org/idn/mail-archive/msg05615.html

It is always easier to point out problems then to provide solution. This
is why the world have more movies critics then movies directors :-)
However, the world value solutions more than we value problems.

And when there is only one obvious solution, it does not mean the
obvious solution is the right one. As Dave points out, "When all you
have is a hammer, everything look like a nail".

The IDN WG is not absolute. It is a beginning of a journey to do I18N in
IETF and the Internet, one that can bring the Internet closer to the
world; OR it can be a end to any further attempts to do I18N in IETF.
Neither is it the only forum whereby all solutions to I18N is to be
conducted. Not all solutions are technical in nature, and even if it is,
not all problems can be solve within IETF.

ps: It is not my intention to be argumentive so this will be my first
and only mail on this thread.

-James Seng

----- Original Message -----
From: Erin Chen
To: Scott Bradner
Cc: [email-masked] ; [email-masked] ;
[email-masked] ; [email-masked] ; [email-masked] ;
[email-masked] ; [email-masked] ; [email-masked] ;
[email-masked] ; [email-masked] ; [email-masked] ;
[email-masked] ; [email-masked] ; [email-masked] ; [email-masked]
; [email-masked] ; [email-masked] ; [email-masked] ;
[email-masked] ; [email-masked] ; [email-masked] ;
[email-masked] ; [email-masked] ; [email-masked] ;
[email-masked] ; [email-masked] ; [email-masked] ;
[email-masked] ; [email-masked] ; [email-masked] ;
[email-masked] ; [email-masked] ; [email-masked]
Sent: Monday, February 04, 2002 6:41 PM
Subject: Re: [idn] Re: Chinese Domain Name Consortium (CDNC) Declaration

Dear Scott,
Before I describe how often are Chinese Names written using a mixture of
TC & SC, I explain how the user behavior and the current Input
MethodEnvironment of applicaiton which user would face to.1. In the user
behavior of written Chinese in ones daily life, it is    VERY often of
using mixture of TC & SC. I say VERY often here   because it is the user
written custom. It is difficult to have a    statistics to describe how
often.2. In the current Input Method Environment of applicaiton, user
would   face to a IME of could type mixture TC & SC very easily. And
because   of the custom of using mixture TC & SC, the boundary of TC &
SC is    becoming indistinct. It is also difficult to distinguish them
by user.Although it is difficult to give a general statics. But I can
offer somestatics of TWNIC idn.tw testbed. In our testbed there are
regist
erd 27,665 idn.tw Chinese domain name now. In TWNIC idn.tw testbed, we
have make a experiment on a TC & SC set(U+81FA, U+53F0). May be our
experimental statics can used as explainhow often. In the experiment we
allow registrants to choose one of the(U+81FA, U+53F0) if they need in
their Chinese domain name. In our statics, there are 2,311 Chinese
domain name choose TC U+81FA and 3,938 choose SC U+53F0In A Complete Set
of Simplified Chinese Characters, there are well defined more then 2,000
set of TC & SC which are frequently used in Chinese daily live. Erin
Chen
Scott Bradner wrote:

Assume Chinese name of 10 glyphs. Each one may take 2 versions,TC or SC.
how often are Chinese names written using a mixture of TC & SC?Scott

---

From: John C Klensin <[email-masked]>
Date: Mon, 04 Feb 2002 08:35:28 -0500

Scott (and others),

Let me see if I can make an analogy that explains the way I
understand this problem.

Erin (and others), 

If I have this wrong, please correct me --I still have a great
deal to learn.  If I do not, please think about the examples and
cases given, as they may help to explain why many of the users
of alphabetic languages have been reluctant to put TC<->SC
mapping into nameprep or some other IDNA step.

---------

As you know, there have been several attempts to "reform"
English spelling in the last couple of centuries.  The intent
has been to remove the silent characters, reduce idiosyncracies
due to the different language of origin of different words, to
get the spellings into a purely phonetic form or some
combination of them.   Few of these have gone anywhere, but
suppose that a system that rationalized only the thousand most
frequently used words had been adopted in the US and was in
moderately wide use, and that the change had been made within
the lifetime of many, or most, of English-users in the US.

Now, what we would end up with is an odd set of combinations.
Many words within that thousand would not be changed (how could
you simplify "a" or "is" ?).  Others would exist in post-change
form in ordinary writing, but in pre-change form in, e.g., names
and trademarks of companies that had existed and been in wide
use pre-change (e.g., I can't imagine a well-know breakfast food
firm being happy trying to change all of its corporate and
advertising materials to "Kelogz") and in older texts.   The
change would be slower, or would not be adopted at all, in other
English-speaking countries, with enclaves of English speakers in
non-English-speaking countries making the change most slowly of
all.  But all users of English would need to get used to seeing,
or puzzling out, the variants or they would discover that their
ability to read the language --as written-- would deteriorate
over time.  Would these things get mixed?  Of course they would:
To use an example I have used before in different form, consider
the well-known firm
   toz-(Cyrillic Ya)-us
and its likely insistence that, somehow, 
   toys-r-us
   toys-(Cyrillic Ya)-us
   toz-r-us
and maybe even
   toysrus
and that set of four additional variations.

Now, that is a case in which there are _only_ two words that
have variations and only two cases for each word.   But suppose,
as would be reasonable once one started down this slippery
slope, one also expected British and American spellings to
match: after all, the American spellings are just
simplifications from an earlier time.  And assume that many
names were more complex compounds than just having three
components.  So now there are three variations for many words
(raising the multiplier for multiple-word phrases) and phrases
with more words (creating a larger multiplier).   

And all of that assumes that the company doesn't get it into its
head to want toys.r.us and its variants, and that a "toz" TLD
isn't created in which they would, presumably, want to be
"us.r.toz".  This is a different case, in that no one would
expect (I hope) "us.r.toz" to match "toys-(Cyrillic Ya)-us", but
consider the implications to the administrative hierarchy.

And, of course, the new spelling system, unless it is much more
radical than the examples I have given above, introduces some
new homographs and other ambiguities.  E.g., does the simplified
"toz" match the traditional "toys", or "toes", or maybe even
"togs", or the made-up traditional word "toz"?  Or perhaps
several of them?  Does the present tense of "to read" get mapped
into "reed" while the past tense gets mapped into "red"?  And,
if so, what happens to the plant and the color (colour? colur?
:-( ).

Now, if you treat each "word" or "spelling variation" as if it
were a single character, you probably have a first-order
approximation to the Chinese situation.  And, to a greater or
lesser extent, the following statements are all true:

	(i) Modulo that ASCII case-matching rule, the DNS
	matches on code points, not on some abstraction of what
	a "character" is about, so this discussion is irrelevant.
	
	(ii) Things that people (end-users, not just computer
	geeks) see as the same ought to be treated as the same,
	or we will have vast confusion.
	
	(iii) Since, in Chinese, the things we see as "words" in
	English are really single characters, one ought to be
	able to do the extended mapping and matching things we
	now do for "case" in ASCII, and for case and other
	simplifications that are proposed for Nameprep, for
	these simplifications/variations also... even if it
	clearly is not compatible with the DNS to do them for
	English word-spelling variations.

	(iv) If we need to do this for Chinese, then
	"simplifications" that have been introduced through the
	years in "Latin" alphabets ought to match as well.  E.g.,
	    "IVLIVS" ought to match "julius"
	right?

Obviously (I hope) these principles are inconsistent.  And many
people are going to be upset with any decision that is made.  In
particular, if simplification-matching is not going to be
permitted for Chinese, then it is possible that some of the
mappings that are now made in Nameprep should be eliminated
because they are a bit too close to that (or to (iv) above) to
be comfortable.

    john

--On Monday, 04 February, 2002 18:41 +0800 Erin Chen
<[email-masked]> wrote:

> Dear Scott,
> 
> Before I describe how often are Chinese Names written using a
> mixture of  TC & SC, I explainhow theuser behavior and the
> current Input Method Environment ofapplicaiton which user
> would face to.
> 
> 1. In the user behavior of writtenChinese in ones daily
> life,it is     VERY often of using mixture of TC & SC. I say
>...

---

From: "Mark Davis" <[email-masked]>
Date: Mon, 4 Feb 2002 07:49:53 -0800

The test data, as you describe it, is completely insufficient to draw any conclusions. The purpose
of a test would be to determine whether the combinatorics of mixing TC and SC characters were an
real issue, not simply whether people choose a particular SC vs. TC character.

To do an accurate survey, you would have to be much more rigorous, doing at least the following:

- Divide all CJK characters into three classes: TC-only, SC-only, TC-or-SC.
Post this list so that it can be scrutinized by others.

- Test all names to see which contain at least one character from TC-only and one from SC-only.
Post that list so that it can also be scrutinized by others.

- Of those, see how many would match under a TC-SC mapping.
Publicize both the mapping and the list so that they can also be scrutinized by others.

- See what the percentages are: mixed / total and matching / mixed.

Note that the accuracy of the test also depends heavily upon the accuracy of the division of
characters in the original three classes, and the accuracy of the TC-SC mapping.

Mark
—————

Πόλλ’ ἠπίστατο ἔργα, κακῶς δ’ ἠπίστατο πάντα — Ὁμήρου Μαργίτῃ
[For transliteration, see http://oss.software.ibm.com/cgi-bin/icu/tr]

http://www.macchiato.com

  ----- Original Message -----
  From: Erin Chen
  To: Scott Bradner
  Cc: [email-masked] ; [email-masked] ; [email-masked] ; [email-masked] ;
[email-masked] ; [email-masked] ; [email-masked] ; [email-masked] ;
[email-masked] ; [email-masked] ; [email-masked] ; [email-masked] ;
[email-masked] ; [email-masked] ; [email-masked] ; [email-masked] ; [email-masked] ;
[email-masked] ; [email-masked] ; [email-masked] ; [email-masked] ; [email-masked] ;
[email-masked] ; [email-masked] ; [email-masked] ; [email-masked] ;
[email-masked] ; [email-masked] ; [email-masked] ; [email-masked] ; [email-masked]
; [email-masked] ; [email-masked]
  Sent: Monday, February 04, 2002 02:41
  Subject: Re: [idn] Re: Chinese Domain Name Consortium (CDNC) Declaration

  Dear Scott,
Before I describe how often are Chinese Names written using a mixture of TC & SC, I explain how the
user behavior and the current Input MethodEnvironment of applicaiton which user would face to.1. In
the user behavior of written Chinese in ones daily life, it is    VERY often of using mixture of TC
& SC. I say VERY often here   because it is the user written custom. It is difficult to have a
statistics to describe how often.2. In the current Input Method Environment of applicaiton, user
would   face to a IME of could type mixture TC & SC very easily. And because   of the custom of
using mixture TC & SC, the boundary of TC & SC is    becoming indistinct. It is also difficult to
distinguish them by user.Although it is difficult to give a general statics. But I can offer
somestatics of TWNIC idn.tw testbed. In our testbed there are regist
erd 27,665 idn.tw Chinese domain name now. In TWNIC idn.tw testbed, we have make a experiment on a
TC & SC set(U+81FA, U+53F0). May be our experimental statics can used as explainhow often. In the
experiment we allow registrants to choose one of the(U+81FA, U+53F0) if they need in their Chinese
domain name. In our statics, there are 2,311 Chinese domain name choose TC U+81FA and 3,938 choose
SC U+53F0In A Complete Set of Simplified Chinese Characters, there are well defined more then 2,000
set of TC & SC which are frequently used in Chinese daily live. Erin Chen
  Scott Bradner wrote:

---

From: Dave Crocker <[email-masked]>
Date: Mon, 04 Feb 2002 08:01:41 -0800

>them have managed to stand up to the scuruity of the working group.

Folks,

For the moment, let us ignore the fact that this issue is outside the scope 
of this working group and let us ignore the scrutiny of the IDN working 
group.

Let us consider the scrutiny of end-users.

Which solutions to TC/SC have gained wide acceptance in the rest of the 
world?

That is, please point us to a specification of a solution that has already 
gained wide acceptance in the global community using TC and SC.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.273.6464

---

From: "xiang deng" <[email-masked]>
Date: Tue, 5 Feb 2002 08:46:46 +0800

Dear John:

Let me do comparision and make this issue more clearly. 
There are a thousand reasons to agree/object a argument, but fact is fact.

--------------examples ---------------|string/character|sound|glyph-|meaning|usage|cannotation|extention
toz-------------togs-toys-toes---------|string(word)----|same-|------|same---|same-|same-------|same    
color-----------colour colur-----------|string(word)----|same-|------|same---|same-|same-------|same         
<u+4e2d u+534e>-<u+4e2d u+83ef>-|string(word)----|same-|------|same---|same-|same-------|same   

<u+534e>--------<u+83ef>-----------|character-------|same-|unlike|same---|same-|same-------|same   
A---------------a----------------------|character-------|same-|unlike|same---|same-|same-------|same  

Best Regards
Deng xiang

----- Original Message ----- 
From: "John C Klensin" <[email-masked]>

---

From: "xiang deng" <[email-masked]>
Date: Tue, 5 Feb 2002 11:27:27 +0800

----- Original Message ----- 
From: "James Seng/Personal" <[email-masked]>

>     http://www.cjk.org/cjk/c2c/c2cbasis.htm.

The whole TC/SC issue include 1:1  1:n 1:(n+1)  n:m and so on.
I had explained it times and times:
1:1   is accurate mapping from 1 to 1, it is defined in "A Complete set of 
        Simplified and Traditional Chinese character". (2237 pairs) They cover
        83.6% of current CDN registered data in our database (500,000 samples)

1:n 1:(n+1) n:m is depended on context, it need compute artificial 
                         intelligent technology to solved.(about 230 pairs)

we are talking about 1:1. 

>     http://www.imc.org/idn/mail-archive/msg04005.html

In my memory, there was a mail in IDN WG maillist, it also said
that xxx from Chinese disagree TC/SC in UTC.
How does it change so quickly?

I'm not challenge Mr.Mark Davis. And I know there is a CJK WG in UTC.

UTC is a organization to collect/clean up different glyphs. They are
experts of language/script/character.

But we are talking about the possibility to solve 1:1 TC/SC issue in CDN.

>     http://www.imc.org/idn/mail-archive/msg05615.html

Yes. We should discuss "language" problem in Lay2/Lay3 designed by Mr.John.
But how to deal with the problem lied in CDN? It's in domain name system.
Do you mean: Because Lay2/Lay3 can solve this problem,then we needn't solve
it in CDN?
But the problem is still here.

Regards.
Deng xiang

---

From: "Kenny Huang" <[email-masked]>
Date: Tue, 5 Feb 2002 13:09:44 +0800

> similar to "A" and "a" is at best misleading. TC/SC is not a simple 1 to

1-1 TC/SC mapping is exact the same as "A" to "a" mapping from 
computing algorithm point of view.  Your question should point to the 
source of table, not to the computational algorithm.

CDN requirement has been proposed. Read the CDN requirement, 
CDN is not Chinese literature. It is an identifier.  Sophisticated language 
issue should be discussed in the upper layers.
 
> part of the sub-subset). One would therefore question the usefulness of

I was chairing the JET meeting. The investigation is about 
1986 pairs (2237 pairs - 251 pairs which overlapping with Japanese
character). 

> not all problems can be solve within IETF.

Unfortunately, IDN deployment is not a reversible process if things 
went wrong. If you do think the obvious solution is not the right one,
then we'd better consider the consequences from adopting the solution.
Trying to do it halfway is worse than not doing it at all.  

Kenny Huang
Secretariat, JET

---

From: hoho <[email-masked]>
Date: Thu, 07 Feb 2002 11:28:45 +0800

Dear Scott,

It's getting more often after  Microsoft launched Windows 2000. Now,
people could use it to input Unicode CJK Han characters including 
TC, SC, and even Japanese Han characters. Please refer to the 
following article.

http://support.microsoft.com/default.aspx?scid=kb;EN-US;q227483

-- Janming