Subject: Re: [idn] opting out of SC/TC equivalence

From: "tsenglm@????.??.tw" <[email-masked]>
Date: Sat, 1 Sep 2001 13:26:27 +0800

----- Original Message -----
From: "Patrik Faltstrom" <[email-masked]>
To: "tsenglm@pºô€€€ß.€€€j.tw" <[email-masked]>; "Harald Tveit
Alvestrand" <[email-masked]>; <[email-masked]>; "hlqian"
<[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Saturday, September 01, 2001 4:40 AM
Subject: Re: [idn] opting out of SC/TC equivalence

> --On 2001-09-01 00.44 +0800
> "=?utf-8?B?dHNlbmdsbUDoqIjntrLkuK3lv4Mu5Lit5aSnLnR3?="
> <[email-masked]> wrote:
>
> >            As Patrik F?Â€ltstr?Â¶m  described , if it is a mechanism in
> > UNICODE consosium , then they have the version update procedures and
IDNA
> > also treat  the version update in UNICODE , why it can not be updated
step
> > by step  to  increse it ?
>
> Very simple answer.
>
> Let's assume that the table day one include the following characters:
>
> A
> B
> C
> D
> E
> F
>
> We agree that the simple equivalence rules for 1-1 mapping maps A->B.
>
> This means that the following characters are available for domain name
> registration:
>
> B
> C
> D
> E
> F
>
> People register domainames such as:
>
> B.example.com.
> C.example.com.
> D.example.com.
> E.example.com.
> F.example.com.
>
> Five different people register these five domains, which are different.
>
> If one type in A.example.com in the browser, nameprep will map that name
to
> B.example.com. So, A.example.com and B.example.com are treated as "the
same
> domain".
>
> Now, let's say that we increase the size of that table, and decide that D
> and E are equivalent, so D->E mapping should be added.
>
> This means that only the following characters are allowed:
>
> B
> C
> E
> F
>
> The domains D.example.com and E.example.com are equivalent.
>
> Now the questions, and I want you to answer them. Short answers are
> required:
>
> (1) Should the holder of D.example.com or the holder of E.example.com be
> forced to give back his domain?
>
> (2) Who is making that descision?
>
> (3) Who will tell the domain name holder that he is not allowed to have
his
> domain anymore?

          This is a very good example we all concerned . I answer your
problem by your
proposal (you  tell  Prof. Ho in another mail about the version update and
control). In  draft-ietf-idn-nameprep-05.txt :

A subsequent version of this document that references a newer version of
ISO/IEC 10646 with new code points will inherently have some code points
move from category U to either D, MN, or AO. For backwards
compatibility, no future version of this document will move code points
from any other category. That is, no current AO, MN, or D code points
will ever change to a different category.

          The approach assign the data set in some states,  From created
state to final fixed state , it is transited in one direction way on the
state diagram, it can not back and loop if you want the results are backward
compatible.   That is say if you want to keep backward compatible , you can
differentiate it step by step but you can not merge them further except you
can force the conflict one disappear.  Our language teacher always tell us
TC is always compatile to old age, that can explain why  there are so many
characters in chinese.
          By this principle,  why partial set of CJK characters that are
partial setted by local language tag can be used with different TLD
(cn,jp,tw..). Because the TLD implied the language tag and will let them
differentiable from other TLD .  The problems com from these contraints:
          1. Big table of UNICODE let domain name can be mixing registed but
viewed in part now.
          2. Many duplicate, easy-to-confuse and
un-normalied/non-canonicalied scripts in UNICODE.
          3. No one can change the coding at one night , you can only
transit step by step.
          4. ML.com try to use all code point withou considering the
troubles come from mixing.

L.M.Tseng

---

From: Patrik Fältström <[email-masked]>
Date: Sat, 01 Sep 2001 08:49:54 +0200

--On 2001-09-01 13.26 +0800 "tsenglm@????.??.tw" <[email-masked]>
wrote:

> differentiable from other TLD .

You have to be more precise.

(A) In DNS, we can only use one character set. One only. We have picked
Unicode.

(B) In DNS, the matching algorithm have to be the same in every piece of
software which uses the DNS. It can NOT differ between different languages
used by the client. It can NOT differ between different domains. It can NOT
differ between geographical regions. The matching algorithm we have is
nameprep.

(C) A TLD CAN have a policy which says that only a subset of the characters
allowed according to (A) is allowed.

You don't specify when you say "TLD" and "language tag" above whether you
imply a restriction according to (C) (which is ok) or if you have a
requirement which is a vilation to (B), or if you have a violation to (A).

Be more specific.

The proposals I have seen maybe look nice from a functional standpoint, but
they all have completely ignored the technical constraints the DNS system
has. If some of you which oppse the use of one character set and nameprep
came up with a solution which makes sense technically, we could talk.

In this mail, you once again show that you completely have missed the point
of the comments from the other members of this working group.

You have to differ between comments on flaws in the technical solution
proposed and flaws in the functional specification you present.

>  The problems com from these contraints:
>           1. Big table of UNICODE let domain name can be mixing registed
> but viewed in part now.

Yes.

> un-normalied/non-canonicalied scripts in UNICODE.

Correct. And your problem is? There is a reason why not more normalization
has been done in Unicode. UTC and ISO has been working for years on this
problem. Why do you think that IETF is better on doing the work than ISO
and UTC?

>           3. No one can change the coding at one night , you can only

That is why we propose the ACE encoding.

> troubles come from mixing.

If you use all code points without mixing them, what is the problem? The
problem comes when you try to mix characters from different scripts. The
problem comes when you try to normalize between characters in different
scripts. This is why UTC and ISO has not done more normalization. It is
also explained in the Unicode Standard 3.0 if I don't remember wrong. Yes,
on page 961 and 962 in the Unicode Standard 3.0.

What you and Liana are doing is claiming that the work the last 20 years,
the cooperation between all countries which use CJK characters in ISO is
wrong, and that IETF should do the job instead.

I think you should be VERY careful in your statements when you say that the
work in ISO has flaws.

This discussion has for me been a completely vaste of time, and I will
really think twice before I respond to another message in this thread.

I will not respond if not _constructive_ proposals which do work given the
technical constraints we have in DNS.

Just waving with hands doesn't work.

      paf

---

From: "James Seng/Personal" <[email-masked]>
Date: Sat, 1 Sep 2001 16:14:50 +0800

Dear Prof Tseng,

While using "TLD" to determine language or charset is one possible
*policy* solution, unfortunately, DNS is an resolution infrastructure
which should be independent of "TLD". It is by historical co-incident
that most TLDs are given to each country.

Suppose I have a .sg domain names, why *must* it be in Chinese? Why not
in Tamil or European? If I am a french company having an office in
Taiwan with a .tw domain names, why cant I use french names for my own
machine? Perhaps it *must* because it is of some policy set by SGNIC or
TWNIC but that is a policy by respective NICs, and should not come from
IETF. We engineers needs to design our protocol to be as flexible as
possible.

And when we say IDN, we are not even talking about 2LD..it could be 3LD,
4LD or even TLD itself. And in DNS infrastructure, it is not only used
for domain names as we know it. Many other research are using DNS in
some other ways which both you and me have no idea (I know of one
research which is trying to do biochemical naming using DNS).

Therefore, it is important that technical solution should be as
independent of any policy and not biased against any TLD, 2LD etc in
particular.

-James Seng

---

From: "Soobok Lee" <[email-masked]>
Date: Sat, 1 Sep 2001 17:36:44 +0900

> troubles come from mixing.

Even   labels  from single script block ( e.g. cyrillic )
 cause severe problems with  already established LATIN 2LD domains in gTLD
(roughly around 30,000,000 ).

This IDN WG is so lucky that the big companies mentioned in the enclosed
examples  have not  raised any trademark concerns yet.  Surely, still we have
some time to make more concerete our standard efforts
before they and the public become aware of these problems.

IMO, this problems could be handled  best on the  upper layer
above nameprep  by human beings aided by  tools and policies.

-LSB
----------------------------------------------------------------
look into these   <all cyrillic>.com

%u0406%u0412%u041C.com
ІВМ.com

%u0410%u0422%u0422.com
АТТ.com

%u0421%u0406%u0405%u0421%u041E.com
СІЅСО.com

%u0405%u0415%u0425.com
ЅЕХ.com

%u041D%u041E%u041C%u0415.com
НОМЕ.com

%u0410%u0412%u0421.com
АВС.com

http://www.postel.co.kr/etc/f6.html

---

From: [email-masked] <[email-masked]>
Date: Sat, 1 Sep 2001 17:59:50 +0800

> You have to be more precise.
               We talk the version , updating and how to keep backward
compatible.
And you give a good example:
Let's assume that the table day one include the following characters:
{A,B,C,D,E,F,G,H,c}
We agree that the simple equivalence rules for 1-1 mapping maps A->B.
This means that the following characters are available for domain name
registration:
{B,C,D,E,F}

First , let us assume G,H are reserved  , because they are not frequently
used. and A->B is not the reason of  easy-to-confuse, it is different font
shape but with the same meanning. c->C  because they are easy-to-confuse not
only in meannig but also in similar shape.
Comparing to the above two set, the 1-1 mapping let {A,c} is not allowed in
registration  table. {G ,H} are also not in the registration table  It is
like to let {A,G,H,c} are limited to use for further extension. So ,  you
have further chance to let A included in the furture set . After
A.example.com is allowed in registration table and you can also let it to be
assigned as B.example.com  for backward compatible first .  Then you can let
"A" available to other .
         But the {c,C} is in the set of easy-to-confuse , so {c} is fixed
mapping to {C}.
(BC.tw , Bc.cn) (AD.tw , BD.cn)  are all workable and may  not be confused
in {A,B}{C,c} . But  (BC.com  Bc.com ) (AD.com BD.com) will be confused in
the same domain .
         If you must let {A,G,H,c} in .com ,  We suggest to divided them
into 3 parts ,  {c,C} in nameprep and {A,B} in language related keyword
system , {G,H} reserved. By letting the character set size small in
frequently using, the problems can be fixed easily in 1-1 mapping.

> requirement which is a vilation to (B), or if you have a violation to (A).
        The (C) is policy based , so how do you restrict .COM   not  to
produce more confusing ?   In a ccTLD like cn,   japan language domain name
will not be registed in their domain,
so , the confusing can be restricted to easy-to-confuse characters only,
not need to concern the sementic reated characters.
> came up with a solution which makes sense technically, we could talk.
          The solution of  character mapping in nameprep  will assign a
standard characters in the equivalent set , so  CJK area will not happy in
this situation. But if  you let them find later that  confusing in
characters mixing in gTLD will happen then they will be more angry.
Realname's keyword is fine in language separation but it also need to keep
the uniqueness in name.

> of the comments from the other members of this working group.
        Sorry,  I do not against your comments but I am not so insist to
suggest my draft , if  there are better solution existed I will accept them.
I do not like to hear someone hate we use a 8bit clean  BIND compatible DNS
server that can do multibyte characters lookup and  TC/SC equivalence
comparations. MicroSoft has showed all the possibility ,  I think there are
a few differences .

> problem. Why do you think that IETF is better on doing the work than ISO
          I  do not expect it .  We know the constraints in our language
scripts , so we  suggest the approach to avoid the big trouble in TC/SC and
hope it can be improved step by step. I also hope this approach will help
other language.
> wrong, and that IETF should do the job instead.
        It is impossible to IETF.  The TC/SC problems come from PRC and
Taiwan in internal civial war, the mixing of  TC/SC will happen in HK, Mo.
Even now , TC/SC is not a good topic in Taiwan.

> I think you should be VERY careful in your statements when you say that
the
         Actually, in CJK area only library people like to use ISO 10646
code set , at least it contains all scripts in all old books. You can find
the true and ask why they all use native code set in other area.
> technical constraints we have in DNS.
      For your patients,  I will  ask Prof. Ho to  ajust our draft and try
to merge it with your,  if it is possible .

L.M.Tseng

---

From: Patrik Fältström <[email-masked]>
Date: Sat, 01 Sep 2001 11:29:24 +0200

> above nameprep  by human beings aided by  tools and policies.

Bingo! Exactly right.

  paf

---

From: Patrik Fältström <[email-masked]>
Date: Sat, 01 Sep 2001 12:21:11 +0200

--On 2001-09-01 17.59 +0800
"=?UTF-8?B?dHNlbmdsbUDoqIjntrLkuK3lv4Mu5Lit5aSnLnR3?="
<[email-masked]> wrote:

> code set , at least it contains all scripts in all old books.

Not the librarians I have talked with. They like some ideas of 10646, but
they don't use it.

They use the version of GB or BIG5 (or whatever) depending on what
character they need. Further, if they write text which is supposed to be
possible to read some time from now, they choose only one character set per
text, and leave the characters which doesn't exist as "non-existing". I.e.
mixing BIG5 and GB in the same text is a bad thing.

One reason for not using 10646/Unicode is because no font exists for it.

   paf

---

From: Patrik Fältström <[email-masked]>
Date: Sat, 01 Sep 2001 12:37:38 +0200

--On 2001-09-01 17.59 +0800
"=?UTF-8?B?dHNlbmdsbUDoqIjntrLkuK3lv4Mu5Lit5aSnLnR3?="
<[email-masked]> wrote:

>> >           By this principle,  why partial set of CJK characters that
>> >           are partial setted by local language tag can be used with
>> > different TLD (cn,jp,tw..). Because the TLD implied the language tag
>> > and will let them differentiable from other TLD .
>> 

No, if the mapping is from A->B. Then you can use {B,C,D,E,F,G,H,c}.

Further, A is not allowed in the zonefile, but a user can type 'A'. I.e. he
can use 'A' as input to a domain name lookup.

> used.

What do you mean by "reserved"? Do you mean "forbidden"?

> and A->B is not the reason of  easy-to-confuse, it is different font
> shape but with the same meanning. c->C  because they are easy-to-confuse
> not only in meannig but also in similar shape.

Wrong. The mapping A->B is there because the Unicode Consortium Technical
Report #15 has defined the normalization process that way.

You start talking about "confusion", and that is a term which this WG has
defined being out of scope. The users WILL be confused. They are already
today.

> Comparing to the above two set, the 1-1 mapping let {A,c} is not allowed
> in registration  table.

Wrong. When a user want to register something with 'A', 'B' will be
registered. This means that when a user types A, he will get a match on B.

When a user types 'c', he will match 'C'.

Only 'B' and 'C' is registered.

> {G ,H} are also not in the registration table

No, because they are forbidden (I guess).

That is a different problem.

> It
> is like to let {A,G,H,c} are limited to use for further extension.

Wrong. A and c are already included in the mapping tables, to B and C
respectively.

Only G and H are not used.

> So ,
> you have further chance to let A included in the furture set .

No. That is completely wrong!

> After
> A.example.com is allowed in registration table and you can also let it to
> be assigned as B.example.com  for backward compatible first .

No.

> Then you
> can let "A" available to other .

No.

>          But the {c,C} is in the set of easy-to-confuse , so {c} is fixed

Correct.

> in {A,B}{C,c} .

It will, as 'A' and 'B' are defined to be the same character.

> But  (BC.com  Bc.com ) (AD.com BD.com) will be confused in
> the same domain .

They are already defined to be the same as we have the mapping 'A'->'B'.

So, they are the same. You can not register both A.tw and B.tw, just like
you can not register A.com and B.com. It doesn't matter what TLD you talk
about.

The matchin rules must be the same.

I have told you this several times now, and I am REALLY tired on repeating
myself all the time!

> frequently using, the problems can be fixed easily in 1-1 mapping.

This will not work. Full stop! See above.

>> software which uses the DNS. It can NOT differ between different
>> languages used by the client. It can NOT differ between different
>> domains. It can
> NOT
> produce more confusing ?

I don't care.

Read the paragraph above (C) again.

> Realname's keyword is fine in language separation but it also need to keep

Read the drafts by Dr Klensin and Mr Mealling again as you obviously have
not understood at all what they talk about.

> I do not like to hear someone hate we use a 8bit clean  BIND
> compatible DNS server that can do multibyte characters lookup and  TC/SC
> equivalence comparations.

See the rules (A), (B) and (C) above. You break those rules.

> MicroSoft has showed all the possibility ,  I
> think there are a few differences .

No, they have showed that it is impossible.

> Even now , TC/SC is not a good topic in Taiwan.

But, you are the one which say that we should ignore what the
representatives from the governments have agreed to in ISO!

    paf

---

From: [email-masked] <[email-masked]>
Date: Sun, 2 Sep 2001 00:09:16 +0800

Hi!  Fältström :
       Please read it completely again,  I  just  talk  how to reduce the
confusing of mixing scripts by release partial set of CJK  to different area
by virtual using and update them step by step.
If  you want the IDN fail in confusing and no conversion of  CJK , you can
by-pass it.

Let's assume that the table day one include the following characters:
{A,B,C,D,E,F,G,H,c}
We agree that the first simple equivalence rules for 1-1 mapping maps is
A->B and the reason for mapping are they have different font shape but with
the same meanning. the second  1-1 mapping is  c->C  because they are
easy-to-confuse not only in meannig but also in similar shape. Let us also
assume G,H are reserved  , because they are not frequently used and can not
type-in from input methods.

This means that the following characters are available for domain name in
REGISTRATION and zone file : {B,C,D,E,F} , user can select A,c , but in
registration time, he will be noted  A->B , c->C . He can use A as B
virtually if the mapping rule applied , but the primary and actual name is
B.

{G,H} is  "reserved",  so they are "forbidden" in using now . Reserved them
just to reduced the set size to check the equivalence mapping now.

> defined being out of scope.
             Out of scope just mean to neglect and assume they are
non-exist, We know it is not true,  it just ask you must focus now and
solved them later. But if the later term is a fatal error in the proposed
system , it will not work well.

in zone  table.
> registered. This means that when a user types A, he will get a match on B.
{ A ,c } will be present in registration data table , but it will be telled
that A will be  treated as B and c is treated as C .  If  you  use
reversable ACE , only  B and C  are corresponding created in  zone file.
> No, because they are forbidden (I guess).
It is like to let {A,G,H,c} are limited to use for further extension.
> Wrong. A and c are already included in the mapping tables, to B and C
     A and c are in mapping table now . But now we want to let A and B are
treated as different in some furture time and keep {C,c} the same.
If A are allowed only virtually used , you have further chance to move  A
into  the zone file for different address mapping.  In a non IDN-aware
system , they will found B is exist but A are not.

> No. That is completely wrong!
 I  suggest you cold down to think how to expansion and update the table.
You will not take keyword as a actual domain name in name mapping.

 After A.example.com is allowed in registration table and you can also let
it to
be assigned as B.example.com  for backward compatible first .
> No.
 Then the virtually used  "A" can be changed  to a real location in zone
file as needed in furture and can be available to  use as Address RR for
others .

L.M.Tseng

---

From: "tsenglm@????.??.tw" <[email-masked]>
Date: Sun, 2 Sep 2001 01:41:09 +0800

Dear James:
                    I  am here are not to put ccTLD as the IDN TLD.  All the
thing I concerned is not to let SC/TC of CJK fall in a confusing & battle
status especially for users.                        We put SC/TC in tw and
cn and try to forget the trouble in plictical pressure , in this condition
why we against any language or scripts will used in tw ?
                    As a chinese language user , you must konw the  openness
characteristic in chinese characters.  You are younger so you can not
understand what happened in TC/SC
especially in PRC-SC.  But you must know the troubles in less normalization
and people can not easily communicated in Taiwan and China. You also know
that HK and MO are returned to China.  Mixing TC/SC will produce more
confusing in CJK area and that will produce more troubles in gTLD.
                    An Engineer will do right thing and try best not to do
something hurt others. No matter it is client solution or server based
solution , and what level domain that will be applied, the most basic issuse
is to reduce easy-to-confuse term .  You know the frequently used characters
are almost the same in Taiwan and China, most of them are different code
point in UNICODE and many of them are very similar especially in some small
difference in radical.  The most bad thing is people do not know the
confusing is existed in high probability.
                     Language tag is only a technique to reduce the
complexity in TC/SC ,  I do not expect programmer will like many special
case.   I  just enhansize the issuse can not be neglected.  We all know the
registration of ML.com will renew around November,  so there are time
schedule pressure ,  I  hope they can work in right way and produce less
confusings in using TC/SC.  Actually , I should not to worry them if I
assume they have troubles in it, but it is not so fair to users. For CJK
users , we need it work properly.

L.M.Tseng

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Sat, 1 Sep 2001 11:26:56 -0700

At 1:41 AM +0800 9/2/01, tsenglm@????.??.tw wrote:
>I  just enhansize the issuse can not be neglected.

It is not being neglected. It has been addressed repeatedly here. 
Because there cannot be a universally-deployed technical solution in 
IDN, it must be deployed as rule that zone administrators know that 
they should follow.

>   We all know the
>confusings in using TC/SC.

VeriSign GRS is a zone administrator like all other zone 
administrators. If they mess up, the folks controlling them (in this 
case, ICANN) should tell them to fix the problem.

>   Actually , I should not to worry them if I

For *all* users, we need it to work properly. Please remember that 
most people in the world will find interface problems with IDN, just 
the way they find them with the current DNS. (Said a different way, 
we will probably manage to offend almost everyone with whatever we do 
here.) If we can make sound technical decisions to reduce the 
problems, fine; if not, we should finish and let the most-qualified 
people work on the non-technical solutions.

It is clear that we cannot find sound technical solutions to the 
simplified-traditional problem. So, we should move on, making sure 
that whatever technical solution we create doesn't *prevent* a later 
non-technical solution.

If you can show where the current IDNA/ACE/nameprep solutions prevent 
an eventual traditional-simplified solution, please speak up. 
Otherwise, it is probably better to spend your time starting to work 
on the non-technical solution, such as suggestions to zone 
administrators. The sooner that happens, the sooner that the users 
will get what they want.

These statements may sound harsh to the people who have never 
participated in the IETF before now, but they are the way that every 
protocol is made. For every protocol, problems that could not be 
solved technically in the protocol get solved in the marketplace. 
This is true for HTTP, SMTP, Internet telephony, and on and on. IDN 
will be no different.

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: "Mark Davis" <[email-masked]>
Date: Sat, 1 Sep 2001 12:15:40 -0700

The Unicode Technical Committee discussed this issue at meeting # 88 (Aug
14 - Aug 17). It recommends that Traditional-Simplified Chinese mapping not
be added to nameprep. In practice, such names will either all be in
traditional or all be simplified, thus normally having only two variants.
Thus the combinatorics are far less of a problem than in case mapping.

Clarifying text should be added to nameprep or other appropriate places, to
the effect that:

"Nameprep does not account for all of the variations that may occur or that
a user might expect. In particular, it will not account for choice of
spellings within the same language. Examples include "theater.com" vs.
"theatre.com", and "hemoglobin.com" vs. "h<U+00E6>moglobin.com" in American
vs. British English. Other examples are simplified Chinese spellings of
domain names (e.g., "<U+7EDF><U+4E00><U+7801>.com") vs. the equivalent
traditional Chinese spelling (resp. "<U+7D71><U+4E00><U+78BC>.com"). Nor
does nameprep account for language-specific equivalences such as
"aepfel.com" vs. "<U+00E4>pfel.com" (considered equivalent in German). For
such variants, multiple entries in zone files are recommended (such as
multiple registrations).

A different RFC, possibly not on the standards track, that discusses this in
more detail and provides guidelines for avoiding possibly-confusing
registrations could help people to understand the issues more clearly.

Mark Davis
Cathy Wissink

---

From: Eric Brunner <[email-masked]>
Date: Sat, 01 Sep 2001 17:51:10 -0400

Mark,

Please offer your understanding of the UTC-88 comment and the tsconv draft.
Are these consistent, inconsistent, or unrelated?

> be added to nameprep.

Thanks,
Eric

---

From: "Mark Davis" <[email-masked]>
Date: Sun, 2 Sep 2001 16:33:47 -0700

Thanks for your note. I would be glad to try to clarify it. The UTC position
does disagree with the tsconv document, in maintaining that nameprep does
not require the addition of TC/SC folding.

I believe the consensus view* in the UTC is that SC/TC folding -- when done
correctly -- is quite complex (cf http://cjk.org/cjk/c2c/c2cbasis.htm), that
there is no established standard that precisely defines the conversion, and
that it is not a required feature of nameprep, since multiple registrations
will reasonably meet the user requirements. The latter point is based on
there being typically only two variant spellings: one TC and the other SC.

(This is different from the position vis-a-vis case folding; that case
folding is fairly straightforward, that there is an established standard
that precisely defines the conversion, and that the combinatorics of
multiple registrations quickly become overwhelming, and could not reasonably
be met with multiple registrations.)

This is not in any way to deny the legitimate needs of Chinese users, which
are extremely important to the Unicode consortium. The Chinese, Japanese,
and Korean ideographs *far* outnumber all other characters in the Unicode
standard combined. Their content is defined by the IRG (Ideographic
Rapporteur Group, a subcommittee of ISO/IEC JTC 1/SC 2), which is composed
of representatives of the governments of China, Hong Kong, Singapore, Japan,
South Korea, North Korea, Taiwan and Vietnam, plus a representative from the
Unicode consortium (cf
http://www.info.gov.hk/digital21/eng/structure/intro_irg.html).

Mark

* I'm speaking for myself (not the UTC) in the material following the
asterisk, since this material was not an explicit part of the UTC decision.

—————

Γνῶθι σαυτόν — Θαλῆς
[http://www.macchiato.com]

---

From: Eric Brunner <[email-masked]>
Date: Sun, 02 Sep 2001 21:13:51 -0400

> not require the addition of TC/SC folding.

The first being the product of a vendor consortia, the second a product of
the .CN and .TW NICs, and the subject matter being in the context of dns
identifiers. This is doubly awkward.

> there is no established standard that precisely defines the conversion, and

I'm interested in the consensus view in the PRC, and in Taiwan, also.

This next bit, starting at "since" to the end of the sentence.

> there being typically only two variant spellings: one TC and the other SC.

Earlier someone proposed, and eventually withdrew, a proposal to allow
a character-by-character, registrant election of equivalency, which has
combinatorial scaling properties. A vendor may also have advocated this.
I had dinner with their CTO once and got indigestion.

In any event, I can't imagine how a conversation in a code-point standards
body ended with a paraphrasing of "multiple (zone) registrations reasonably
meet user requirements". It is as odd as an IETF conversation ending with
the observation that "multiple code-point allocations reasonably meet the
glyph <foo>'s requirements as a character in scripts <bar> and <baz>".

In this one note I'm troubled to see the UTC presume to know better than
the sources available to the authors of tsconv, and to know better than
the authors of some eventual STANDARD and/or BCP trace APPS or DNS IDs.

I probably misread your mail.

I would like to know how the "typically only two variant spellings" rule
is manifested, or could be expressed as an equivalence rule. We did work
on the question of equivalency rule scope in the discussion of (former)
requirement [30], to motivate an interest in SC/TC conversion at the protocol
vs zone manager (nameprep or not) level.

> asterisk, since this material was not an explicit part of the UTC decision.

Informed guessing is good, and thanks for responding. I hope to see you in
DC next year.

Eric

---

From: [email-masked] <[email-masked]>
Date: Mon, 3 Sep 2001 11:28:17 +0800

> glyph <foo>'s requirements as a character in scripts <bar> and <baz>".
           Current  Letter-Digital-Hyphen DNS  can accept any case
caharacter string in zone file and query lable. It  hidden  to-lower-case
conversion in comparasing time.  The ACE approach let the case mapping
separated in registration time and query time. It must to select one in
equivalence set to be assigned as a standard representation.  Any  CJK
people can not understand why our character should be transformed to other's
character. So it is a big trouble to make concensus in  full scale TC/SC
mapping.
           As you point out there existed a combinatorial scaling problem in
TC/SC mixing if the  case-holding-like problem can not be solved.  In  the
TC/SC equivalence problem of chinese charaters , there are a small set of
them are more critical ,  some of them, especially PRC-SC , are
easy-to-confuse with TC  of  Japan, korean, abd Taiwan.  This part can be
easily checked by each ccNIC  in viewing the script font .  These common and
critical part are the necessary part of character mapping in nameprep.
Adding number of them will reduce the scale of TC/SC combination , but
reducing of them can keep  the flexibility for  different language using.
This is a typical name identifier problem.  All  CJK related NIC may  get
the concensus in knowing these two limitations.  It is not a problem of
language conversion,  it just to do fair for users.
          The value of domain name come from the easy-to-use that cause the
mass population  and the uniqeness of name let user like to keep it .  To
reduce the confusing  will be an important issue.
          To put all of TC/SC word conversion into IDN system is not
necessary and impossible in her large character base. But just to keep  all
TC/SC out of nameprep will be not a good solution too.  Why not try to find
only some easy-to-confusing character set and put them in character mapping
of nameprep ?

L.M.Tseng

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Sun, 2 Sep 2001 20:52:26 -0700

At 9:13 PM -0400 9/2/01, Eric Brunner wrote:
>meet user requirements".

Others might not be as incredulous. As Mark said, there are 
essentially only two zone registrations, and that is really only 
needed for names where the user might do a traditional-to-simplified 
conversion.

>  It is as odd as an IETF conversation ending with

No, that is completely different. In the T-S case, the question is 
how might the user who does an inherent conversion be accommodated, 
and who will be doing the accomodating (the protocol or the zone 
holder); in the code-point case, you are not accomodating a user, and 
you are only making the change in the protocol.

>the authors of some eventual STANDARD and/or BCP trace APPS or DNS IDs.

Why? The IETF looks to outside experts for advice, and I think it can 
reasonably be stated that the UTC are experts on Unicode. Until this 
WG, I had *never* heard anyone claim that a typical person in China 
would do a partial conversion of a name to a mixture of traditional 
and simplified. This is not to say that the CNNIC/TWNIC folks are 
wrong; they are clearly trying to make a strong case for helping 
native Chinese speakers more than the DNS ever has.

Questioning the UTC's expertise in scripts, and deferring instead to 
national name registries who have very different concerns, may make 
sense for a policy-based body like ICANN, but probably not for a 
standard-based body like the IETF.

--Paul Hoffman, Director

---

From: "James Seng/Personal" <[email-masked]>
Date: Mon, 3 Sep 2001 15:20:52 +0800

Dear Prof Tseng,

> confusing is existed in high probability.

Yes, I am very aware of the difficulties Chinese have between TC & SC. I
have no given up hope on it yet of course and still would like to see a
solution. But I am quite convience a solution in the Unicode
Normalization space is not possible because TC-SC is not a simple
codepoint normalization thing. It is far more complex involving
lexemical, orthographical and contextual conversion.

I also understand one of the arguments about doing partial TC-SC. Yes,
it is true that some TC-SC can be done using codepoint normalization.
And we could further reduce that set to avoid conflict with Korean and
Japanese. I have not done enough research in this aspect to give you my
view on this yet.

But AFAICS, the pros as is it does help to avoid TC-SC confusion a bit,
not completely but better than nothing. The cons to this is that it may
mislead people into thinking IDN is able to handle TC-SC completely.

> users , we need it work properly.

Actually I am not concern about ML.com renew or registration. That is a
problem Verisign have and their problem. Neither do we schedule our work
to fit their pressure. This WG have our own charters and goals and I do
not think we should adjust what we do for any particular companies.

If your concern is to help Verisign to do TC/SC correctly in ML.com,
then the best way is to talk to Verisign directly since we (in this wg
or ietf) have no control what they (or any particular company) do
anyway.

-James Seng

---

From: Eric Brunner <[email-masked]>
Date: Mon, 03 Sep 2001 08:08:44 -0400

Paul,

If the UTC wants to dip into zone administrator operational technique, it
doesn't mean they can do so correctly. Desire isn't competency. When they
publish their analysis of the set of mechanisms they evaluated, and a
comment on why the mechanism they selected as "reasonable" was, and on why
the alternatives were not "reasonable", their comment on A records may
meet the usual test for content, coverage and correctness.

I've lost count of the number of times people have consisdered "fixing" a
code point problem in this WG, and have been told it is out of scope. The
fences between us (IETF and UTC, and ICANN, and W3C, and ITU, and ETSI),
help us, they don't hinder us.

I'll pass on your last gem. There are reasons to question the work of any
group, including the UTC, arising from their abilities and accomplishments,
that don't require putting on an ICANN hat. Your point does come back to a
point of whether or not we (the IETF, not this WG) can accomodate national
standards other than ASCII, or can only, due to some form of universalism,
accomodate a (unique) multi-standard, which quite possibly depricates some
national standard(s).

When a national standards body has something to say, particularly when it
does so in disagreement with the UTC, it is a poor choice to pick either
on a basis other than discernable correctness.

Lets stick to the engineering, not haigiography (worship of UTC saints) or
deamonology (worship of ICANN deamons), and work out which, the UTC or the
authors of tsconv, got the better end of the stick, if either.

Eric

---

From: "Mark Davis" <[email-masked]>
Date: Tue, 4 Sep 2001 08:50:54 -0700

Eric, I'll interleave some responses below.

Mark
—————

Γνῶθι σαυτόν — Θαλῆς
----- Original Message -----
From: "Eric Brunner" <[email-masked]>
To: "Mark Davis" <[email-masked]>
Cc: <[email-masked]>; "Paul Hoffman / IMC" <[email-masked]>
Sent: Sunday, September 02, 2001 18:13
Subject: Re: [idn] UTC recommendations on TC/SC

> identifiers. This is doubly awkward.

It is a bit misleading to characterize the Unicode consortium as a 'vendor
consortia', any more than it would be to apply the same moniker to the W3C
or IEEE. The consortium includes more than just vendors; I'll include the
current membership here, in case people find it useful:

Corporate Members

Adobe Systems, Inc.; Apple Computer, Inc.; Basis Technology Corporation;
Compaq Computer Corporation; Government of India Ministry of Information
Technology; Government of Pakistan, National Language Authority;
Hewlett-Packard Company; IBM Corporation; Justsystem Corporation; Microsoft
Corporation  NCR Corporation; Oracle Corporation; PeopleSoft, Inc.; Progress
Software Corporation; The Research Libraries Group, Inc. (RLG); Reuters,
Ltd.; RWS Group, LLC; SAP AG; Sun Microsystems, Inc.; Sybase, Inc.;
Trigeminal Software, Inc.; Unisys Corporation.

Associate Members

Agfa Monotype Corporation; Beijing Zhong Yi Electronics Co.; BMC Software,
Inc.; Booz, Allen, & Hamilton, Inc.; Cable & Wireless HKT Limited;
CDAC-Centre for Development of Advanced Computing; China Electronic
Information Technology Ltd.; The Church of Jesus Christ of Latter-day
Saints; Columbia University; Data Research Associates; DecoType, Inc.;
Endeavor Information Systems, Inc.; eNIC Corporation; epixtech, Inc.;
Ericsson Mobile Communications; eTranslate, Inc.; Ex Libris, Inc.;
GlobalMentor, Inc.; GlobalSight Corporation; The Government of Tamil Nadu,
India; iDNS; i-EMAIL.net Pte Ltd; Innovative Interfaces, Inc.; Internet Mail
Consortium; Langoo.com; Language Analysis Systems, Inc. Language Technology
Research Center; Netscape Communications; Nokia; Nortel Networks; Novell;
OCLC, Inc.; Openwave Systems, Inc.; Optio Software; Palm, Inc.; Production
First Software; The Royal Library, Sweden; Sagent Technology, Inc.; SAS
Institute, Inc.; SHARE; Siebel Systems; SIL International; SIRSI
Corporation; SLANGSOFT; Software AG; StarTV - Satellite Television Asia
Region Ltd.; Symbian, Ltd.; Uniscape, Inc.; Verisign Global Registry
Services; VTLS, Inc.; WALID, Inc.; WordWalla, Inc.; Yet Another Society

> I'm interested in the consensus view in the PRC, and in Taiwan, also.

As are we. The UTC, as other organizations, works with the best available
knowledge available to it: if there is evidence that it is common to mix SC
and TC arbitrarily within names -- and that there are widespread
expectations among users that such arbitrary mixtures must always match,
then that
would certainly be useful information to bring forward.

For example, samples of publications -- books, magazines, journals, etc. --
in which the same names or phrases are spelled with different mixtures of SC
and TC (not just all SC or all TC) would be valuable.

> glyph <foo>'s requirements as a character in scripts <bar> and <baz>".

Perhaps it should have been better phrased (and remember, I was speaking for
myself, not for the UTC). The point is that if there is little arbitrary
mixing of SC and TC in the same names, then there will usually be two
variants.

I do not presume to know the internals of the DNS; but if I were designing a
software library that matched names, and if the matching process was not
well understood, then only having a small number of variants would be a very
important fact that can be taken into account when doing the design. I would
at that point very strongly consider just having the variants in the table,
rather than try to devise software that produced a match (and did not match
items that shouldn't match).

Forgive me if I stepped over some line in the description.

> the authors of some eventual STANDARD and/or BCP trace APPS or DNS IDs.

The UTC does not 'presume to know'; it is making a recommendation for a
solution to a issue that is important to be resolved, and resolved in a
timely manner. It is a recommendation to the idn committee; by no means a
fiat!

> vs zone manager (nameprep or not) level.

I don't quite understand what you mean here. The point I was trying to make
is that if it is
difficult to come up with precise matching algorithm and data, and there are
typically only two variants, then you don't try to institute an equivalence
rule: you simply "register" (using this in a broad sense, not in an IETF
sense) both names.

Thus you don't try to programmatically predict that
"theater" and "theatre" should match, and "maker" and "makre" shouldn't,
nor "aluminum" and "aluminum" but not "lithium" and
"lithum", "catalog" and "catalogue" but not "league" and "leag". (Let alone
"petrol" and "gasoline"). Instead, you simply add both sets of alternatives
to the internal word-list.

(Of course, British and American variants of English are not precisely
analogous to the case of TC and SC, but you see the point.)

> Informed guessing is good, and thanks for responding. I hope to see you in

I would enjoy that. These email discussions are never as interesting nor as
productive as discussions in person! Unfortunately, I probably will not be
in DC, so it will be another time.

---

From: Jack Halpern <[email-masked]>
Date: Wed, 05 Sep 2001 08:25:30 -0500

Greetings

In message "Re: [idn] UTC recommendations on TC/SC",
Mark Davis wrote...
 
 >then that would certainly be useful information to bring forward.
 >
 
I have heard claims of mixing SC and TC from Basis Technology too, but upon
close examination it turned out to be a false alarm. I have never seen "genuine"
SC mixed with "genuine" TC, unless it is in linguistic discussions. I would
like to see actual examples if someone has some. 

It is easy to misjudge this and may requires a deep understanding of the issues.
Keep in mind that many SC charcaters are historically TC abbreviations that 
are often used in proper nouns.  It is also necessary to underastand exactly 
what "SC" and "TC" actually refer to (discussed in detail in my paper.

Just my two yen/yuan.

Regards, Jack Halpern
	President, The CJK Dictionary Institute, Inc. 
        http://www.cjk.org Phone: +81-48-473-3508