Subject: Re: Layer 2 and "idn identities" (was: Re: [idn] what are the IDN identifiers?)

From: liana Ye <[email-masked]>
Date: Thu, 6 Dec 2001 13:58:12 -0600

We are discussing how the registras to 
"to avoid registering mixed scripts 'names' ".
Can you suggest any way of doing this, or
any feasible guide lines?

On Wed, 5 Dec 2001 16:52:40 -0800 "Michel Suignard"
<[email-masked]> writes:
> This has nothing to do with the CJK TC/SC issue. It should be a
> guideline to registars to avoid registering mixed scripts 'names' 
> when
> they don't make sense. And finding any similiarities between Latin
> characters and Armenian characters is a far fetched example. It 
> doesn't
> take much when reading an Armenian document that it is not Latin, 
> even a
> single word is sufficient. Thinking that Armenian or Hebrew for that
> matter could be confused with Latin characters is simply incorrect.
> It is a global issue of any large character set repertoire that you 
> may
> have some confusion possible between 'look-alike' characters. 
> Nameprep
> is there to simplify somehow the issue, registars guidelines are 
> another
> part .
> 
> And I don't know what you are talking about when you say that you 
> know
> the user's language from input processing, globalized platforms have
> input mechanism that are typically independent from language 
> context.

Then what about your IMEs, are they independent from 
language context?  What is the percentage code points
in UCS under IMEs' processing?

> The only thing that is known at input time is a code point. Upper 
> layers
> may decide to apply heuristic to guess a language but that is 
> clearly
> beyond the scope of input mechanism in modern input processing
> mechanism. And there is not such a thing as the selection of a 
> 'language symbol set'.
> 
> The only case where there is a reasonable determination of 
> 'language'
> for input is East Asian Input Method Editors (IMEs), and it could be
> reasonable to assume that an application layer could offer some 
> TC/SC
> services before feeding the code points to a DNS service in that 
> case
> for CJK characters (but even that is not a simple case to solve 
> because
> of the contextual ambiguity as mentioned several times in this 
> forum). 
> 
> Frankly you are not helping by creating these unrelated 'analogies' 
> with
> other languages and scripts that have nothing to do with the issue 
> at
> hand.
> 
> Michel
> 
> (PS I should add that I am also a Unicode technical director and the
> project editor for ISO 10646)

Now, I got it.  So,  I particularly think that because you are 
not an DNS expert, and you are not using IMEs to select 
codepoins to express yourself much either, that there are 
other dimensions you should look into, before you make
accusations like 

>And finding any similiarities between Latin
> characters and Armenian characters is a far fetched example.

Liana 

> -----Original Message-----
> From: liana Ye [mailto:[email-masked]] 
> Sent: Wednesday, December 05, 2001 11:29 AM
> To: Michel Suignard
> Cc: [email-masked]; [email-masked]; [email-masked];
> [email-masked]
> Subject: Re: Layer 2 and "idn identities" (was: Re: [idn] what are 
> the
> IDN identifiers?)
> 
> 
> It doesn't matter from input processing, 
> because you know the users' language. It is 
> defined by the user when they select the 
> language symbol set.  
> 
> But when you turn those symbols into codepoints
> and strip the language context, then comparing 
> them in languageless context of IDN, the problem is
> arised.  That is the reason I have to put Latin
> together with Armenian to make the CJK problem 
> a little easier for people do not know CJK.  There are 
> 6 lower case letters similar with Latin [a-z].
> 
> Liana
> 
> On Wed, 5 Dec 2001 12:09:00 -0800 "Michel Suignard"
> <[email-masked]> writes:
> > Liana, stop associating Armenian with Latin in your explanation. 
> All 
> > writing systems based on Latin and the single writing system using 
> the
> > Armenian share nothing except maybe some punctuation. It doesn't 
> > make
> > sense to make a parallel between
> > (Latin + Armenian + Cyrillic + Hebrew) and (CJK), because in the 
> > first
> > group no writing system share characters between the subsets 
> > (although
> > I would even object at creating such a 'logical' collection of 
> > largely
> > unrelated scripts), while in CJK all writing systems use 
> characters 
> > in
> > the same CJK blocks.
> > 
> > I doesn't help to explain the issue of TC/SC which is a valid
> > concern
> > for CJK users by using flawed analogy to a non existing model.
> > 
> > I myself see a need to help Chinese users deal with TC/SC, but I
> > don't
> > see it to belong in the scope currently covered by IDN.
> > 
> > Michel
> > 
> > -----Original Message-----
> > From: liana Ye [mailto:[email-masked]]
> > Sent: Wednesday, December 05, 2001 8:14 AM
> > To: [email-masked]
> > Cc: [email-masked]; [email-masked]; [email-masked]
> > Subject: Re: Layer 2 and "idn identities" (was: Re: [idn] what are 
> 
> > the
> > IDN identifiers?)
> > 
> > 
> > 
> > On Wed, 5 Dec 2001 01:17:36 EST [email-masked] writes:
> > > In a message dated 2001-12-04 20:11:19 Pacific Standard Time, 
> > > [email-masked] writes:
> > > 
> > > >> SC/TC equivalence itself is far simpler than the "four winds,
> > two
> > > eggs"
> > > >> equivalences, and has quite a bit of merit. I won't express 
> any
> > 
> > > >> real opinion on it until I study it further.
> > > >
> > > > It is not so simple as to be able to be done _accurately_ by 
> an
> > > code-based1-1
> > > > bit-string matching process. There are semantic, syntactic and
> > > contextual
> > > > considerations that require at the very least a morphological
> > > analysis
> > > process
> > > > in order for TC/SC to be done with a reasonable amount of
> > accuracy
> > > (i.e.
> > > > orthographically).
> > > 
> > > Thanks for saying with some authority what I have apparently 
> been 
> > > unable to communicate effectively, namely that TC/SC is not 
> merely a
> 
> > > 1-1 operation
> > > comparable to Latin case folding.
> > > 
> > > -Doug Ewell
> > >  Fullerton, California
> > > 
> > 
> > Excuse me for jump in, I have been keep silent on this
> > view,  and I'd like to comment on this issue now. 
> > 
> > TC/SC is not merely a 1-1 operation, if you only compare it
> > with Latin case folding in what the names imply:
> > 
> > TC/SC is a subset of Han, and Han is subset of C,J,K.
> > Latin is a super set of English, French,....
> > 
> > Can you see the flaw on such a comparison?
> > 
> > So when you look at Latin in the context of UCS code points,
> > since UCS is the set we are hoping to use blanketly in IDN, then 
> > Latin
> > is a subset of (Latin + Armenian + Cyrillic + Hebrew) since I 
> think 
> > this
> > is the area that Latin is mostly likely be 
> > used too.  
> > 
> > So this means if you compare TC/SC set of 1-1 cases
> > then the  Latin is 1-1.
> > 
> > If you compare TC/SC with 1-n, n-1, 1-1, that is in Chinese,
> > then Latin should be put into UCS Plane 0, 1, 2 too. 
> > So this Latin is n-1, 1-n too.
> > 
> > If you compare TC/SC in the sense of C,J,K block,
> > then Latin + Armenian is the minimum case to think about.
> > 
> > Cheers.
> > 
> > Liana
> >

---

From: "James Seng/Personal" <[email-masked]>
Date: Fri, 7 Dec 2001 11:36:37 +0800

Liana,

I think you are way out of line here so please drop that line of
argument, especially one which get personal.

You may be frustrated at a evil that does not exist. I shall try to
summaries what John's email to Deng few months back said, hope that
will help you understand the IETF.

To get IETF interested in a solution, there are typically two step.
1. Conviencing the IETF it is a worthwhile problem to solve
2. Presenting the solution to the IETF

For TC-SC, I think the group is pretty much convienced of the problem
and something we should at least attempts to solve. We already cross
that barrier. Any further examples of why TC-SC should be solved etc
would only frustrate others who is already in part 2 of the problem, ie
finding the solution.

Any analogy of other scripts to TC-SC is never going to be accurate,
whether you use Latin + Armeniam, or Japanese, or anything. Human
language by definition is complex, that makes our job even harder, and
each language is unique in its own way. In this sense, actually Michel
has a point, more than you care to admit.

Going back to our topic of the two step, given that the group is in part
2, what we need are proposed solutions. And so far, I dont think the
group here is convienced of any solution is worthwhile to pursue.
Sometimes we may have to say, "Sorry it is not possible to be done with
the existing technology we have now" even though we may agree with part
1.
Repeating part 1 again and again will not help anything because we
already
convienced of it, just not part 2, the solution.

To give you an analogy (one that is harder to dispute of course :-),
when IDN first started, we always on a basis that the group may declare
"no solution" as the conclusion. In fact, we may jolly well still do so
now, ie, we declare "no solution".

And to give you another example of the 2 step process, reordering is one
which we have a reasonable solution (part 2). Unfortunately, the group
here is not convience at the problem it solves is worth solving (part
1). In IETF, this is typically known as "solution looking for a
problem".  Therefore, do we still keep reordering in the pool in hope
that there will be a "problem" which will crop up in the future, or lets
drop it? It is a question the group have to ask ourselves.

-James Seng

---

From: Rick H Wesson <[email-masked]>
Date: Thu, 6 Dec 2001 20:00:40 -0800 (PST)

James,

On Fri, 7 Dec 2001, James Seng/Personal wrote:

[snip]

> would only frustrate others who is already in part 2 of the problem, ie

I think you miss-characterize the group think here. IMNSHO, TC-SC may be a
problem but it is out of scope for our work. If its to be solved it
shouldn't be solved by this working group and probably not by the IETF.

Also, I remember our discussions about the complexity of ACE encodings and
everyone worrying about complexity. TC-SC makes all those discussions pale
to the complexity raised by TC-SC and I can only assume there are more
similar issues lurking around the corner in other scripts.

-rick

---

From: "James Seng/Personal" <[email-masked]>
Date: Fri, 7 Dec 2001 12:15:59 +0800

I dont really think I have mis-characterized the group. I am pretty sure
we have an agreement on step 1, ie, TC-SC is a problem. So lets put it
aside, move on and stop discussing about why TC-SC is a problem. It aint
useful.

However, I also agree with you that the question if TC-SC can be solve
within IDN WG or IETF is a valid one, one of the first question we
should ask in step 2.

-James Seng

----- Original Message -----
From: "Rick H Wesson" <[email-masked]>
To: "James Seng/Personal" <[email-masked]>
Cc: <[email-masked]>; "liana Ye" <[email-masked]>;
<[email-masked]>; "Maynard Kang" <[email-masked]>;
<[email-masked]>; <[email-masked]>
Sent: Friday, December 07, 2001 12:00 PM
Subject: Re: Layer 2 and "idn identities" (was: Re: [idn] what are the
IDN identifiers?)

---

From: David Hopwood <[email-masked]>
Date: Fri, 07 Dec 2001 06:27:56 +0000

-----BEGIN PGP SIGNED MESSAGE-----

liana Ye wrote:
> We are discussing how the registrars to "avoid registering mixed scripts
> 'names' ". Can you suggest any way of doing this, or any feasible guidelines?

It's fairly trivial: define a formal syntax that registrars can check
against for domains that should exclude mixed names.

The general form of that syntax would be:

['\' means set difference, '/' means union, '&' means intersection]

  SCONLY = ... ; set of Han ideograph code points that are only used in SC
  TCONLY = ... ; set of Han ideograph code points that are only used in TC

  ; a string that is not mixed either contains no SCONLY code points,
  ; or no TCONLY codepoints.
  NotMixed = *(UNICODE \ SCONLY) / *(UNICODE \ TCONLY)

  HostString = ... ; whatever syntax is decided on
  NotMixedHostString = NotMixed & HostString

If you want to allow mixed TC/SC in strings that also contain Katakana,
Hiragana, Bopomofo or Hangul (on the grounds that these are not Chinese),
then define:

  MixedAllowed = *UNICODE (Scripts:HIRAGANA /
                           Scripts:KATAKANA /
                           Scripts:BOPOMOFO /
                           Scripts:HANGUL) *UNICODE

  AllowedHostString = (NotMixed / MixedAllowed) & HostString

Note that SCONLY and TCONLY can include code points that would have
1-many and many-1 mappings for TC/SC conversion, as well as 1-1 mappings.

- -- 
David Hopwood <[email-masked]>

Home page & PGP public key: http://www.users.zetnet.co.uk/hopwood/
RSA 2048-bit; fingerprint 71 8E A6 23 0E D3 4C E5  0F 69 8C D4 FA 66 15 01
Nothing in this message is intended to be legally binding. If I revoke a
public key but refuse to specify why, it is because the private key has been
seized under the Regulation of Investigatory Powers Act; see www.fipr.org/rip

---

From: Rick H Wesson <[email-masked]>
Date: Thu, 6 Dec 2001 22:48:23 -0800 (PST)

David,

> The general form of that syntax would be:

The only entity that can define what is and is not allowed in domain names
(not host names) is ICANN and although they have doen it before, it
doesn't happen often.

If you want to continue down this path I suggest you submit a proposal to
the DNSO and that it is not a path we wish to take in this wg.

-rick