Subject: Re: [idn] nameprep (Korean Nameprep)

From: "Soobok Lee" <[email-masked]>
Date: Fri, 1 Jun 2001 13:39:04 +0900

We (KRNIC NC) found a KC normalization implementation
in nameprep-03 does not preserve
the hangul compatibility jamo character sequences,
but combines them into hangul syllables with errornously leaving
jongseong jamo behind (CVL -> (CV)(L) [false],  (CVL) is right).

I think we may consider taking away this hangul combining actions
from nameprep-implementation of KC normalization.

and more work should be done on whether to prohibit
the 4 hangul filler characters,
and on case-folding of hangul jamo characters in 4 forms
(choseong,jongseong,compatibility jamo,narrow-form).

we are studying the ambiguity  that may arise between hangul and
many other language character sets. Japanese katakana 'ro' and
hangul jamo 'mieum' look the same!
If multilingual names are introduced in [email-masked], http://ML.com,
this ambiguity issues become the internet security issues, i am afraid.

We'll continue to follow up IDN wg's nameprep efforts and submit a
draft if necessary.

Soobok Lee
[email-masked]

----- Original Message -----
From: "Paul Hoffman / IMC" <[email-masked]>
To: "Adam M. Costello" <[email-masked]>; <[email-masked]>
Sent: Friday, June 01, 2001 12:38 PM
Subject: Re: [idn] nameprep

> At 3:07 AM +0000 6/1/01, Adam M. Costello wrote:
> >Paul Hoffman / IMC <[email-masked]> wrote:
> >
> >>  need to finish nameprep, which at this point is finished other than it
> >>  does not address the new characters added in Unicode 3.1.
> >
> >Nameprep currently includes the following steps:
> >
> >     KC-normalize
> >     map
> >     KC-normalize
> >     prohibit
>
> Er, no it doesn't. It only includes map->normalize->prohibit.
>
> --Paul Hoffman, Director
> --Internet Mail Consortium
>

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 31 May 2001 23:11:17 -0700

1. It is correct that neither the compatibility Jamo nor the half-width Jamo
are combined into Hangul syllables by the normalization algorithm. However,
those Jamo are not intented for use in constructing Hangul syllables --
instead, the conjoining Jamo are. And those are correctly composed into
syllables, if individual Jamo are input.

It would be possible to deal separately with those characters in nameprep --
although without more information it is not possible to completely eliminate
ambiguity in choosing whether a consonant is leading or trailing. Do you
believe this is, in practice, a really problem; that there are a large
number of people that produce Korean text with compatibility or half-width
Jamo, instead of Hangul syllables?

2. I don't have a strong position (nor do I think the Unicode Consortium
would) on the issue of whether to disallow the filler characters. They are
not required for correct representation of Korean, and could be either
prohibited or filtered out.

3. As to the issue of visual ambiguity, that is already present across other
scripts. Greek O, Latin O, and Cyrillic O are identical. The problem is that
the amount of allowable variation between acceptable glyphs for given
characters in different scripts (or symbols and punctuation!) is not easy to
quantify, nor is there any clear-cut data upon which to draw in this area.
This topic has arisen before, and I believe the consensus is to leave
matters as they are.

Mark

----- Original Message -----
From: "Soobok Lee" <[email-masked]>
To: "Adam M. Costello" <[email-masked]>; <[email-masked]>; "Paul
Hoffman / IMC" <[email-masked]>
Sent: Thursday, May 31, 2001 21:39
Subject: Re: [idn] nameprep (Korean Nameprep)

---

From: "Adam M. Costello" <[email-masked]>
Date: Fri, 1 Jun 2001 21:21:06 +0000

> Er, no it doesn't. It only includes map->normalize->prohibit.

Oops, I was misremembering the part about the mapping step being
designed to capture the operation fold->normalize->fold->normalize.

Anyway, if nameprep is map->normalize->prohibit, that's equivalent to:

    map
    compatible-decompose
    canonical-compose
    prohibit

We can still ask whether it would be convenient to swap the first two
steps.  It would make the mapping table smaller.

Also, I continue to be nervous about inviting all the weird characters
into domain names right away.  Maybe applications should be liberal
in what the allow in domain names, but DNS servers should be more
conservative, at least in the beginning.  People might want to look at
ISO/IEC TR 10176:1998, or equivalently, Annex D of the C99 spec (ISO/IEC
9899:1999), which lists the set of characters allowed in C identifiers.
Another set to look at is the name characters of XML, which are defined
in terms of the Unicode character classes:

http://www.w3.org/TR/2000/REC-xml-20001006#sec-common-syn                       
http://www.w3.org/TR/2000/REC-xml-20001006#CharClasses                          

AMC

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Fri, 1 Jun 2001 15:01:59 -0700

>steps.  It would make the mapping table smaller.

What is your motivation here? The mapping table is quite small: it 
fits in about 4K of memory. Can you say how many entries would be 
saved, and why saving probably 1K of a table would be worth it? Given 
that many toolkits have a single step for normalization, having 
people split out the two steps for saving 1K of table space seems 
like overkill.

>into domain names right away.

And others are nervous about us subsetting what should be allowed 
initially and allowing more in later. Transitions are hard to manage. 
(And calling someone's characters "weird" is also difficult for some 
of us.)

>People might want to look at

And what do C identifiers have to do with it? We're talking about 
people's names, company names, product naems, and so on.

>in terms of the Unicode character classes:

Same problem.

--Paul Hoffman, Director

---

From: "Adam M. Costello" <[email-masked]>
Date: Fri, 1 Jun 2001 23:16:54 +0000

Paul Hoffman / IMC <[email-masked]> wrote:

> > We can still ask whether it would be convenient to swap the first
> > two steps.  It would make the mapping table smaller.
>
> saved, and why saving probably 1K of a table would be worth it?

Every now and then people find it necessary to go through lists of
Unicode characters manually and think about each one, like whether
some policy or algorithm under consideration will correctly handle
it.  Similarly, I expect that people will occasionally go through the
nameprep mapping rules and think about how each rule will interact with
other things (like keyboard input methods or other mappings to/from
other charsets).

> initially and allowing more in later. Transitions are hard to manage. 

That's a good point.

> (And calling someone's characters "weird" is also difficult for some

I had in mind things like:

Letterlike Symbols
Number Forms
Arrows
Mathematical Operators
Miscellaneous Technical
Control Pictures
Optical Character Recognition
Enclosed Alphanumerics
Box Drawing
Block Elements
Geometric Shapes
Miscellaneous Symbols
Dingbats

AMC

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Fri, 1 Jun 2001 17:15:29 -0700

>Optical Character Recognition

Those are all easy ones. They pretty much just display, and have few 
if any funny properties. Thus, taking them out and being sure that 
everyone took them out is actually much harder than leaving them in. 
You are right, these will probably not appear in names, but the act 
of taking them out makes nameprep less stable, not more stable.

To date, the WG has tended towards "few rules and fewer special cases 
are better". If that changes, adding "characters that probably won't 
appear in names" could be considered.

--Paul Hoffman, Director