Subject: Re: [idn] Report from the ACE design team

From: Kenneth Whistler <[email-masked]>
Date: Tue, 26 Jun 2001 20:18:41 -0700 (PDT)

Soobok Lee wrote:

> I believe long natural sentence/phrase domains escape your arguments.
> 
> In CJK, "What is the nearest macdonald hamberger shop from here?".(kr|cn|jp|tw)

??

Why would someone want to type in:

"konomawariniwaichibanchikaimakudonarudohambaaganomisewadokodeshooka.co.jp"

when they could go to

mcdonalds.co.jp

click on the "zenkoku no mise listo" (List of all stores in the country)
button on the home page and get the same results?

If people wanted to do creative PR marketing, they could create

my-makudonarudo.co.jp  (with "makudonarudo" in Katakana, if you like)

and have the custom site *remember* where you live and which restaurant
you like to go to nearby.

Maybe movie names or band names, and stuff like that -- sure. Marketers
love that stuff.

> domain
> may  raise interests and easier to remember and that was not possible
> with LDH domains  for CJK peoples.

Yes, but use of excessively long natural phrase domains like this is just
misuse of what domain names should be for, in my opinion, and will be
quickly outcompeted by more effective ways to get people to visit your
sites and get the information or whatever else they are looking for.

--Ken

> 
> .kr .cn .jp domain suffices may be omitted in future localized web browsers, and
> it will make japanese and chinese sentence domains good for PR marketing.
> 
> I think it is safe for us to leave much rooms for these creative naming
> conventions
> in our ACE proposals for the future.

---

From: "Soobok Lee" <[email-masked]>
Date: Wed, 27 Jun 2001 13:05:35 +0900

----- Original Message -----
From: "Kenneth Whistler" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: Wednesday, June 27, 2001 12:18 PM
Subject: Re: [idn] Report from the ACE design team

> "konomawariniwaichibanchikaimakudonarudohambaaganomisewadokodeshooka.co.jp"
    1                                10                              20
30
This japanese example sentence exceeds 33 ganas!   :-))
In CJK, sentences/phrases with 15~20 syllables are regarded as very SHORT ones.

> Maybe movie names or band names, and stuff like that -- sure. Marketers

It's premature to be pessimistic  about the usefulness and future of
sentence/phrase domains that are not actively experimented enough yet.

> sites and get the information or whatever else they are looking for.

IMHO, IDN will partly replace Directory/Search/Keyword services (still useful).
And that's one of the purposes of introducing IDN.

Soobok Lee

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Wed, 27 Jun 2001 09:42:42 -0700

At 10:20 AM -0400 6/27/01, Marc Blanchet wrote:
>At/À 20:18 2001-06-26 -0700, Kenneth Whistler you wrote/vous écriviez:
>>Soobok Lee wrote:
>>
>Sorry Ken, but your argument of long-names-will-not-be-used does not 
>stand. In the .com zone file, there is _already_ some labels which 
>are 63 characters long.

There is a difference between novelty names and names that will 
really be used. Your point about 63-character names is an excellent 
example of this. All names that are 63 characters (and probably all 
names longer than 30 characters) are probably for novelty use. If 127 
characters were allowed, someone would surely register them as well. 
Why don't we change the DNS to handle these as well?

So far, the only argument that has been brought up for Han names >15 
characters is that some companies would want to have a registration 
for their official names, but they would probably use a shorter, 
easier-to-use name in daily use. The proposals to use more difficult 
ACEs to accomodate companies (and probably some people) to put their 
whole name in a single name part ignore two important points:

- Regardless of the ACE we chose, some companies will not be able to 
encode their whole name in a single name part. We have to draw the 
line somewhere, and someone will always be aggrieved.

- All of the companies in question could easily encode their name in 
two adjacent name parts. Thus, no one is prevented from registering 
their full name.

As for the people who say "there are many sentences that won't fit 
into the ACE encoding", that is certainly true. It is just as true 
for the current 63-octet limit in ASCII. Sentences are not names, and 
we should ignore them when trying to come up with a good encoding 
scheme for names.

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: Marc Blanchet <[email-masked]>
Date: Wed, 27 Jun 2001 10:20:16 -0400

>"konomawariniwaichibanchikaimakudonarudohambaaganomisewadokodeshooka.co.jp"

Sorry Ken, but your argument of long-names-will-not-be-used does not stand. 
In the .com zone file, there is _already_ some labels which are 63 
characters long.  So even in english, there is a use (that one can 
question, but...) of all space available.  The idn tax that give us ~ 20 
chars max for some scripts is already a big tax compared to what is 
available and _used_ in all ascii labels.  To me, the best compression we 
can get is the best, (without necessarily forgetting the complexity).

Marc.

---

From: "Soobok Lee" <[email-masked]>
Date: Thu, 28 Jun 2001 06:22:11 +0900

In IDNA, all domain labels are NAMEPREPed and then ACEed.

If any one would ever look into the  NAMEPREP implementations,
he/she could not understand why we adhere to  simple ACEs.

NAMEPREP already has HUGE unicode mapping tables and 
long lists of prohibited characters. It has  KC normalization 
code in it. Thousands lines of code for all.

Moreover, NAMEPREP and ACE may be coded into 
a single IDNA library by application vendors.
Choosing the simplest ACE (and saving  only one hundred 
lines of code)  does not greatly simplify  IDNA implementations and 
so lose its justification 
as long as we have still complex NAMEPREP process paired with ACE.

Soobok Lee, [email-masked]

----- Original Message ----- 
From: "Marc Blanchet" <[email-masked]>
To: "Kenneth Whistler" <[email-masked]>
Cc: <[email-masked]>
Sent: Wednesday, June 27, 2001 11:20 PM
Subject: Re: [idn] Report from the ACE design team

> At/?20:18 2001-06-26 -0700, Kenneth Whistler you wrote/vous ?riviez:
> >Soobok Lee wrote:
> >

---

From: "Brian W. Spolarich" <[email-masked]>
Date: Wed, 27 Jun 2001 17:33:18 -0400

| Yes, but use of excessively long natural phrase domains like this is just
| misuse of what domain names should be for, in my opinion, and will be
| quickly outcompeted by more effective ways to get people to visit your
| sites and get the information or whatever else they are looking for.

  The issue here is not to acheive maximum compression, but to achieve
enough compression to accomodate reasonable use.  Ken's arguments are in
line with my thinking, although he does it better than I could.

  As a westerner with only modest experience with CJK language issues and
speakers I hesitate to define "reasonable" here.  However the examples that
some have given do not seem reasonable to me, and sound more like edge
cases.  The DNS is not intended to be a registration repository for company
names, for example.  Some of these new requirements may better be placed
elsewhere than IDN.

  My gut feeling is that there is a diminishing return relationship between
compression and complexity, and that DUDE-02 comes fairly close, but still
to the left, of the optimum point.  I would support changes to DUDE that
could provide some incremental efficiency but still retains the simplicity
of design and approach.

  I haven't had an opportunity to extensively review the
reorder-before-encode approaches that have been submitted.  Conceptually
this seems like a good idea, but this is something of a slippery slope and
would need to be carefully evaluated.

  -bws

---

From: "Mark Davis" <[email-masked]>
Date: Wed, 27 Jun 2001 15:21:34 -0700

Normalization does take some data files, but it does not take thousands of
lines of code. See
http://www.macchiato.com/unicode/normalization_footprint.htm for more
information.

Mark

----- Original Message -----
From: "Soobok Lee" <[email-masked]>
To: "Kenneth Whistler" <[email-masked]>; "Marc Blanchet"
<[email-masked]>
Cc: <[email-masked]>
Sent: Wednesday, June 27, 2001 14:22
Subject: Re: [idn] Report from the ACE design team

> > Sorry Ken, but your argument of long-names-will-not-be-used does not
stand.

---

From: "Soobok Lee" <[email-masked]>
Date: Thu, 28 Jun 2001 21:44:41 +0900

For whom had never looked into NAMEPREP codes in MDNkit of JPNIC,

[root@bora lib]# pwd
/home/lsb/mdn/mdnkit-2.0.1-src/lib

[root@bora lib]# wc name*[hc] uni*[hc]
    296    1109    8554 nameprep.c
    136     804    5475 nameprep_template.c
   1694   11778   73804 nameprepdata.c
    484    1822   12314 unicode.c
   6806   38573  327222 unicodedata.c
   9416   54086  427369 total

[root@bora lib]# ls -l *ace*c dud*c
-rw-r--r--   1 3364     wheel        7968 Mar  7 09:58 ace.c
-rw-r--r--   1 3364     wheel       19370 Apr  4 15:11 amcacem.c
-rw-r--r--   1 3364     wheel       11963 Mar 30 11:35 amcacer.c
-rw-r--r--   1 3364     wheel       15803 Mar  7 18:34 brace.c
-rw-r--r--   1 3364     wheel        7892 Mar 26 10:45 dude.c
-rw-r--r--   1 3364     wheel       11671 Mar 28 15:37 lace.c
-rw-r--r--   1 3364     wheel       11826 Mar  7 18:34 race.c

[root@bora lib]# wc  *ace*c dud*c
    291    1186    7968 ace.c
    829    3024   19370 amcacem.c
    496    1751   11963 amcacer.c
    666    2331   15803 brace.c
    448    1837   11671 lace.c
    462    1842   11826 race.c
    313    1224    7892 dude.c
   3505   13195   86493 total

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 28 Jun 2001 08:46:08 -0700

I agree that NamePrep is more complex than the ACEs. However, it is not that
complicated, and there is a very extensive compliance test for NFKC on the
Unicode site.

Mark
----- Original Message -----
From: "Makoto Ishisone" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, June 28, 2001 08:01
Subject: Re: [idn] complexity/simplicity: NAMEPREP code vs ACE codes

> In message <[email-masked]>,
> "Soobok Lee" <[email-masked]> wrote:
> >  ...
> If you look closer, you'll find that nameprepdata.c and unicodedata.c
> above contain only data -- some large tables, which are generated from
> NAMEPREP draft and Unicode Character Database.  So I don't think it is
> fair to count them when you compare complexity.  On the other hand
> you overlooked unormalize.c, which implements Unicode Normalization
> Forms.
>
> Anyway I agree that NAMEPREP (NFKC in particular) is no simpler than
> most of the proposed ACEs.  Before implementing NFKC you have to read
> the specification, which is longer than any ACE I-Ds, and relevant
> documents, understand what's going on, and generate tables from the
> data...  Also I think it is harder to test the correctness of the
> implementation.
>
> -- [email-masked]
>
>

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 28 Jun 2001 08:44:23 -0700

The algorithm for KC normalization is a couple of pages of code. It should
not be particularly difficult.

Mark

----- Original Message -----
From: "Soobok Lee" <[email-masked]>
To: "Makoto Ishisone" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, June 28, 2001 08:25
Subject: Re: [idn] complexity/simplicity: NAMEPREP code vs ACE codes

> You are right.
> KC Norm is hard to learn and implement from scratch.
>
> To find more KC norm related sources in MDNkit.
> [root@bora lib]# wc *norm*c
>     632    1975   16581 normalizer.c
>     459    1710   12201 unormalize.c ( not related to KC norm ???)
>    1091    3685   28782 total
>
> If all of you think even huge mapping tables do not add complexity,
> My 'reorder_by_char_frequency-before-encode' idea
> adds no complexity to DUDE, as it adds only simple
> mapping functions and tables.  that's not bad.    :-)
>
> http://www.postel.co.kr/idn-lsb-00.txt
> (I am now adding SC/TW support to this. 15%~20% improvement measured
> by adding tables for most frequent 2048 han syllables).
>
> Soobok Lee
>
>
>
>
> Sent: Friday, June 29, 2001 12:01 AM

---

From: Makoto Ishisone <[email-masked]>
Date: Fri, 29 Jun 2001 00:01:54 +0900

-- [email-masked]

---

From: "Soobok Lee" <[email-masked]>
Date: Fri, 29 Jun 2001 00:25:25 +0900

You are right.  

Soobok Lee

---

From: Makoto Ishisone <[email-masked]>
Date: Fri, 29 Jun 2001 18:15:12 +0900

In message <009001c0ffe9$72f278d0$0c680b41@c1340594a>,
> complicated, and there is a very extensive compliance test for NFKC on the

Yes.  I used your extensive test data to check my implementation
(thanks!).

I think this is an important point.  Although NAMEPREP (NFKC) is a
rather complex operation, you can make sure that your implementation
is correct because there is a good set of test data.

This also applies to ACE.  While simplicity is very important in order
to guarantee the interoperability among various implementations, some
degree of complexity could well be compensated by a good test data.
And for most of the proposed ACEs, the amount of the required test
data is much smaller than what is needed for NFKC.  DUDE and MACE
drafts already have such test data, and I believe authors of other
ACEs can also supply them.

						-- [email-masked]