Subject: Re: [idn] call for comments for REORDERING

From: [email-masked]
Date: Fri, 19 Oct 2001 02:05:53 EDT

In a message dated 2001-10-18 21:33:55 Pacific Daylight Time, 
[email-masked] writes:

>  1) saturations in TLD namespaces would require longer names for which
>      REORDERING is designed to give greater benefits/compression ratio.

Is it not the case that logographic/ideographic writing systems such as Han 
and the syllable-oriented Unicode encoding of Hangul, with their large 
numbers of characters, convey more information per character than alphabetic 
scripts?  How long, conceptually, does a domain name really need to be?

I would probably have a warmer spot in my heart for these script-specific 
compression proposals if I were convinced that the users of Han and Hangul 
were somehow being cheated by not being able to register the equivalent of 
"FourthDistrictCentralProvincialLibraryBoardOfDirectors.org" as a domain name.

>      2.0) the character frequency table are constructed from
>           Verisign GRS' ML.com testbeds.
>           Even for chinese han script, their
>            registrations came from China/TAIWAN/JAPAN/KOREA and other
>             non-asian squatters.

I know that Soobok is not specifically listing this as a goal, but I thought 
efforts to accommodate the 15-character-and-longer Han domain names that have 
already been registered was explicitly a non-goal, since those businesses and 
individuals were supposed to have waited until an IDN solution was in place.

-Doug Ewell
 Fullerton, California

---

From: "Soobok Lee" <[email-masked]>
Date: Fri, 19 Oct 2001 15:31:50 +0900

----- Original Message ----- 
From: <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>; <[email-masked]>
Sent: Friday, October 19, 2001 3:05 PM
Subject: Re: [idn] call for comments for REORDERING

> "FourthDistrictCentralProvincialLibraryBoardOfDirectors.org" as a domain name.

First) REORDERING is  useful for very common  han/hangeul labels of length 4~8.

Second) If we take into consideration the additional length of subdomains of a
IDN, the total length of  <ML-subdomain>.<ML>.com  often  exceeds
14. ANd if we think about mailbox name in [email-masked], it will exceed 20.
IN that case, the Sum of the lengths of  saved characters in each substring in [email-masked] will be great, around 15~20, i guess.

http://www.cnnic.net/daily/2001-8/9.shtml
will answer some of your questions.
We should not attempt to draw an upper limit on future  applications of 
han/hangeul IDN domains by imposing unfair disadvantage on max length 
of Han/hangeul labels .

Soobok Lee

---

From: Martin Duerst <[email-masked]>
Date: Fri, 19 Oct 2001 16:05:35 +0900

At 02:05 01/10/19 -0400, [email-masked] wrote:
>scripts?

Very much so, of course.

>How long, conceptually, does a domain name really need to be?

That's a very subjective question, because 'real need' is very
subjective.

>"FourthDistrictCentralProvincialLibraryBoardOfDirectors.org" as a domain name.

In various Japanese translations, I get between 14 and 18 characters.
But I think the fact that such domain names can be (and is some strange
cases even are) registered in English is rather by accident (the designers
of the internal DNS format decided to reserve 2 out of 8 bits, which leaves
6 for the label length, thus a max length of 64. If it were 5 bits, it
would be 31. That wouldn't have affected the spread of the Internet and
the Web in any way.

Regards,   Martin.

---

From: "Soobok Lee" <[email-masked]>
Date: Fri, 19 Oct 2001 17:54:35 +0900

----- Original Message ----- 
From: "Martin Duerst" <[email-masked]>
To: "Soobok Lee" <[email-masked]>; <[email-masked]>; <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, October 19, 2001 5:12 PM
Subject: Re: [idn] call for comments for REORDERING

> At 15:31 01/10/19 +0900, Soobok Lee wrote:
> 
> >IN that case, the Sum of the lengths of  saved characters in each 
> >substring in [email-masked] will be great, around 15~20, i guess.
> 
> It is a very long standing lesson when working with computers
> that in case there is a tradeoff between a more stable and
> a more compact solution, it's in most cases, and definitely
> in the long term, much better to take the more stable and
> simpler one. The Y2K problem is the best example, but there
> are many other ones.
> 

Your menttion of Stability are presume the variations of frequency
distribtion of characters in a script block. But, as i stated before,
the influence is marginal and REORDERING is not optiomal solution,
but sub-optimal solution that have clear benefits for a long time.

Please read  2500-years-old chinese classic books, then you can 
find there has been no great change in frequent han characters.
Really!!!

You should suggest the proof that the frequency
change will be great in the future and hurt the statbility of
REOERDERING tables.

> I don't see any unfair disadvantage. Both in Latin and in Han/Hangul,
> it's already possible to make names longer than really useful.

You missed what the above examples really shows.
Please think again the sum of length of saved characters in
[email-masked].

> To say that overly long names have been cut off at a different
> point for Latin and for Han or Hangul may be true, but it is
> rather irrelevant. And even with the most advanced compression,
> we will never be able to make sure that we are completely fair
> to everybody.
> 
> 
> Regards,   Martin.
>

---

From: Martin Duerst <[email-masked]>
Date: Fri, 19 Oct 2001 17:12:05 +0900

Regards,   Martin.

---

From: "Soobok Lee" <[email-masked]>
Date: Fri, 19 Oct 2001 21:27:57 +0900

----- Original Message ----- 
From: "Martin Duerst" <[email-masked]>
To: <[email-masked]>; <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Friday, October 19, 2001 4:05 PM
Subject: Re: [idn] call for comments for REORDERING

> >numbers of characters, convey more information per character than alphabetic

han/hangeul have huge lists of characters (20992/11172). 
To encode them, we need more digits and bits than latin letters. 
That ACE encoding overhead is so big that it cannot be compensated by the superiority in the information capacity of a han letter compared to a latin letter. The sizes of Latin alphabets or variants do not exceed 30.

In UCS-2, 2*N octets are need for latin/han lables of length N.
But ACE/UTF8 which favor latin script, require roughly 3.0*N octets for long han labels. REORDERING reduce those requirement into 2.2*N, close to that of UCS-2.

Soobok Lee

---

From: Martin Duerst <[email-masked]>
Date: Mon, 22 Oct 2001 12:23:26 +0900

>To encode them, we need more digits and bits than latin letters.

It may be interesting to give some background information on
Hangul encoding in Unicode. In Unicode 1.0 and 1.1, only a
subset of the 11172 modern hangul syllables, 2000-3000
(taken from KS5601) were encoded, in the place where now
CJK Extension A is located. These represented the subset
of Hangul in actual practical use. However, at the request
of the Korean (ROK) national delegation to ISO/IEC JTC1
SC2/WG2, and after a long and painful discussions, these
codepoints were moved to the current location, and completed
in a regular fashion to the current 11172. A lot of reasons
were given at the time for having to make this change.
It was the only time at all that codepoints were moved,
and it resulted in very strong criticism (very much including
people from the IETF). The main benefit was that it made
the bodies involved understand how important stability is
to the users.

Anyway, if this change hadn't been done, all the frequent
Hangul would now be quite a bit closer together.

Regards,   Martin.

>That ACE encoding overhead is so big that it cannot be compensated by the 
>superiority in the information capacity of a han letter compared to a 
>latin letter. The sizes of Latin alphabets or variants do not exceed 30.
>
>But ACE/UTF8 which favor latin script, require roughly 3.0*N octets for 
>long han labels. REORDERING reduce those requirement into 2.2*N, close to 
>that of UCS-2.
>
>Soobok Lee

---

From: Martin Duerst <[email-masked]>
Date: Mon, 22 Oct 2001 12:28:35 +0900

>find there has been no great change in frequent han characters.

In some sense, there hasn't been much change. But there have
been very clear changes. The vocabulary has changed, and therefore
the more frequent and less frequent characters have changed.
Also, the introduction of simplified characters around 1950
has changed frequencies quite a bit; many of the simplified
characters are quite frequent but don't turn up in old texts.

Regards,  Martin.

---

From: "Soobok Lee" <[email-masked]>
Date: Mon, 22 Oct 2001 20:24:48 +0900

----- Original Message ----- 
From: "Martin Duerst" <[email-masked]>
To: "Soobok Lee" <[email-masked]>; <[email-masked]>; <[email-masked]>
Cc: <[email-masked]>
Sent: Monday, October 22, 2001 12:23 PM
Subject: Re: [idn] call for comments for REORDERING

> the bodies involved understand how important stability is

Thanks for detailed history of Hangul in unicode.
But including 11172 hangul made Unicode more useful and 
widely acceptable to korean community, because existing 
KS5601's 2350 or more hangul has been criticized for not being
able to serve many scholarly purpose or book publishings.

The pain for inclufing 11172 hangul  was REWARDED!

I know current NFC/NFKC has *severely flawed* hangul jamo handling behaviors
that was put into UTR15 because of lack of careful reviews by both
Koreans represenatives and UTC members.
This pain was not from including 11172, but from careless reviewers.

I hope UTC cures this BUGGY NFC/NFKC jamo handling as an exceptional case
while it is in  strict adherence to UTR's backward compatiblity policy & promise.

Soobok Lee

---

From: Martin Duerst <[email-masked]>
Date: Tue, 23 Oct 2001 14:19:48 +0900

>The pain for inclufing 11172 hangul  was REWARDED!

It was quite a pain. And it is the reason you are now
asking for reordering, which would again be a very
serious pain, with not enough rewards.

If you could chose among:

a) Having the Unicode 1.1 layout, with the rest of the
    modern syllables either encoded with Jamos or in
    a separate area, thus favoring compression with
    ACE-Z over regularity.

b) Having the current block, without reordering, and
    therefore with slight inefficiency for ACE-Z.

Which one would you chose? Which one would other
people in Korea prefer, according to your opinion?

At some point, we cannot have the cake and eat it too.

Regards,   Martin.