Subject: Re: [idn] summary of reordering discussion

From: "James Seng/Personal" <[email-masked]>
Date: Fri, 9 Nov 2001 12:33:53 +0800

> Soobok Lee wrote:
> > Moreover, nameprep hidden complexity around  NFKC is hidden in the
> UTR15
> > document. REORDERINg has no external authority to quote character
> > frequency statistics  in order to make the drafting work look simple
> > as nameprep refers to UTC NFKC.
>
> This is the biggest problem I have with reordering, ie,

Didnt finish the sentence, sorry:

This is the biggest problem I have with reordering, ie, the lack to
reference a creditable table. And yes, there is no table to reference
unfortunately.

-James Seng

---

From: "Soobok Lee" <[email-masked]>
Date: Fri, 9 Nov 2001 13:41:01 +0900

----- Original Message ----- 
From: "James Seng/Personal" <[email-masked]>
> reference a creditable table. And yes, there is no table to reference

I have found some govermental authorites that published
its official script character frequency statistics.

---

From: "James Seng/Personal" <[email-masked]>
Date: Fri, 9 Nov 2001 12:45:47 +0800

Great :-) At least we going somewhere.

Can you tell us which authorities that does that? What policy it have on
the statistic? And how you going to use this statistics to create your
table and how your table is going to evolve as the statistic changes?

Thanks.

-James Seng

----- Original Message -----
From: "Soobok Lee" <[email-masked]>
To: "James Seng/Personal" <[email-masked]>; "Bruce Thomson"
<[email-masked]>; <[email-masked]>
Sent: Friday, November 09, 2001 12:41 PM
Subject: Re: [idn] summary of reordering discussion

---

From: "xiaodong lee" <[email-masked]>
Date: Fri, 9 Nov 2001 19:29:01 +0800

That is great.
We need to do it and find some org to support it. not to deny it simply.
If we use some authoritative data to make some result, it will be 
more useful for people to use.

---

From: "Soobok Lee" <[email-masked]>
Date: Sat, 10 Nov 2001 00:57:25 +0900

Thanks for your interests and comments.

A korean standard body had  made hangul character frequency statistics  to  choose
most frequent 2350 hangul characters (freq > 0.001%)  for KSC5601 legacy hangul code system.
Reordering table v3.0  may contain  the full- 2350 or top-1024 KSC5601 hangul characters.   
It's the  perfect set that korean gov approved with some statistics.  :-)

I know  CN/TW/JP/KR gov standard body/commitees have made or maintains
han char statistics for similar purposes. CN/TW/KR/JP have 4000~13000 frequent han chars of legacy chinese code system. It's very clear  the  # of common TC characters of CN/TW/KR/JP are less than 4800. And For each TC character in the common set, 
we add its SC/Kanji variants  into the common set. Then we get about 6000~7000 han
characters which  CN/TW/KR/JP GOVs approved as frequent ones and have some statistics on .

But even such authoritative statistics cannot be the optimal ones. ALWAYS sub-optimal,
just have authoritative and maintained sources. Some additional mixing/tuning/training within those statistics are needed . Welcome any suggestions.

Regards,

Soobok Lee