Subject: [Ltru] Re: UTF-8

From: "Doug Ewell" <[email-masked]>
Date: Sun, 17 Sep 2006 09:52:53 -0700

Martin Duerst <duerst at it dot aoyama dot ac dot jp> wrote:

>> John's "net-UTF-8" draft isn't ready, and I had some troubles to 
>> understand his last published version.
>
> That isn't very relevant, except that it suggests normalization to 
> NFC, which I assume we would do anyway, and which is already an issue 
> in the case of NCRs.

Both the current Registry and the prototype 4646bis version are in NFC. 
Frankly, it hadn't occurred to me to do it any other way.

This raises an interesting subpoint:  I suppose either 4646bis or 
4645bis should state explicitly that the Registry is in NFC.

>> Another issue are fonts, for RFC 4646 there were proposals to 
>> restrict the registry to Latn.
>
> Bad idea. Any average browser installation has way wider coverage.

I'm not sure whether "ASCII," "Latin-1," or "the Latin script" was 
intended here.  More below.

Frank Ellermann <nobody at xyzzy dot claranet dot de> replied:

> Or way less, almost certainly not "all Latn", the next least common 
> denominator after Latin-1 could be MES-1.

The Registry presented in 4645bis will tentatively contain:

* 497 instances of characters not in ASCII
* 4 instances of characters not in Latin-1
* 1 instance of a character not in MES-1 (U+02BB, the "smart" 
apostrophe/modifier letter added to Ge'ez).  It is in MES-2.

--
Doug Ewell
Fullerton, California, USA
http://users.adelphia.net/~dewell/
RFC 4645  *  UTN #14

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Doug Ewell" <[email-masked]>
Date: Sun, 17 Sep 2006 09:59:38 -0700

Frank Ellermann <nobody at xyzzy dot claranet dot de> wrote:

>> Add an explicit "irregular" production
>
> Interesting idea.  Maybe add "zh-hakka" as "almost irregular", or 
> let's _define_ hakka as deprecated zh variant.

"Irregular" means "not conforming to the ABNF of 4646."  I don't see the 
problem syntactically with "zh-hakka".  It probably would have been made 
into a language-variant pair already, if not for the fact that ISO 639-3 
will allow "zh-hak".

> For zh-cmn I hope 4646bis will solve a "missing cmn" issue as side 
> effect.

There's no problem with "zh-cmn".  One grandfathered tag is the 
Preferred-Value for another.

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 18 Sep 2006 03:24:10 +0200

>> Interesting idea.  Maybe add "zh-hakka" as "almost
>> irregular", or let's _define_ hakka as deprecated zh
>> variant.
 
> "Irregular" means "not conforming to the ABNF of 4646." 
> I don't see the problem syntactically with "zh-hakka".

"Almost irregular" is a registered tag with a reference
to an unregistered subtag, and "zh-hakka" is the only tag
with this strange "feature" (after 4645bis adds cmn).

> There's no problem with "zh-cmn".

At the moment (4646) it's another tag with this "feature".
You can see why it's a real oddity in the gawk script in
<http://purl.net/xyzzy/home/ltru>

All other tags minus yi-latn are straight forward.

Frank

---

From: John Cowan <[email-masked]>
Date: Sun, 17 Sep 2006 21:36:46 -0400

> with this strange "feature" (after 4645bis adds cmn).

How is zh-hakka different from zh-xiang or zh-guoyu (except
that the latter is deprecated)?

-- 
John Cowan        http://ccil.org/~cowan   [email-masked]
Lope de Vega: "It wonders me I can speak at all.  Some caitiff rogue did
rudely yerk me on the knob, wherefrom my wits still wander."
An Englishman: "Ay, a filchman to the nab betimes 'll leave a man
crank for a spell." --Harry Turtledove, Ruled Britannia

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 18 Sep 2006 03:41:37 +0200

Doug Ewell wrote:

> Both the current Registry and the prototype 4646bis version
> are in NFC.  Frankly, it hadn't occurred to me to do it any
> other way.

The other possibility is to say whatever the sources say.  I
didn't note that you modified the source descriptions in some
way.  Please don't do this without a rule for it in 4646bis.

> I suppose either 4646bis or 4645bis should state explicitly
> that the Registry is in NFC.

If that's the case they should most definitely say so.  But I
don't see the point, just copy what the sources say as long as
it's an assigned code-point for a character.

> 1 instance of a character not in MES-1 (U+02BB, the "smart"

Good that we didn't introduce an MES-1 restriction for 4646 :-)

Frank

---

From: Martin Duerst <[email-masked]>
Date: Mon, 18 Sep 2006 12:46:33 +0900

At 01:52 06/09/18, Doug Ewell wrote:
>>> John's "net-UTF-8" draft isn't ready, and I had some troubles to understand his last published version.
>>
>> That isn't very relevant, except that it suggests normalization to NFC, which I assume we would do anyway, and which is already an issue in the case of NCRs.
>
>Both the current Registry and the prototype 4646bis version are in NFC. Frankly, it hadn't occurred to me to do it any other way.
>
>This raises an interesting subpoint:  I suppose either 4646bis or 4645bis should state explicitly that the Registry is in NFC.

I agree. Frank said we should just use what the source uses,
but officially, the soure in most cases still is paper.

Regards,    Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 18 Sep 2006 06:43:22 +0200

> that the latter is deprecated)?

zh-xiang etc. are not referenced anywhere.  zh-hakka has a
reference from i-hak, it appears in a Preferred-Value field.

Similarly zh-cmn appears in the Preferred-Value of zh-guoyu,
and at the moment there's no subtag cmn.  A dangling pointer.

Frank

---

From: "Doug Ewell" <[email-masked]>
Date: Sun, 17 Sep 2006 21:50:48 -0700

Martin Duerst <duerst at it dot aoyama dot ac dot jp> wrote:

> I agree. Frank said we should just use what the source uses, but 
> officially, the soure in most cases still is paper.

As much for the archives as for Martin:

The question of "use what the source uses" is a long-standing, divisive 
one.  The sources for the 4646 Registry were electronic: code lists 
found on the home pages of the ISO MA's and on the UNSD site.  Sometimes 
these appeared inconsistent, with themselves and particularly with each 
other, on things like apostrophe usage.  Sometimes they yielded bizarre 
results, like the acute accent (U+00B4) used in the name Gwich'in.

There are WG members (and ietf-languages contributors) who feel very 
strongly that the Description fields in the Registry must match the 
source standards exactly, to enable automated cross-referencing and 
generally to avoid arbitrariness.  Then there are WG members (and 
ietf-languages contributors) who feel equally strongly that we should be 
able to improve on the names, either typographically or to improve 
clarity and avoid ambiguity.  These two groups are at polar opposites 
and the only way to satisfy both is to admit both Descriptions, even 
when the end result seems preposterous.

---

From: Martin Duerst <[email-masked]>
Date: Tue, 19 Sep 2006 22:04:08 +0900

It seems that a third alternative would be to give the sources
a hint that they might want to do a little cleanup. But I
guess somebody has thought about that, and somebody has tried it.

Regards,    Martin.

At 13:50 06/09/18, Doug Ewell wrote:
>> I agree. Frank said we should just use what the source uses, but officially, the soure in most cases still is paper.
>
>The question of "use what the source uses" is a long-standing, divisive one.  The sources for the 4646 Registry were electronic: code lists found on the home pages of the ISO MA's and on the UNSD site.  Sometimes these appeared inconsistent, with themselves and particularly with each other, on things like apostrophe usage.  Sometimes they yielded bizarre results, like the acute accent (U+00B4) used in the name Gwich'in.
>
>There are WG members (and ietf-languages contributors) who feel very strongly that the Description fields in the Registry must match the source standards exactly, to enable automated cross-referencing and generally to avoid arbitrariness.  Then there are WG members (and ietf-languages contributors) who feel equally strongly that we should be able to improve on the names, either typographically or to improve clarity and avoid ambiguity.  These two groups are at polar opposites and the only way to satisfy both is to admit both Descriptions, even when the end result seems preposterous.
>
>--
>Doug Ewell