Subject: [Ltru] ISO 639-3 description changes (was: Re: 3066ter encompassed languages)

From: "Doug Ewell" <[email-masked]>
Date: Sun, 10 Sep 2006 18:47:31 -0700

We have been talking about the descriptions ("reference names") used in 
the current draft tables for ISO 639-3, and how they differ sometimes 
from those used in ISO 639-1 and 639-2 and thus in the Language Subtag 
Registry.

In this post, I will classify the differences into two groups, and argue 
that names in the first group should be added to the Registry, while 
names in the second group should replace the existing descriptions.

Disclaimer:  I know that the currently available 639-3 tables are draft, 
and are subject to change, and that the JAC plans to discuss cases such 
as Malay and possibly assign different names before the standard is 
published.  However, our first draft documents for LTRU 2.0 are due in 3 
weeks and I feel we have to discuss this issue and develop some rules 
now, even if the exact languages involved may change in the future.

Group A:
These are cases where ISO 639-3 assigns a (usually slightly) different 
name from 639-2.  Often it is simply an inverted form of what is already 
there.  In only one case -- "Gaelic (Scots)" -- does the new name 
include a parenthetical comment, and this appears to be explanatory in 
nature rather than necessary to make a distinction.  For these cases, I 
suggest *adding* the 639-3 name as an additional Description field.  I 
don't expect any of these to be controversial.

cu: Church Slavic; Old Slavonic; Church Slavonic; Old Bulgarian; Old 
Church Slavonic
639-3 adds: Slavic, Church

frr: Northern Frisian
639-3 adds:  Frisian, Northern

fy: Western Frisian
639-3 adds: Frisian, Western

gd: Gaelic; Scottish Gaelic
639-3 adds: Gaelic (Scots)

ii: Sichuan Yi
639-3 adds: Yi, Sichuan

mh: Marshallese
639-3 adds: Marshall

alt: Southern Altai
639-3 adds: Altai, Southern

dsb: Lower Sorbian
639-3 adds: Sorbian, Lower

frs: Eastern Frisian
639-3 adds: Frisian, Eastern

gsw: Swiss German; Alemanic
639-3 adds: Schwyzerdütsch
(which of course would go into the Registry as "Schwyzerd&#xFC;tsch")

hsb: Upper Sorbian
639-3 adds: Sorbian, Upper

nso: Northern Sotho; Pedi; Sepedi
639-3 adds: Sotho, Northern

nwc: Classical Newari; Old Newari; Classical Nepal Bhasa
639-3 adds: Newari, Classical

rup: Aromanian; Arumanian; Macedo-Romanian
639-3 adds: Romanian, Macedo

sam: Samaritan Aramaic
639-3 adds: Aramaic, Samaritan

srn: Sranan Tongo
639-3 adds: Sranan

Group B:
These are cases where the ISO 639-3 name is the same as in 639-2, but 
with a parenthetical note added.  This note is always either:

(a) the word "generic", used to distinguish a macrolanguage from a 
specific language indicated by the word "specific", OR

(b) the name of a country, used to distinguish the language from a 
completely different language with the same name, associated with a 
different country.

For these cases, there will be a pair of language subtags with the same 
base name -- for example, "Kamba (Kenya)" and "Kamba (Brazil)".  I 
contend that the presence of both such languages means it would be 
misleading and inappropriate to list simply "Kamba" as the Description 
for one of them, as we currently do for subtag "kam".  Therefore, I 
believe we should *replace* the existing (ISO 639-2-based) Description 
with the reference name from ISO 639-3.

Note that I do *not* consider this tantamount to "editorializing" or 
trying to "improve upon the raw data" in the sense of playing with 
apostrophes.  I believe this is necessary to avoid confusion and 
mistagging.

ms: Malay
639-3 changes this to: Malay (generic)

sw: Swahili
639-3 changes this to: Swahili (generic)

ain: Ainu
639-3 changes this to: Ainu (Japan)

bas: Basa
639-3 changes this to: Basa (Cameroon)

bem: Bemba
639-3 changes this to: Bemba (Zambia)

chm: Mari
639-3 changes this to: Mari (Russia)

doi: Dogri
639-3 changes this to: Dogri (generic)

fan: Fang
639-3 changes this to: Fang (Equatorial Guinea)

gba: Gbaya
639-3 changes this to: Gbaya (Central African Republic)

kam: Kamba
639-3 changes this to: Kamba (Kenya)

kok: Konkani
639-3 changes this to: Konkani (generic)

men: Mende
639-3 changes this to: Mende (Sierra Leone)

war: Waray
639-3 changes this to: Waray (Philippines)

--
Doug Ewell
Fullerton, California, USA
http://users.adelphia.net/~dewell/
RFC 4645  •  UTN #14

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Doug Ewell" <[email-masked]>
Date: Sun, 10 Sep 2006 19:00:18 -0700

Frank Ellermann <nobody at xyzzy dot claranet dot de> wrote:

>> What if the descriptions are noticeably different between 639-2 and 
>> 639-3?  Where do we draw the line without being arbitrary?
>
> The line is "whatever 639-2 says" for a tag derived from 639-2.

I claim it is more complicated than that, when 639-2 has coded a 
"language" called Malay that 639-3 actually treats as several languages, 
one of which is called Malay and the others of which are called 
something different.

>> Which is better for avoiding confusion with "Waray (Australia)"?
>
> 639-3 <g>.  Propose a rule how to smuggle Waray (Philippines) into the 
> 639-2 entry.  "Comment" is my first idea.  Or "let's see how they sort 
> it out" - Peter said that will be done in time.

I don't like using comments for information that is crucial to choosing 
the right subtag.  Suppose you're looking for the Ainu language spoken 
in China, which is related to Uyghur and has many more speakers than the 
Japanese language of the same name which exists in 639-2.  Should you 
have to look in the comments to find which Ainu is which?

>> I'm in favor of adding the parenthesized country names introduced in 
>> 639-3, when they serve to distinguish two languages with otherwise 
>> identical names.
>
> If it has an obvious pattern we can create a rule for it.

See my previous post.  When 639-3 changes a 639-2 name by adding a 
country name or the word "generic", replace the old Description with the 
new one.  Otherwise, add the new Description to the existing ones. 
That's a rule based on an obvious pattern.

---

From: John Cowan <[email-masked]>
Date: Sun, 10 Sep 2006 23:27:41 -0400

> names in the second group should replace the existing descriptions.

Now that I've seen the data, I agree entirely with Doug's proposals.
Additional name, add it.  Disambiguation, use it instead.  Anything
else will just induce confusion.

-- 
While staying with the Asonu, I met a man from      John Cowan
the Candensian plane, which is very much like       [email-masked]
ours, only more of it consists of Toronto.          http://:www.ccil.org/~cowan
        --Ursula K. Le Guin, Changing Planes

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 11 Sep 2006 16:04:50 +0200

Doug Ewell wrote:

> names in the first group should be added to the Registry

Okay, my KISS attack "ignore 639-3 if a tag from another
source is already there" turned out to be a bad idea.

> Disclaimer

[x] read

> I contend that the presence of both such languages means it
> would be misleading and inappropriate to list simply "Kamba"
> as the Description for one of them, as we currently do for
> subtag "kam".  Therefore, I believe we should *replace* the
> existing (ISO 639-2-based) Description with the reference
> name from ISO 639-3.

Here I disagree, let 639-2 replace it if necessary.  If you
insist on it we can add a rule in 4646bis allowing the review
list to remove mainly redundant and potentially misleading
descriptions based on 639-2 individually at any time (in the
4645bis "date C" registry or at registration time).

> I do *not* consider this tantamount to "editorializing" or
> trying to "improve upon the raw data"

It's very near to cherry picking, and users looking for the
precise description as published in 639-2 might be confused.
Debbie mentioned use cases where what you propose could
remove identifiers (URNs or similar) somewhere.

Weak analogoy:  I could propose to replace <control> by NAK
in column 2 of u+0015.  Highly desirable from my POV, but
the Unicode folks would eat me alive, it breaks applications
expecting <control> in column 2.

Frank

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 11 Sep 2006 16:35:43 +0200

Doug Ewell wrote:
 
> I don't like using comments for information that is crucial
> to choosing the right subtag.

They're as good or bad as the "description" for this purpose.
The source is different, "comment" is added by the review list,
no "comment" if the ISO "description" is good enough.

> Should you have to look in the comments to find which Ainu
> is which?

As soon as I find out that there are more Ainu than the one
I've heard of before (which happens to be the JP Ainu) I better
check what's going on.  We've to add instructions for human
readers trying to figure out a subtag for a given type and
description anyway - they better use a list sorted by type
and description (or only description).  A paragraph with one
simple example, nothing extraordinary.

 [proposed rule]
> When 639-3 changes a 639-2 name by adding a country name or
> the word "generic", replace the old Description with the
> new one.

Slightly messy, because the registered 639-2 tag apparently
isn't what 639-3 considers as "generic".  As a tag it has the
semantics of their "specific" code.  Which isn't the registered
tag.  Now that's confusing as soon as you try to talk about it.

While "Malay is Malay (unless you want Ambonese etc.)" should
be clear for readers not interested in these subtleties.  

> Otherwise, add the new Description to the existing ones.

Yes, but I don't like its outcome.  Can we do this without the
"otherwise" qualifier, simply add it always as is ?  For the
few cases you found we don't need optimizations and convoluted
special rules.

Frank