Subject: Re: [Ltru] text proposition for extlang filtering by rfc 4647

From: Nicolas Krebs <[email-masked]>
Date: Sun, 4 May 2008 23:03:57 +0200

Doug Ewell wrote 

>Nicolas Krebs <nicolas1 dot krebs3 at netcourrier dot com> wrote:
>
>> If no tag matches the language priority list, application MAY try/try 
>> find/search the macrolanguage (as defined by [ISO 639-3]
>> http://www.sil.org/iso639-3/scope.asp#M
>> http://www.sil.org/iso639-3/scope.asp#C
>> http://www.sil.org/iso639-3/macrolanguages.asp
>> draft-ietf-ltru-4646bis-13 ) of the remaining primary language subtag.
>>
>> Examples:
>> 1 cmn -> zh
>> 2 hsb -> wen
>> 3 nb -> no
>> 4 src -> sc
>
>We are already planning to include the ISO 639-3-defined macrolanguage 
>in the Registry, in a new "Macrolanguage" field:
>
>%%
>Type: language
>Subtag: nb
>Description: Norwegian Bokmål
>Added: 2005-10-16
>Macrolanguage: no
>Suppress-Script: Latn
>%%
>
>See http://www.ietf.org/internet-drafts/draft-ietf-ltru-4645bis-04.txt, 
>soon to be replaced by -05.
>
>This saves the user from having to go to the SIL site to find 
>macrolanguage information, which is consistent with the philosophy of 
>the Registry: to save users from having to go all over the place to get 
>tagging information.

ACK. 
"Macrolanguage Field" is already in draft-ietf-ltru-4646bis-09. 
I could have seen by reading it one more time. 
Thanks for the remark. 

New draft (i have also corrected the examples with values of 
draft-ietf-ltru-4645bis-04): 

If no tag matches the language priority list, application MAY try to 
match the macrolanguage (if exist in the registry) of the remaining 
primary language subtag. See draft-ietf-ltru-4646bis section 3.1.9 
"Macrolanguage Field"
http://tools.ietf.org/html/draft-ietf-ltru-4646bis-09#section-3.1.9

Examples: 
1 cmn -> zh
2 nn -> no
3 src -> sc
4 sr -> sh

Interoperability issue
Other macrolanguage lookup developed for one application 
SHOULD NOT be assumed interoperable and SHOULD 
not be shared between application 
(see draft-ietf-ltru-4646bis-13 section 4.5). 

Examples: 
1 zdj -> swb
2 hsb -> wen
3 gsw-fr -> fr
4 gsw -> gem

_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: Leif Halvard Silli <[email-masked]>
Date: Mon, 05 May 2008 00:55:15 +0200

> http://tools.ietf.org/html/draft-ietf-ltru-4646bis-09#section-3.1.9

For a macrolanguage with 3 codes but only two options, trying to match 
the macrolangauge allready  implies that one are able to accept both 
options.  And since the above text says that an application should go 
through the priority list before consulting the macrolanguage, then, 
when consulting the macrolanguage, it would be illogical if the 
application stopped after having tried to get a match for the 
macrolanguage. In other words, it should also try to match the last and, 
in this context, synonymous tag.

In 639-3 I counted around 22 macrolanguages, including Norwegian, for 
which the above is true.

Hence, I propose this text, to follow immediately after Nicolas' text above:

    An application which tries to match a macrolanguage of a primary
    language subtag, as specified above, SHOULD, on the condition that
    the macrolanguage covers only 2 languages, also lookup the
    alternative representation of that macrolanguage.

    Examples:
    1 nn  -> no -> nb
    2 nb  -> no -> nn
    3 tw  -> ak -> fat
    4 fat -> ak -> tw

-- 
leif halvard silli

---

From: "Mark Davis" <[email-masked]>
Date: Sun, 4 May 2008 16:29:56 -0700

The goal for fallback is to provide the most likely satisfactory language,
in the absence of a exact match. A poor choice of fallback causes problems;
where the fallback language is not likely to be understood, it can produce
much worse results than fallback to a default (or a subsequent language in a
Language Priority List).

Macrolanguages have to be used very, very carefully for fallback, because
there no guarantee at all that speakers of two microlanguages (of the same
macro language) are mutually comprehensible. In some cases it is true, but
in most cases, it is not.

In particular cases, such as Norwegian, what you list is probably a desired
fallback, with one addition.

   1. nn -> no -> nb
   2. nb -> no -> nn
   3. no -> nb -> nn

But Norwegian is a particularly easy case for macrolanguages, because it is
associated with a single country and script, and most readers of nn can also
read nb.

The desired fallback chain with macro/microlanguages in other cases gets
much more complicated, where the languages are spoken in multiple regions,
or have multiple scripts, or vary in degrees of comprehension. Moreover,
focusing on macrolanguages can overshadow other better fallback
relationships; ro and mo are far more closely related (and useful for
fallbacks) than most microlanguages are, as are tl and fil. Moreover, even
unrelated languages may be better fallbacks than closely related ones, such
as br -> fr.

So macrolanguages are best viewed as very useful information for people to
use in creating fallbacks, but should not be taken as defining those
fallbacks.

Mark

-- 
Mark

---

From: Leif Halvard Silli <[email-masked]>
Date: Mon, 05 May 2008 02:34:06 +0200

Mark Davis 2008-05-05 01.29:
> [....]

You forgot 4. no -> nn -> nb. However, you are blurring my message.

Specifying what applications MAY do when the language preferences are 
set up using the macrolanguage tag, is a different usecase from the one 
mentioned in the text of Nicolas. Therefore I did not propose any thing 
for that usecase. And I do not want to make any proposal for it either. 
(And if I should change my mind about that, then I do not want to have 
it mangled up with the issue which I take up in this text.)

> But Norwegian is a particularly easy case [...]
>   
> [...] Moreover, even unrelated languages may be better 
> fallbacks than closely related ones, such as br -> fr.

There are several languages which are simple to cover, when we limit the 
options to the usecase for which Nicolas' proposal which said that 
"application *MAY* try to  match the macrolanguage".

My proposal simply said that when that application chooses to do what it 
MAY do, namely to look up the macrolanguage, then, as long as the 
macrolanguage only cover two languages,  then it SHOULD also look up the 
other languga, since looking up the macrolanguage by default is a yes to 
the risk of getting the other language anyhow. For those macrolanguages 
which covers more than 2 languages, there is no mathematical certainity 
about this, so therefore I did not make any proposal for them.

Thus I maintain my proposal.

-- 
leif halvard silli