Subject: [Ltru] my technical position on extlang

From: Martin Duerst <[email-masked]>
Date: Sun, 18 May 2008 10:45:39 +0900

As a technical contributor, this is my technical position on extlang:

To me, the very recent discussion has shown that going back to
extlangs is preferable. The main reason for this, for me, is that
it's easier to explain and deploy. There are very few people,
even on this list, who can exactly explain why and where extlangs
pose problems, and why they might be (somewhat) less desirable
than not having them in some circumstances. On the other hand,
explaining to people that zh is Chinese, zh-yue is Cantonese, and
zh-cmn is Mandarin, seems more straightforward. Designing good
standards and good technology means looking at all the details,
but then going back to make sure the big picture is right.

In my understanding, one of the main points against extlangs was
that the "remove from the right" fallback hierarchy may not be
right. That may or may not be true, depending on the case at hand,
but that may or may not be true also for the relative order of
script and region. Here, there is an advantage of having the
hierarchy in the tag. More advanced matching algorithms can be
developped without having to do registry lookup.

So as a consequence, I personally favor going back to using
extlangs, preferably exclusively.

I'm okay with using extlangs only for those cases that already
have a macrolanguage subtag currently available, or some other
reasonable, not too 'cherry-picky' cutoff.

I'm opposed to allowing both extlang and independent tags
(i.e. both zh-yue and yue), but if that's what we as a WG
decide, I can live with it.

Regards,    Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]     

_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: John Cowan <[email-masked]>
Date: Sat, 17 May 2008 21:57:50 -0400

Martin Duerst scripsit:

> right. 

The problem is, AFAIU, that too much information is thrown away:
zh-yue-Hant does not fall back to zh-Hant, but first to zh-yue and
then to zh.

> decide, I can live with it.

I don't think anyone has ever suggested that: it's either/or.

-- 
LEAR: Dost thou call me fool, boy?      John Cowan
FOOL: All thy other titles              http://www.ccil.org/~cowan
             thou hast given away:      [email-masked]
      That thou wast born with.
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Gerard Meijssen" <[email-masked]>
Date: Sun, 18 May 2008 08:29:07 +0200

Hoi,
I respectfully disagree, when yue, cmn are considered separate languages it
is impossible to explain why they should be called zh-yue, zh-cmn. When sign
languages are considered seperate languages it is impossible to explain why
they would be sgn-ase.

It is nice when you are deep into the thought processes as the people on
this list are that this thinking can be explained. I have tried to explain
it to people and I see eyes glaze over each time. When a language is a
language, it should be allowed to use them as such without any consideration
of the ballast of the future. In the end it is computers that  interpret
these codes, they do not have the same problems with synonymy
Thanks,
     Gerard

---

From: "Debbie Garside" <[email-masked]>
Date: Sun, 18 May 2008 09:53:20 +0100

I agree with Martin very strongly.

However, I do think that there is a danger that the end user will mistag
'zh-yue' as 'yue' and 'zh-cmn' as 'cmn'; purely from looking at ISO 639-3.
I think that this should be highlighted in the document and that users
should be advised to allow for it; where 'yue' is presented it SHOULD be
converted to 'zh-yue' etc.

It really must be made perfectly clear that 'zh' is only to be used when
referring to Chinese languages as a whole or when the individual language
within 'zh' is unknown.

best regards

Debbie

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Martin Duerst
> Sent: 18 May 2008 02:46
> To: LTRU Working Group
> Subject: [Ltru] my technical position on extlang
>
> To me, the very recent discussion has shown that going back
> to extlangs is preferable. The main reason for this, for me,
> is that it's easier to explain and deploy. There are very few
> people, even on this list, who can exactly explain why and
> where extlangs pose problems, and why they might be
> (somewhat) less desirable than not having them in some
> circumstances. On the other hand, explaining to people that
> zh is Chinese, zh-yue is Cantonese, and zh-cmn is Mandarin,
> seems more straightforward. Designing good standards and good
> technology means looking at all the details, but then going
> back to make sure the big picture is right.
>
> In my understanding, one of the main points against extlangs
> was that the "remove from the right" fallback hierarchy may
> not be right. That may or may not be true, depending on the
> case at hand, but that may or may not be true also for the
> relative order of script and region. Here, there is an
> advantage of having the hierarchy in the tag. More advanced
> matching algorithms can be developped without having to do
> registry lookup.
>
> I'm okay with using extlangs only for those cases that
> already have a macrolanguage subtag currently available, or
> some other reasonable, not too 'cherry-picky' cutoff.
>

---

From: Peter Constable <[email-masked]>
Date: Sun, 18 May 2008 21:56:08 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Martin Duerst

> reasonable, not too 'cherry-picky' cutoff.

I think the cases for which extlangs might be most beneficial are those in which the macrolanguage ID has been in existing usage. For instance, we entertain the possibility in the case of Chinese because 'zh' has been in established use; if we had never used 'zh' before but had been using 'cmn', 'yue, etc., then we wouldn't hesitate to reject the suggestion if introducing 'zh' and changing 'cmn' etc. to be extlangs. Similarly, if people are just starting to use 'scs' and 'xsl' (North Slavey and South Slavey, respectively -- two of the official language of the Northwest Territories), then there's probably no particular benefit to using the ID for the encompassing macrolanguage and making them extlangs -- 'den-scs and 'den-xsl'.

Peter

---

From: Leif Halvard Silli <[email-masked]>
Date: Tue, 20 May 2008 12:40:18 +0200

Martin Duerst 2008-05-18 03.45:

> I'm opposed to allowing both extlang and independent tags (i.e. both
> zh-yue and yue), but if that's what we as a WG decide, I can live
> with it.

I would *prefer* both to be allowed. However, I think one should be 
required to choose between

    * EITHER to use the macrolanguage tag as first subtag and then to
      "sub qualify" the macrolanguage tag with a encompassed language
      tag. Region tag would here be forbidden.
    * OR to use the encompassed language subtag as the first subtag, and
      then to "sub qualify" it with the region subtag.
    * The above means that 'zh-cmn' and 'cmn-tw' would be legal. While
      'zh-cmn-tw' would be illegal/deprecated/forbidden.

The "macrolanguage-extlang" (e.g. 'zh-yue') *SHOULD* be used when one 
*wishes* to use the macrolanguage info for fallback! But if one do not 
want to express any such recommended fallback, then dropping the 
macrolanguage subtag (writing instead "yue" or "yue-script") is the 
right choie. (Implementations *MAY* still use it the macrolanguage info 
for fallback, though.)

This EITHER/OR way should, in my view (Doug, do you disagree?), open the 
way for also allowing the "old" macrolanguages from 639-1, such as 
Norwegian, to be used to form tags like "no-nn" and "no-nb".

If one forms the tag using the 'macrolanguage-encompassed' construct, 
then no region information can be stated (according to the EITHER/OR 
rule I suggested above.) The use of  the 'macrolanguage-encompassed' 
contstruct means that the encompassed language information is being 
considerred more relevant information than the region code value.

Regards,
Leif Halvard Silli

---

From: Leif Halvard Silli <[email-masked]>
Date: Tue, 20 May 2008 13:41:17 +0200

> zh-yue-Hant does not fall back to zh-Hant, but first to zh-yue and

If we say the tag must EITHER be formed as 'zh-yue-Hant' OR as 
'yue-(region)-Hant', then we can have different fallback rules for 
either. For 'zh-yue-Hant', then "take away the second subtag" could be 
the rule.

I.e. the use of 'macroLang-encompLang' could defined as the preferred 
construction when a fallback to the macrolanguage is recommended (by the 
author and/or the spec) as a good choice and/or when the 
"macroLang-encompLang" construct is considered more important/relevant 
than "encompLang-Region".

Thus this way we would permit the tagger to state whether fallback to 
the macrolanguage tag SHOULD happen or not.

In case the tagger uses 'encompLang-(whaterver)' then the same rules as 
currently proposed counts: Then implementations MAY use the 
macrolanguage information for fallback decisions.

> I don't think anyone has ever suggested that: it's either/or.

See Phillips, Addison 2008-05-13 01.11, under the subject "Re: [Ltru] 
extlang or not extlang", and Karen's reply. Above, and in my previous 
post, I have hopefully given the "better spelled out usage rules" that 
Karen asked for.
-- 
leif halvard silli
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Doug Ewell" <[email-masked]>
Date: Tue, 20 May 2008 07:29:59 -0600

I had written:

> I will have nothing further to say on this topic, except to respond to 
> proposals that violate the syntax of RFC 4646.

Leif Halvard Silli <lhs at malform dot no> wrote:

> This EITHER/OR way should, in my view (Doug, do you disagree?), open 
> the way for also allowing the "old" macrolanguages from 639-1, such as 

If 'nn' and 'nb' in these tags are intended to mean Nynorsk and Bokmål, 
then this proposal violates the syntax of RFC 4646.  Non-initial, 
non-private-use, two-letter subtags in non-grandfathered tags can *only* 
be region subtags, not some other type of subtag.

See sections 2.1 and 2.2.4 in RFC 4646, and similar text in the current 
draft.  It *must* be possible to identify the type of a subtag in a 
regular, non-grandfathered tag based solely on its length and position 
within the tag, without having to consult the Registry.  This is a basic 
requirement.

I almost included "no-nn" and "no-nb" as examples of violating the 
syntax of RFC 4646, but deleted them at the last second because this had 
already been mentioned on the list, and I thought it would not be 
necessary to call out Leif in this way.

--
Doug Ewell  *  Arvada, Colorado, USA  *  RFC 4645  *  UTN #14
http://www.ewellic.org
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages  ˆ

---

From: Leif Halvard Silli <[email-masked]>
Date: Thu, 22 May 2008 03:48:57 +0200

Doug Ewell 2008-05-20 15.29:
>> the way for also allowing the "old" macrolanguages from 639-1, such 
>> as Norwegian, to be used to form tags like "no-nn" and "no-nb".
>

They are.

> non-private-use, two-letter subtags in non-grandfathered tags can 
> *only* be region subtags, not some other type of subtag.
>
> See sections 2.1 and 2.2.4 in RFC 4646, and similar text in the 
> current draft.  It *must* be possible to identify the type of a subtag 
> in a regular, non-grandfathered tag based solely on its length and 
> position within the tag, without having to consult the Registry.  This 
> is a basic requirement.

Actually, my proposal does fulfil that requirement.

Because, I said that one should have to choose between using a region 
subtag or a extlang subtag.

By implication, this means that I propose putting and end to the 
"region" subtag class as such. And instead replacing it with a "region 
and extlang class", as one class. One would then not need to consult the 
Registry in order to know that in "no-nn" the "nn" belongs to the 
"region and extlang class".

After all, this is a language tagging system. It is not a locale tagging 
system. The tag "en-GB" does not mean "English made in Britain". It 
simply means "English following the Great-Britain variant". (I have seen 
many sites outside the British domain that uses "en-GB".)

> syntax of RFC 4646, but deleted them at the last second because this 
> had already been mentioned on the list, and I thought it would not be 

I did bring them into this list as examples of usage which, 
unfortunately, is wrong.
-- 
leif halvard silli
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Doug Ewell" <[email-masked]>
Date: Wed, 21 May 2008 22:04:50 -0600

Leif Halvard Silli <lhs at malform dot no> wrote:

> and extlang class", as one class. One would then not need to consult 
> the Registry in order to know that in "no-nn" the "nn" belongs to the 

Hmm.  Well, I guess we'll have to see how the rest of the group feels 
about making this major, fundamental, breaking change.

---

From: Peter Constable <[email-masked]>
Date: Wed, 21 May 2008 23:24:09 -0700

> Doug Ewell

> > and extlang class", as one class...

> about making this major, fundamental, breaking change.

Like breakfast at Milliways: let's do this after we've done six other impossible things.

Peter

---

From: Martin Duerst <[email-masked]>
Date: Fri, 23 May 2008 16:07:06 +0900

Just for the record, my technical opinion is that we should not make
such a change.

As a co-chair, I wouldn't even be sure whether such a major change
would be allowed by our current charter, but I think unless I suddenly
see a huge groundswell of support for Leif's proposal, I won't
need to form an opinion on this.

Regards,    Martin.

At 13:04 08/05/22, Doug Ewell wrote:
>> By implication, this means that I propose putting and end to the "region" subtag class as such. And instead replacing it with a "region and extlang class", as one class. One would then not need to consult the Registry in order to know that in "no-nn" the "nn" belongs to the "region and extlang class".
>
>Hmm.  Well, I guess we'll have to see how the rest of the group feels about making this major, fundamental, breaking change.
>
>--
>http://www.alvestrand.no/mailman/listinfo/ietf-languages  〓
>