Subject: [Ltru] Re: Great Script Debate "the Next Generation"...

From: "Doug Ewell" <[email-masked]>
Date: Sat, 21 Oct 2006 13:05:39 -0700

Kent Karlsson <kent dot karlsson14 at comhem dot se> wrote:

> As I mentioned, registering a *language* variant subtag for this 
> (digit preference for **generated** decimal digits, not (generated) 
> spellout digits) would be inappropriate, I think (I would not support 
> such a registration, at least). So would using a private use subtag.

I'd say those mechanisms are quite a bit more appropriate than the 
REGION subtag for making that distinction.

If you were making the argument that Thai as used in Thailand uses one 
convention for digits whereas Thai as used in some other country uses 
another convention, then I could see your point.  What we are talking 
about is the decision someone made when naming a locale.

If you are not comfortable with using a variant (more analogous to 
"1901" and "1996" than to the others) or a private-use tag, or even --  
God forbid -- an extension RFC, to identify this convention, then I 
think we will simply have to agree than locales and language tags are 
not the same thing and cannot always match 1-to-1.

--
Doug Ewell  *  Fullerton, California, USA  *  RFC 4645  *  UTN #14
http://users.adelphia.net/~dewell/
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Kent Karlsson" <[email-masked]>
Date: Sat, 21 Oct 2006 23:20:45 +0200

> REGION subtag for making that distinction.

No, the current name uses the region subtag twice, but I suggested to use
the SCRIPT subtag explicitly for making the distinction "th_Thai" vs. "th".

> or even --  

Addison seems to think that extension(s) would be appropriate for this.
While I don't find it ideal, its not the worst suggestion regarding this
(my nomination for worst suggestion would be the use of private-use
subtags for this).

		/kent k

---

From: "Mark Davis" <[email-masked]>
Date: Sat, 21 Oct 2006 16:37:21 -0700

It makes no sense to try to use the script Thai to distinguish between
orthographic variants of Thai that are only different according to whether
native digits are used or not. The absence of the script tag clearly does
not mean "use latin digits but otherwise not latin characters"). If such a
distinction is needed, then the only way to do it with language tags is via
a variant or extension.

Mark

---

From: "Kent Karlsson" <[email-masked]>
Date: Sun, 22 Oct 2006 13:26:34 +0200

Mark Davis wrote:

It makes no sense to try to use the script Thai to distinguish between orthographic variants of Thai that are only different according to whether native digits are used or not. The absence of the script tag clearly does not mean "use latin digits but otherwise not latin characters"). 

That holds even more true for using the region tag for making this distinction. But the difference does have something to do which which *script* is used...

 If such a distinction is needed, then the only way to do it with language tags is via a variant or extension. 

So it cannot currently be made using registered subtags.
 
        /kent k

---

From: "Mark Davis" <[email-masked]>
Date: Sun, 22 Oct 2006 08:52:54 -0700

No. Nor can many other choices. That's why systems such as CLDR have
additional variants to provide for finer distinctions in the orthography.

Mark

On 10/22/06, Kent Karlsson <[email-masked]> wrote:
> not mean "use latin digits but otherwise not latin characters").
>
> That holds even more true for using the region tag for making this
> distinction. But the difference *does* have something to do which which **
> script** is used...
>

---

From: "Doug Ewell" <[email-masked]>
Date: Sun, 22 Oct 2006 11:11:24 -0700

Just so I understand the discussion correctly:

Are we talking about language tags per se, or are we talking about 
locale names transformed int olanguage-tag syntax?

Language tags will never incorporate all of the distinctions that are 
made in locales.  Look at the locale names that have been invented to 
identify character set, sort order, euro vs. non-euro, and so forth. 
CLDR allows for these distinctions because it is a LOCALE data 
repository, but it should not be inferred that they should necessarily 
propagate back to language tags.

----- Original Message ----- 
From: Mark Davis
To: Kent Karlsson
Cc: Doug Ewell ; LTRU Working Group
Sent: Sunday, October 22, 2006 8:52
Subject: Re: [Ltru] Re: Great Script Debate "the Next Generation"...

No. Nor can many other choices. That's why systems such as CLDR have 
additional variants to provide for finer distinctions in the 
orthography.

Mark

On 10/22/06, Kent Karlsson <[email-masked]> wrote:
Mark Davis wrote:
It makes no sense to try to use the script Thai to distinguish between 
orthographic variants of Thai that are only different according to 
whether native digits are used or not. The absence of the script tag 
clearly does not mean "use latin digits but otherwise not latin 
characters").
That holds even more true for using the region tag for making this 
distinction. But the difference does have something to do which which 
*script* is used...
 If such a distinction is needed, then the only way to do it with 
language tags is via a variant or extension.
So it cannot currently be made using registered subtags.

        /kent k

---

From: "Kent Karlsson" <[email-masked]>
Date: Sun, 22 Oct 2006 21:44:02 +0200

> locale names transformed int olanguage-tag syntax?

The difference is quite fuzzy. And indeed W3C's LTLI makes
no difference, and CLDR makes very little difference.

> identify character set,

Has no business in locales. I know that POSIX locales incorporate
this rather major design flaw.

> sort order,

Seems to belong in locales (or rather locale data sets, being
selected by locale name).

> euro vs. non-euro,

Has no business in locales. The unit of a value can never be
determined by a locale. I know that there are locale systems
that incorporate this major design mistake. No well-written
program should ever rely on that part of a locale system that
incorporates that. 1$ and 1¥ are really different, and the unit
must be tied to the value *regardless* of locale. TRANSLATING
unit names (including currency names) can well be done in locales.

> and so forth. 

Like paper size? Again, that has no business in locales. (Page size
a per-printer default, and may be overridden by documents that
include a page size, which then even may vary per page.)

Like "imperial" vs. SI units? See above re. currency (and other!!) units.

> repository, but it should not be inferred that they should 
> necessarily propagate back to language tags.

Most of the variant designations in CLDR do properly belong as
language variant subtags (but a registration may change the name used).
SAAHO designates a dialect, and LSTR has already dialect variants.
I see no immediate reason why SAAHO should not be registered
as a variant subtag in LSTR. POLYTONI and REFORMED refers to
spell reforms, and LSTR already has spell reform variants. I see no
immediate reason why these spell reforms should not be registered
in LSTR (under different names). POSIX and "root" are indeed special
cases, though I think a private use variant (or indeed an extension
variant) should be used for POSIX; but it is indeed a special case.

The case of decimal digit script has already been made a locale
difference, and I see no particular mistake or special case being
warranted for demoting this from being a (properly designed) locale
difference. I do think it should be designated by a *language tag*
name. Either by using the script subtag as an "emphasis" (which I
still find to be the best in the particular case of Thai digit generation)
or by using extension subtags (NOT private use). In this particular
case I don't think an ordinary variant should be used, but I'll abide
with that if one is registered.

Think of web forms. The language tag, and language "negotiation",
seems to be an appropriate way of choosing between having number
and date fields in Thai digits or in "Latin" digits in an otherwise
Thai language form. (Using spellout, and which kind of spellout,
seems much less appropriate to select in that way. But I agree that
that may be a borderline case, especially for CJK.)

	/kent k

---

From: "Mark Davis" <[email-masked]>
Date: Sun, 22 Oct 2006 14:33:07 -0700

I need to correct some core misconceptions. While CLDR has what is called a
locale ID, for historical reasons, it really functions (with a couple of
exceptions) as a language ID, and accesses language-based data. There used
to be some information that was improperly included in the language-based
data, like default currency and weekend ranges, but that was removed over
time; moved to supplemental files like
http://unicode.org/cldr/data/common/supplemental/ (for an HTML view, see
http://www.unicode.org/cldr/data/charts/supplemental/index.html). This is
not even really "locale" data so much as it is just non-language data.

And as to the specific items mentioned,

   - CLDR locales have never been distinguished by charset: all of the
   data is in Unicode. (There are tools for generating POSIX charset-specific
   data, but those are separate.)
   - Euro vs non-Euro is not part of the language-based data
   - Sort order is part of the language-based data, and to my mind
   properly so: sort order is keyed off of language.

Now, what are the exceptions I mentioned above? For backwards compatibility,
CLDR IDs do permit setting a default calendar, currency, and timezone. All
of these are really orthogonal to language-data:

   - The use of a timezone in a CLDR ID would be a pretty clear case of
   the use of a CLDR ID as signifying a locale, according to most concepts of
   what a "locale" means.
   - The choice of calendar is somewhat fuzzier, but probably would be
   included in what many people mean by a locale.
   - A currency is really a property of a transaction, and not really
   properly part of a locale at all.

For ICU, a major implementation using CLDR, these are actually not
necessary; there are alternate, preferred mechanisms for dealing with these
objects. For example, a Currency is supplied directly to the API that
formats a monetary amount.

Mark

---

From: "Mark Davis" <[email-masked]>
Date: Sun, 22 Oct 2006 14:47:36 -0700

We would like to not need any special CLDR variant subtags. We only have a
few (non-deprecated ones) in CLDR 1.4.

   - hy_AM_REVISED will be replaced by the recent variants for armenian
   added to the LTR.
   - ar_ER_SAAHO will be replaced once we have ssy.
   - el_POLYTONI. The ideal result for this would be in the script
   position, since that will function better. But the script JAC, while happy
   to register other things that aren't scripts, has rejected it. So we'll
   probably apply for a registered variant, even tho' that will be suboptimal
   for polytonic users.
   - en_US_POSIX. This is a bit tricky. POSIX is an artifical variant
   used to get the default "C" style for numbers, dates, etc. It'd be handy to
   get it registered, but I doubt that there are enough pragmatists on the
   ietf-languages group to get it done.

Mark

---

From: "Doug Ewell" <[email-masked]>
Date: Sun, 22 Oct 2006 17:51:31 -0700

Mark Davis wrote:

> I need to correct some core misconceptions. While CLDR has what is 
> called a locale ID, for historical reasons, it really functions (with 
> a couple of exceptions) as a language ID, and accesses language-based 
> data. There used to be some information that was improperly included 
> in the language-based data, like default currency and weekend ranges, 
> but that was removed over time; moved to supplemental files.... This 
> is not even really "locale" data so much as it is just non-language 
> data.

"Locale data" is a very gray area; if you give N informed people a 
long-enough list of items (sort order, script, charset, calendar, 
currency, time zone, SI vs. imperial units) and ask them to pick which 
ones belong in a locale and which ones don't, you will likely get at 
least (N - 1) result sets.  I don't want to belabor that, except to say 
that "locale data" often does include items that go beyond "language 
identification" and probably don't belong in RFC 4646(bis).

Only one specific item:

> Sort order is part of the language-based data, and to my mind properly 
> so: sort order is keyed off of language.

Here's what I meant:  Internet Explorer 7, which has come a long way by 
adopting RFC 4646 tags to indicate "languages," still identifies "es-ES" 
with "Spanish (International Sort)".  There are a lot of differences 
between Spanish in Spain (which itself is not uniform) and Spanish 
elsewhere, such as pronunciation and vocabulary and vosotros/ustedes, 
that may be more linguistically important than the pre-1994 sort order.

There is a fine line, somewhere between pure language distinctions and 
things that are clearly not language, and I'm sure we could battle for 
ages on exactly where that line is.  I do think we should try to avoid 
using RFC 4646 directly for things that are clearly not language. 
(Using it indirectly, as CLDR does, is fine.)  In particular:

> en_US_POSIX. This is a bit tricky. POSIX is an artifical variant used 
> to get the default "C" style for numbers, dates, etc. It'd be handy to 
> get it registered, but I doubt that there are enough pragmatists on 
> the ietf-languages group to get it done.

... you can definitely count me as one of the pragmatists in this case.

---

From: "Randy Presuhn" <[email-masked]>
Date: Sun, 22 Oct 2006 19:50:01 -0700

Hi -

To everyone and no one in particular....

This thread is veering far from this working group's
chartered space: language tag registry update.  Please
focus on the work at hand, as spelled out by our charter.
And as we are working with tags for the identification of
languages, please be aware that while there may be all sorts
of other possible tags for various purposes, they are outside
our scope.  These other discussions, while perhaps interesting
to some, belong elsewhere.

Randy
ltru co-chair