Subject: [Ltru] I'm really confused by chinese in 3066bis

From: Shawn Steele <[email-masked]>
Date: Fri, 2 May 2008 11:39:34 -0700

I'm confused by the text surrounding Chinese.  I've been out for a few weeks so I may have missed some of the discussion.  The draft seems contradictory, but with the draft as my only guide, my interpretation of what I'm reading now for Chinese is:

* zh should be pretty much assumed to be Mandarin.
* yue should be used if you want Cantonese.
* zh-cmn should be used if you want to say "don't just assume this is Mandarin, it really is Mandarin."

I get this from the end of 4.1 (choice) http://www.inter-locale.com/ID/draft-ietf-ltru-4646bis-13.html#choice:

  "In general, use the most specific subtag to form the language tag. However, where the macrolanguage tag has been historically used to denote a dominant encompassed language, it SHOULD be used in place of the subtag specific to that encompassed language unless it is necessary to clearly distinguish the macrolanguage as a whole from that enclosed dominant language variety."

So this sounds like I'm supposed to use "zh" instead of "cmn" "unless it is necessary"

Earlier in 2. of 4.1 it says:

  "When labeling content that is unwritten (such as a recording of human speech), the script subtag should not be used, even if the language is customarily written in several scripts. Thus the subtitles to a movie might use the tag "zh-cmn-Hant" (Chinese, Mandarin, Traditional script), but the audio track for the same language would be tagged "zh-cmn". "

So apparently it is "necessary" to use a tag for Mandarin in some cases.  This example uses an irregular grandfathered tag though, I don't get that.  This section is "choice", presumably I shouldn't choose a grandfathered tag.

Later we have:

  "In particular, the Chinese family of languages call for special consideration. Because the written form is very similar for most languages having 'zh' (Chinese) as a macrolanguage (and because historically subtags for the various encompassed languages were not available), languages such as 'yue' (Cantonese) have historically used either 'zh' or a tag (now grandfathered) beginning with 'zh'. This means that macrolanguage information can be usefully applied when searching for content or when providing fallbacks in language negotiation. For example, the information that 'yue' has a macrolangauge of 'zh' could be used in the Lookup algorithm to fallback from a request for "yue-Hans-CN" to "zh-Hans-CN" without losing the script and region information (even though the user did not specify "zh-Hans-CN" in their request). "

This is very different from the Mandarin description in 4.1 (2.)

So from the text of the draft I'm not entirely certain what is expected.  As mentioned, I was out, so this may have been discussed and I missed it, but if I only have the draft as context I would have to assume:

* zh == cmn == zh-cmn and I need to do lookup for each of those
* yue is explicitly Cantonese, but I may need to look at zh during lookup.

Can someone clarify what the expectation is?  I would guess that by the table and the macrolanguage SHOULD be used statement that:
* Only zh should be used to represent Mandarin, and that
* only yue should be used to represent Cantonese, which optional fallback lookup to zh if desired.

If that is the case, then the zh-cmn-Hant / zh-cmn example should be corrected to zh-Hant and zh.  Also cmn should then be listed as grandfathered if zh always means Mandarin and is preferred.  If my guess isn't correct, then whatever appropriate additional clarifications should be made.

- Shawn

Shawn Steele

Senior Software Design Engineer
Windows International / Microsoft .Net
Microsoft

_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: John Cowan <[email-masked]>
Date: Fri, 2 May 2008 15:03:59 -0400

Shawn Steele scripsit:

> I'm confused by the text surrounding Chinese.  I've been out for a few
> weeks so I may have missed some of the discussion.  The draft seems
> contradictory, but with the draft as my only guide, my interpretation
> of what I'm reading now for Chinese is:

I assume that when you say "3066bis" you actually mean "4646bis", and are
asking about the meaning of the new draft rather than of the existing RFC.

> * yue should be used if you want Cantonese.

Yes.

> Mandarin, it really is Mandarin."

Rather, the appropriate tag for that meaning in new material will be
"cmn".  "zh-cmn" becomes a grandfathered form equivalent to "cmn".

> I get this from the end of 4.1 (choice)
> http://www.inter-locale.com/ID/draft-ietf-ltru-4646bis-13.html#choice:
> 
>   "In general, use the most specific subtag to form the language
>   tag. However, where the macrolanguage tag has been historically
>   used to denote a dominant encompassed language, it SHOULD be used
>   in place of the subtag specific to that encompassed language unless
>   it is necessary to clearly distinguish the macrolanguage as a whole
>   from that enclosed dominant language variety."
> 

Yes, provided that "pretty much assumed" does not mean "must be used for".
There are occasional reasons not to use it.

>   "When labeling content that is unwritten (such as a recording of human
>   speech), the script subtag should not be used, even if the language
>   is customarily written in several scripts. Thus the subtitles to a
>   movie might use the tag "zh-cmn-Hant" (Chinese, Mandarin, Traditional
>   script), but the audio track for the same language would be tagged
>   "zh-cmn". "

Oops, that should be fixed to remove the "zh-" part.

Addison, can you make this fix quickly and look for other places where
zh- or ar- might appear and see if they are now correct?

> So apparently it is "necessary" to use a tag for Mandarin in some
> cases.  This example uses an irregular grandfathered tag though,
> I don't get that.  This section is "choice", presumably I shouldn't
> choose a grandfathered tag.

Right.

> Later we have:
> 
>   "In particular, the Chinese family of languages call for special
>   consideration. Because the written form is very similar for most
>   languages having 'zh' (Chinese) as a macrolanguage (and because
>   historically subtags for the various encompassed languages were not
>   available), languages such as 'yue' (Cantonese) have historically
>   used either 'zh' or a tag (now grandfathered) beginning with
>   'zh'. This means that macrolanguage information can be usefully
>   applied when searching for content or when providing fallbacks in
>   language negotiation. For example, the information that 'yue' has
>   a macrolangauge of 'zh' could be used in the Lookup algorithm to
>   fallback from a request for "yue-Hans-CN" to "zh-Hans-CN" without
>   losing the script and region information (even though the user did
>   not specify "zh-Hans-CN" in their request). "
> 

Not really.  The core idea is that the "zh" tag is vague/inclusive and can
be used for any variety of Chinese.  However, most spoken Chinese is in fact
Mandarin, and almost all written Chinese is in fact Mandarin, so there is
a tendency to assume that "zh" simply means Mandarin.  In order to benefit
people who must distinguish, the tag "cmn" is introduced to mean "specifically
Mandarin, excluding all non-Mandarin varieties".

> So from the text of the draft I'm not entirely certain what is expected.
> As mentioned, I was out, so this may have been discussed and I missed
> it, but if I only have the draft as context I would have to assume:
> 

Most Mandarin is tagged zh, but some may be cmn == zh-cmn instead.
Cantonese might be any of zh, zh-yue, or yue.

> Can someone clarify what the expectation is?  I would guess that by
> the table and the macrolanguage SHOULD be used statement that:

Not so.  "zh" should be used unless there is a compelling reason to use
"cmn".

> * only yue should be used to represent Cantonese, which optional fallback lookup to zh if desired.

Yes.

-- 
Real FORTRAN programmers can program FORTRAN    John Cowan
in any language.  --Allen Brown                 [email-masked]
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: [email-masked]
Date: Fri, 2 May 2008 12:25:37 -0700

It seems like we're suggesting the following:

1) The shortest tag for written Mandarin is "zh".
2) The shortest tag for spoken Mandarin is "cmn" unless of course you feel 
like using "zh" (which, psst, might be a good idea because that's how it's 
been used even if that isn't exactly what it means) and by the way, you 
can also use "zh-cmn" or "zh-guoyu."

I think that the shortest tag for Mandarin in each case -- if extlang is 
removed and we do not want to change the tag semantics -- should be "cmn." 
If it's not "cmn," you've changed the semantic of the "zh" tag to 
disinclude "yue."  With extlang, you have "zh-cmn" and "zh-yue" and the 
semantics of "zh" remain consistent. The argument that an overwhelming 
number of documents tagged with "zh" are Mandarin does not sway me. I have 
Cantonese subtitles. 

There are differences of opinion on how I should accurately tag my 
Taiwanese Mandarin -- this seems to indicate that there is something in 
the draft that is ambiguous. 

In written form, it seems like the variants of what ISO calls Chinese are 
mutually intelligible (dialects). In spoken form, they are languages. This 
seems to create some odd scenarios and no clear recommendation on the best 
tag to use for these languages. I agree with Shawn that there is something 
wrong here. I just wish I could better express my reservations.

Regards,

Karen Broome

---

From: "Randy Presuhn" <[email-masked]>
Date: Fri, 2 May 2008 12:46:02 -0600

Hi -

As a technical contributor...

> From: <[email-masked]>
> To: "John Cowan" <[email-masked]>
> Cc: "LTRU Working Group" <[email-masked]>
> Sent: Friday, May 02, 2008 1:25 PM
> Subject: Re: [Ltru] I'm really confused by chinese in 3066bis
>

I completely agree.
 
> the draft that is ambiguous. 

The precision (hopefully we tag everything "accurately" :-) of tagging
needed depends on the application.  You're probably in the best position
to make that call.

> wrong here. I just wish I could better express my reservations.

I think there's a temptation to allow the fallback heuristic to be the tail that
wags the dog.  Instead of "advising" anyone to use "zh" for Mandarin,
perhaps it would be better to give it as an example of a case where, owing
to the sociolinguistic situation in which Mandarin fins itself, one might,
for some (but not all!) significant applications, find it advantageous
to tag somewhat imprecisely (but still accurately).

Randy

---

From: Shawn Steele <[email-masked]>
Date: Fri, 2 May 2008 13:07:39 -0700

> asking about the meaning of the new draft rather than of the existing RFC.

Duh, yes :)

> Oops, that should be fixed to remove the "zh-" part.

That solves 90% of my confusion :)  Thanks.

> Not so.  "zh" should be used unless there is a compelling reason to use

Can someone please explain that if apps are mostly assuming that zh is usually Mandarin, when would it be compelling to use cmn?

Karen is also concerned because this effectively means that zh disincludes yue, which does seem like a valid conceptual concern.

If the desire is for Mandarin to be distinct from zh, then it seems to me that it should always be "cmn", unless someone really didn't know if it was Mandarin or another sublang.

If the desire is to say "zh is already Mandarin so we don't want to touch it", then I think that cmn should NOT be an option and zh should always be used.  I recognize that not all zh data may currently be Mandarin, so this is perhaps not quite accurate, which would argue that future data should maybe be tagged as "Mandarin".

Should the current message of "use zh unless its compelling to use cmn" is the final result, then as a content provider I think I'd always use cmn (I wouldn't want it to be confused with the perhaps-ambiguous presumed cmn that is zh).  For lookup I'd have to assume that cmn == zh or miss all the existing content.  When setting up http-accept-lang in a browser I'd have to pick both cmn & zh.  The net effect would be that cmn will be an alias for zh, and that we'll all have to have extra code just to make sure they're always treated similarly.

For that to not happen I'm going to need a MUCH better definition of "compelling" and when cmn should be used.

For Arabic I don't think the problem has nearly the same scale and I don't think content taggers or developers would have the same problem.

Sorry, but I'll be out for the weekend starting about now.

- Shawn

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 2 May 2008 13:27:15 -0700

I see it a bit differently:

   - cmn means specifically Mandarin Chinese, spoken or written (eg
   cmn-HK)
   - yue means specifically Cantonese Chinese, spoken or written (eg
   yue-HK)
   - zh means any Chinese, spoken or written, thus encompassing Mandarin,
   Cantonese, and others. It thus is formally correct to tag either Mandarin
   Chinese content or Cantonese content with 'zh'. (eg zh-HK).
   - However, because of the de facto predominance of Mandarin, 'zh' has
   long been used to mean Mandarin only. Because of this:
   - In practice, 'zh' and 'cmn' are best treated as synonyms
      - Where backwards compatibility is important, 'zh' is preferred
      over 'cmn'.
      - Content that is not Mandarin (eg Cantonese) should not be
      tagged with 'zh'
   - Some other macrolanguages behave the same way, while other
   macrolanguages do not have these predominance/compatibility issues, and
   should not be treated as synonyms with one of their encompassed languages.

Mark

-- 
Mark

---

From: <[email-masked]>
Date: Fri, 2 May 2008 14:15:29 -0700

Comments.

[email-masked] wrote on 05/02/2008 01:27:15 PM:

> cmn means specifically Mandarin Chinese, spoken or written (eg cmn-HK)
> yue means specifically Cantonese Chinese, spoken or written (eg yue-HK)
> zh means any Chinese, spoken or written, thus encompassing Mandarin,
> Cantonese, and others. It thus is formally correct to tag either 
> Mandarin Chinese content or Cantonese content with 'zh'. (eg zh-HK).

So far, so good.

> However, because of the de facto predominance of Mandarin, 'zh' has 
> long been used to mean Mandarin only. 

Disagree. It was simply the most specific tag one could use for the 
Mandarin at the time. That doesn't mean the tag ever meant "Mandarin" to 
anyone. We might as well say zh-TW means Traditional Mandarin as this tag 
has been used more often than zh-Hant. 

> In practice, 'zh' and 'cmn' are best treated as synonyms

OK, so you're changing the semantics of the tag in practice. I don't think 
there can be any other interpretation of that statement. I think this 
subtlety will be utterly lost on anyone who did not participate in this 
working group. Where else do we give users a choice as far as which 
language subtag to use? How am I going to tell people they can't use "eng" 
but that they have a choice of "zh" or "cmn" when it comes to Mandarin? 

> Where backwards compatibility is important, 'zh' is preferred over 
'cmn'.
> Content that is not Mandarin (eg Cantonese) should not be tagged with 
'zh'

Your average developer just gets a language requirement and wants to match 
it to a code. They may not be interested in assessing their need for 
backwards compatibility and multiple language modes as part of this task. 
This wasn't something that needed to be considered in the past.  Forcing 
the user to use conditional logic to figure out which tag to use is not 
likely to result in any sort of consistency of tagging and could even 
prevent the adoption and utility of this work.

Regards,

Karen Broome

> > > * zh-cmn should be used if you want to say "don't just assume this 
is
> > > Mandarin, it really is Mandarin."
> >
http://www.inter-locale.com/ID/draft-ietf-ltru-4646bis-13.html#choice:
> > > So this sounds like I'm supposed to use "zh" instead of "cmn" 
"unless
> > > it is necessary"
> > be used for any variety of Chinese.  However, most spoken Chinese is 
in
> fact

---

From: "Debbie Garside" <[email-masked]>
Date: Fri, 2 May 2008 23:16:08 +0100

I agree with Karen.

Debbie

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of [email-masked]
> Sent: 02 May 2008 22:15
> To: Mark Davis
> Cc: LTRU Working Group; [email-masked]
> > yue means specifically Cantonese Chinese, spoken or written (eg
> > yue-HK) zh means any Chinese, spoken or written, thus encompassing
> > Mandarin, Cantonese, and others. It thus is formally correct to tag
> > either Mandarin Chinese content or Cantonese content with
> 'zh'. (eg zh-HK).
>
> So far, so good.
>
> Disagree. It was simply the most specific tag one could use
> for the Mandarin at the time. That doesn't mean the tag ever
> meant "Mandarin" to anyone. We might as well say zh-TW means
> Traditional Mandarin as this tag has been used more often
> than zh-Hant.
>
> OK, so you're changing the semantics of the tag in practice.
> I don't think there can be any other interpretation of that
> statement. I think this subtlety will be utterly lost on
> anyone who did not participate in this working group. Where
> else do we give users a choice as far as which language
> subtag to use? How am I going to tell people they can't use "eng"
> > 2) The shortest tag for spoken Mandarin is "cmn" unless of
> course you
> feel
> > like using "zh" (which, psst, might be a good idea because
> that's how
> it's
> > number of documents tagged with "zh" are Mandarin does not
> sway me. I
> have
> > Cantonese subtitles.
> >
> > In written form, it seems like the variants of what ISO
> calls Chinese
> are
> > mutually intelligible (dialects). In spoken form, they are
> languages.
> This
> > seems to create some odd scenarios and no clear
> recommendation on the
> best
> > > > I'm confused by the text surrounding Chinese.  I've
> been out for a
> few
> > > I assume that when you say "3066bis" you actually mean
> "4646bis", and
> > are
> > > asking about the meaning of the new draft rather than of
> the existing
> > RFC.
> > >
> > > Yes, provided that "pretty much assumed" does not mean
> "must be used
> > for".
> > > >   "When labeling content that is unwritten (such as a
> recording of
> > human
> > > >   historically subtags for the various encompassed
> languages were
> not
> > > Not really.  The core idea is that the "zh" tag is
> vague/inclusive and
> > can
> > > Mandarin, and almost all written Chinese is in fact
> Mandarin, so there
> > is
> > > > * yue is explicitly Cantonese, but I may need to look
> at zh during
> > lookup.
> > >
> > > Not so.  "zh" should be used unless there is a compelling
> reason to
> use
> > > "cmn".
> > >

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 2 May 2008 15:19:31 -0700

> has been used more often than zh-Hant.

I stand by what I said: it has long been *used* to mean just Mandarin. If
you make a request using 'zh' to products from Google, Microsoft, Apple,
Sun, and so on, *in practice* that does *not* mean:

*Give me some kind of Chinese, I don't care whether it is Mandarin or
Cantonese or Hakka.*

Instead, *in practice*, it means:

***Give me Mandarin*.

> prevent the adoption and utility of this work.

Personally, if someone in my company came to me and said "what should I
use", I would unconditionally say "zh". If someone were to hand us 'cmn-CN',
I'd say, map that first thing to "zh-CN". Backward compatibility dominates.

The important thing for us is to get the *new* capability offered by "yue",
etc., not to make a distinction between 'zh' and 'cmn' that would be
extremely painful to implement and return *no* extra value for our users.

If you are in an isolated, new environment where you didn't have any
backwards compatibility issues, it'd be fine to make a distinction between
'zh' and 'cmn'. Those environments are going to probably be rather rare.

---

From: Leif Halvard Silli <[email-masked]>
Date: Sat, 03 May 2008 02:40:43 +0200

Mark Davis 2008-05-03 00.19:
> Fri, May 2, 2008 at 2:15 PM, <[email-masked]>:

> [...] it has long been *used* to mean just Mandarin. If you make a
> request using 'zh' to products from Google, Microsoft, Apple, Sun,

> [...] *in practice*, it means:
>
> ***Give me Mandarin*.
>   

Google:  The main Google product, www.google.com, only lets me search 
for "Chinese (Simplified)" and "Chinese (traditional)". Should I then 
assume that it, in practise, *excludes* texts in Cantonese and Hakka? 
And note: They say 'Chinese'. They do not say 'Mandarin'.

Apple:  Mac OS X directly uses the ISO language tags in their 
localisation system. And on Mac OS X, folders using the 'en' tag, are 
displayed as 'English' in menus etc in the GUI, despite the fact that 
they are filled with US English.  The Apple practise here does not mean 
that 'en' has changed meaning to 'en-US'.

> >> 'cmn'. Content that is not Mandarin (eg Cantonese) should not be
> >> tagged with 'zh'
> >

I agree with Karen about the need for making things very clear for 
developers.

Apple, again, uses zh-CN and zh-TW. But what will happen the day Apple 
will start to support Cantonese or Hakka?  To perhaps get a clue, I will 
tell you what happened the day Apple started to support Nynorsk.

When they started to do so in OS X 10.5 they made the fatal error to 
change the meaning of the 'no' from 'Norwegian' to 'Bokmål'.  In 
creating a Nynorsk localisation of iCab (www.icab.de), using OS X 10.4, 
I placed the localised files inside a 'no' folder. Then OS X 10.5 users 
began complaining that "oh, intereface says 'Bokmål', but is actually 
'Nynorsk'". Still, I could not use the "new thing", 'nn', since it was 
not backwards compatible and since nearly zero Norwegain OS X 10.5 users 
would then see it. So, the new feature - nn - kind of disabled itself.

This is an perfect example of why the macro langauge tag must keep its 
general name, even if you - as operative system vendor -  choose to use 
it only for one spesific language.  You are allowed to use the general 
tag for only spesific language, but you do not have the right to tell 
others that it means anything else than what it does mean. Users know 
the difference between Bokmål and Norwegian. The common 'no' tag is 
there for a reason.

Again, Apple uses folders tagged as 'en', filled with US English, but 
still diplayed as "English". It would have been wrong if they said that 
'en' means "US English". (I guess several users would not have used that 
localisation if Apple openly said "US English".)

So, basically, the day Apple introduces support for Cantonese, then I 
assume the general 'zh' tag will be quite important. It is then that it 
is important that its proper meaning as a general tag covering all 
chinese, is kept.

> extremely painful to implement and return *no* extra value for our users.

One important *new* thing (for the developer) is that 'yue' can fall 
back to 'zh' or 'cmn' when needed. This is where the Nynorsk 
implementations failed in OS X.

And also, if you want your new, Cantonese translated app to be of use in 
old OS versions not supporting Cantonese (such as with my iCab 
translation in Nynorsk), you must have the general term available for 
that as well.

> 'zh' and 'cmn'. Those environments are going to probably be rather rare.

It, of course, depend on how things will be implemented. If 'yue' and 
'cmn' would live completely independent lives in a given OS, then both 
'yue' and 'cmn' could - independently - be associatede with the 'zh'.

But if 'yue' support will be implemented the way 'nn' support was (or 
should have been) enabled in Mac OS X, then this ain't so: Currently, 
there is no Nynorsk translation of Mac OS X. So, Nynorsk users must - 
and expect - to get the Bokmål interface when they open an applications. 
(Windows works that way.) And it is in this very thing that Mac OS X 
currently fails: Enabling (only) Nynorsk, I only get English interface 
for all applications. (Using the Nynorsk "interface" still has a 
purpose, e.g. for spell checking and other things, such as 
Accept-Language in Safari.)
-- 
leif halvard silli

---

From: Peter Constable <[email-masked]>
Date: Fri, 2 May 2008 17:54:01 -0700

From: [email-masked] [mailto:[email-masked]] On Behalf Of Mark Davis

> I stand by what I said: it has long been used to mean just Mandarin.

Actually, I think the truth is slightly different from both, though closer to Karen: it has long been used to mean just "Chinese", with content that most of the time happened to be Mandarin.

> If you make a request using 'zh' to products from Google, Microsoft,
> Apple, Sun, and so on, in practice that does not mean:
> Give me some kind of Chinese, I don't care whether it is Mandarin or
> Cantonese or Hakka.
> Instead, in practice, it means:
> Give me Mandarin.

Well, it might not mean "Give me _some kind_ of Chinese," but I think it *does* mean "Give me Chinese," period, because zh was created with exactly that vague meaning -- when all those implementations that use it were created, nobody was thinking that the Mandarin/Cantonese distinction might ever be relevant for those applications.

(Hence this part of the definition of "macrolanguage": it's treated in some applications as though it were a single language.)

Peter

---

From: [email-masked]
Date: Fri, 2 May 2008 18:26:39 -0700

As much as we'd like to rewrite history, it's a semantic change.  There is 
a whole list of grandfathered tags such as zh-min, zh-yue, zh-hakka, and 
others that will make no sense after this new semantic goes into effect 
with RFC 4646bis.  You've not only changed the semantic of "zh",  you've 
made all of those tags senseless as well.  The very existence of those 
tags that identify other Chinese languages indicates to me that the 
registrants of those tags, myself included, did NOT interpret "zh" to mean 
Mandarin. 

When I look at this page, I can see that John Cowan, Clay Compton, Kai-hus 
Tai, and I all registered tags that assume a more generic definition of 
"zh". This seems to contradict your statement that "zh" has always meant 
Mandarin in practice. Do you assume that these tag registrants registered 
these tags and then never put them to use? 

        http://www.iana.org/assignments/language-tags

Looking at the registrations you made, it appears possible that you were 
operating under this assumption, but I'm not so sure this is the case with 
every user.

Regards,

Karen Broome

[email-masked] wrote on 05/02/2008 03:19:31 PM:

yue-HK)

> Personally, if someone in my company came to me and said "what 
> should I use", I would unconditionally say "zh". If someone were to 
> hand us 'cmn-CN', I'd say, map that first thing to "zh-CN". Backward
> compatibility dominates.
> 
> The important thing for us is to get the *new* capability offered by
> "yue", etc., not to make a distinction between 'zh' and 'cmn' that 
> would be extremely painful to implement and return no extra value 
> for our users.
> 
> backwards compatibility issues, it'd be fine to make a distinction 
> between 'zh' and 'cmn'. Those environments are going to probably be 
> rather rare.
> 
> 
> 
> Regards,
> 
> Karen Broome
> 
> 
> 
> > > I assume that when you say "3066bis" you actually mean "4646bis", 
and
> > are
> > > asking about the meaning of the new draft rather than of the 
existing
> > > Not really.  The core idea is that the "zh" tag is vague/inclusive 
and
> > > Mandarin, and almost all written Chinese is in fact Mandarin, so 
there

> 
> 
> 
> -- 
> Mark _______________________________________________
> Ltru mailing list
> [email-masked]

---

From: "Andrew Cunningham" <[email-masked]>
Date: Sat, 3 May 2008 11:31:58 +1000

> In practice, 'zh' and 'cmn' are best treated as synonyms

Personally, I'd assume that all content marked up as "zh" is Chinese
of some variety. Just because a lot of content is assumed to be in
Mandarin, or even if most content is in Mandarin, some content will
not be in Mandarin.

The issue is that not all content marked up as "zh" is actually
Mandarin. So applications shouldn't make assumptions., and just treat
it as Chinese of some variety.

Considering Chinese content produced locally in Australia, I'd want to
verify each single document before assuming it was a particular
Chinese language. I wouldn't trust a language tag "zh". Its too
imprecise for my needs.

Andrew

-- 
Andrew Cunningham
Vicnet Research and Development Coordinator
State Library of Victoria
Australia

[email-masked]
[email-masked]

---

From: "Andrew Cunningham" <[email-masked]>
Date: Sat, 3 May 2008 11:42:57 +1000

if you look at most web based applications, and most word processing,
most users don't even think of language.

zh-TW, zg-HK, zh-Hant to most uses means give me Traditional Chinese characters

zh-CN, zh-Hans means give me Simplified Chinese characters.

and in Australia, "zh" could be anything, considering only a
proportion of the Chinese speaking population here understand
Mandarin.

most web developers, are ignorant of language issues, they'll just
treat it as generic Chinese.

Microsoft, Apple, etc may treat "zh" as mandarin, not sure about that,
since support for Zh-HK would mean that applications would need ot be
able to support Cantonese as well as Mandarin.

Some users would be right to assumeand expect that any application
that supported zh-CN would be able to handle all Chinese languages
using hanzi.

Andrew

2008/5/3 Mark Davis <[email-masked]>:

> I stand by what I said: it has long been used to mean just Mandarin. If you
> make a request using 'zh' to products from Google, Microsoft, Apple, Sun,
> and so on, in practice that does not mean:
>

[email-masked]
[email-masked]

---

From: Leif Halvard Silli <[email-masked]>
Date: Sat, 03 May 2008 04:06:20 +0200

> Microsoft, Apple, etc may treat "zh" as mandarin, not sure about that,

Correct to be unsure: OS X 10.4 and 10.5 uses the codes zh_CN and zh_TW, 
and in the interface (in the tooltips - because I can't read "Chinese", 
they call it "Chinese (Traditional)" and "Chinese (Simplified)". In 
other words, they do not call it Mandarin.

OS X treats 'en' as US English in many ways. But it is still only 
referred to as English - which is correct. Putting English as first 
langauge of OS X means that Safari sends out a request to websites for 
the 'en' language tag.
-- 
leif halvard silli
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 2 May 2008 19:47:20 -0700

I'm obviously not making myself clear, because you are leaping to
conclusions that are not founded in what I'm saying.

*I am NOT saying that zh is the same as cmn.* I'm not changing the semantics
of 'zh' at all*.
What I'm saying is: *although they are semantically distinct,* my advice to
most users would be that in practice for backwards compatibility in programs
they are best off treated as synonyms.

Maybe it would be clearer with another example. I might say, for practical
purposes on input in a particular program the value $99.999999 is best
handled as though it were synonymous with $100 (eg rounding). That doesn't
mean I think that these two numbers are synonyms -- far from it!

Mark

* (Although, frankly, the few words of identification given in ISO 639-1 and
-2 are often very fuzzy, as we've seen time and time again. Many people, for
example, consider Swiss German a kind of German. And "German" was all it
said for 'de' in ISO 639-1. So it would be perfectly natural for people to
think that Swiss German would be de-CH, just like Austrian German is de-AT.
)

---

From: Leif Halvard Silli <[email-masked]>
Date: Sat, 03 May 2008 08:05:58 +0200

> they are best off treated as synonyms.

I do understand how you can offer users such an advice. Information, 
yes. Advice, no. I've never seen an interface where I can opt to select 
two languages as synonyms. In an applicaiton, for example a browser, one 
tag will come before another - hierarchy can't be avoided.

For the majority language, it is very safe to place the most spesific 
tag first. Thus the tag that means Mandarin should be first.

The more spesific you can be, the more targeted can you be in the next 
round.
-- 
leif halvard silli
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: Leif Halvard Silli <[email-masked]>
Date: Sat, 03 May 2008 08:42:21 +0200

Leif Halvard Silli 2008-05-03 08.05:
> I do understand [...]

Sorry. Do NOT understand ...
-- 
leif halvard silli
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: John Cowan <[email-masked]>
Date: Sun, 4 May 2008 15:42:37 -0400

> assume that it, in practise, *excludes* texts in Cantonese and Hakka? 

In practice, it is probably incapable of distinguishing between them
reliably, and even if it can, it probably costs too much.

-- 
Kill Gorgun!  Kill orc-folk!            John Cowan
No other words please Wild Men.         [email-masked]
Drive away bad air and darkness         http://www.ccil.org/~cowan
with bright iron!   --Ghan-buri-Ghan    http://www.ccil.org/~cowan
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: Peter Constable <[email-masked]>
Date: Tue, 6 May 2008 06:10:38 -0700

> From: Shawn Steele
> Sent: Friday, May 02, 2008 1:08 PM

> For Arabic I don't think the problem has nearly the same scale and I
> don't think content taggers or developers would have the same problem.

True... as long as there aren't specific varieties other than the standard variety that are important for tagging. If Karen comes along a few years from now saying, "We have movies dubbed into *Egyptian* spoken Arabic," we may have similar troubles with when to use "ar" versus "arb" for the standard variety.

Peter

---

From: "Andrew Cunningham" <[email-masked]>
Date: Tue, 6 May 2008 23:55:55 +1000

I have a nagging thought at the back of my mind while reading the
whole cmn == zh debate.

I can accept how zh_CN would mainly be Mandarin

I can accept that mostly zh-TW and zh-SG may be equated with Mandarin

But moving to other countries where the majority of Chinese are not
Mandarin speakers, equating "zh" with "cmn" seems to be much more
tenuous.

I'd be more inclined to say zh-CN == cmn, maybe zh-SG, zh-TW == cmn

but feel very uncomfortable asserting zh == cmn

[email-masked]
[email-masked]