Subject: Re: [Ltru] draft updated

From: "Doug Ewell" <[email-masked]>
Date: Wed, 18 Jun 2008 20:35:26 -0600

I am completely fine with Addison's proposed changes to the ABNF, 
despite my earlier objection to splitting the grandfathered and 
redundant tags into separate productions.  The comments accompanying the 
ABNF go a long way toward justifying this approach.

With Kent apparently signing off on the changes as well, and with most 
other issues resolved, I'll be putting together a draft-4645bis-06 in 
the next few days.  If there isn't a response from ISO 639/JAC on the 
"Carib languages" vs. "Galibi Carib" mismatch by then, I'll assume the 
639-2 denotation ("Galibi Carib") takes precedence, based on John 
Cowan's comment from June 8 (msg10936.html) and our general principle of 
preserving the meaning of existing subtags.

--
Doug Ewell  *  Arvada, Colorado, USA  *  RFC 4645  *  UTN #14
http://www.ewellic.org
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages  ˆ

_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: John Cowan <[email-masked]>
Date: Wed, 18 Jun 2008 23:57:12 -0400

Doug Ewell scripsit:

> the next few days.  

Just letting you and the WG know that I continue to *strongly* favor
making "zh-cmn" and "zh-yue" (and the rest) the preferred forms, and
"cmn" and "yue" the pre-deprecated ones.

-- 
"The serene chaos that is Courage, and the phenomenon   [email-masked]
of Unopened Consciousness have been known to the        John Cowan
Great World eons longer than Extaboulism."
"Why is that?" the woman inquired.
"Because I just made that word up", the Master said wisely.
        --Kehlog Albran, The Profit             http://www.ccil.org/~cowan
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 18 Jun 2008 21:14:00 -0700

Hi -

As a technical contributor...

> From: "John Cowan" <[email-masked]>
> To: "Doug Ewell" <[email-masked]>
> Cc: "LTRU Working Group" <[email-masked]>
> Sent: Wednesday, June 18, 2008 8:57 PM
> Subject: Re: [Ltru] draft updated
>
> Doug Ewell scripsit:
> 
...

I'm inclined to agree, but am trying to keep an open mind.

Randy

---

From: "Phillips, Addison" <[email-masked]>
Date: Wed, 18 Jun 2008 22:41:03 -0700

> I'm inclined to agree, but am trying to keep an open mind.

I don't agree.

Notice that this would cause you to gain a subtag. This is the only time this happens (all other mappings cause you to either have the same number of subtags or lose some). Among other things, one might face buffer length issues (some programming languages don't like the buffer to increase in size).

Addison

---

From: John Cowan <[email-masked]>
Date: Thu, 19 Jun 2008 01:54:25 -0400

Phillips, Addison scripsit:

> Notice that this would cause you to gain a subtag. This is the only
> time this happens (all other mappings cause you to either have the
> same number of subtags or lose some). Among other things, one might
> face buffer length issues (some programming languages don't like the
> buffer to increase in size).

Considering that there is still a formidable amount of code that believes
that a language tag is either two or five characters long (xx or xx-YY),
which was not even true in RFC 1766 days, I count such considerations
less than nothing.  We already have perfectly good RFC 4646 methods of
handling such applications.

-- 
John Cowan        http://ccil.org/~cowan   [email-masked]
Lope de Vega: "It wonders me I can speak at all.  Some caitiff rogue did
rudely yerk me on the knob, wherefrom my wits still wander."
An Englishman: "Ay, a filchman to the nab betimes 'll leave a man
crank for a spell." --Harry Turtledove, Ruled Britannia
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: Shawn Steele <[email-masked]>
Date: Thu, 19 Jun 2008 10:25:11 -0700

I'm pretty undecided about which is best, but I don't "buy" the buffer argument.  RFC4646 already allows quite long language tags (once you consider x- and other possibilities).

- Shawn

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Phillips, Addison
Sent: Wednesday, June 18, 2008 10:41 PM
To: Randy Presuhn; LTRU Working Group
Subject: Re: [Ltru] draft updated

I don't agree.

Addison

---

From: "Kent Karlsson" <[email-masked]>
Date: Thu, 19 Jun 2008 19:47:37 +0200

> "cmn" and "yue" the pre-deprecated ones.

This is unrelated to the ABNF discussion of late.

However, I just note that (whichever form is preferred when
extlangs are involved) that (IIUC):

* for 'ar' we are about to add 30 extlangs, *none* of which are
"extlangish" in the current LSR.

* for 'zh' we are about to add 13 extlangs, *only four* of which
are "extlangish" in the current LSR (cmn, gan, wuu, yue).

* for 'sgn' we are about to add 132 extlangs, *none* of which are
"extlangish" in the current LSR.

Was there "rough consensus" to add extlangs for any other primary
languages tags (none of which would be "extlangish" in the current LSR)?

B.t.w., should not the preferred value for the grandfathered
sign language tags maintain the region, like:
 
g.f. tag	cur. p.v.	p.v. keeping region
sgn-BE-FR	sfb	 	sbf-BE (or sgn-sbf-BE)
sgn-BE-NL	vgt		vgt-BE (or sgn-vgt-BE)
sgn-CH-DE	sgg		sgg-CH (or sgn-sgg-CH)

etc. for the other grandfathered sgn-NN tags. (Not that I think
it matters very much, but still.)

	/kent k

---

From: "Phillips, Addison" <[email-masked]>
Date: Thu, 19 Jun 2008 10:56:55 -0700

Actually, the buffer argument was an aside.

My real concerns are:

- adding subtags when canonicalizing is counter-intuitive (I would expect the canonical tag to be more compact).

- some legacy systems (CLDR, for example) have inherent structural/design reliance on the language portion being somewhat atomic

- the Lookup algorithm (i.e. locale-style negotiation) seems to be adversely affected by extlangs. Lookup produces better results when the language and extlang are "atomic". Having a canonical mapping to a primary subtag makes exactly that distinction. If I'm using Lookup, all I have to do is canonicalize the tags and then run lookup normally. The other way I have to change code (and would probably push hard for a 4647bis as a result).

Addison

Addison Phillips
Globalization Architect -- Lab126

Internationalization is not a feature.
It is an architecture.

> From: Shawn Steele [mailto:[email-masked]]
> Sent: Thursday, June 19, 2008 10:25 AM
> To: Phillips, Addison; Randy Presuhn; LTRU Working Group
> Subject: RE: [Ltru] draft updated
>
> I'm pretty undecided about which is best, but I don't "buy" the
> buffer argument.  RFC4646 already allows quite long language tags
> (once you consider x- and other possibilities).
>
> - Shawn
>
> face buffer length issues (some programming languages don't like
> the buffer to increase in size).
>
> Addison

---

From: John Cowan <[email-masked]>
Date: Thu, 19 Jun 2008 14:48:56 -0400

> "extlangish" in the current LSR.

Indeed, which is why I oppose adding 'ar' extlangs (but not strongly).

> are "extlangish" in the current LSR (cmn, gan, wuu, yue).

True, but "zh-hakka", "zh-min-nan", and "zh-xiang" are essentially
similar except for their syntax.  That makes 7 out of 13.

> "extlangish" in the current LSR.

Also true, but 22 existing tags for sign languages already begin
with "sgn-".

> languages tags (none of which would be "extlangish" in the current LSR)?

We also talked about:

Konkani (2 extlangs, 0 in place)
Malay (15 extlangs, 0 in place, 1 deprecated)
Swahili (2 extlangs, 0 in place)
Uzbek (2 extlangs, 0 in place)

All of these are macrolangauges that, like Chinese and Arabic, have
single dominant varieties.

> sgn-CH-DE	sgg		sgg-CH (or sgn-sgg-CH)

sgn-ase (American Sign Language) is spoken in at least 20 countries
beside the U.S.  Treating sgn-ase-US as equivalent to sgn-ase would be
erroneous, as there are undoubtedly local variations.

-- 
Using RELAX NG compact syntax to        John Cowan <[email-masked]>
develop schemas is one of the simple    http://www.ccil.org/~cowan
pleasures in life....
        --Jeni Tennison                 <[email-masked]>
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: John Cowan <[email-masked]>
Date: Thu, 19 Jun 2008 14:53:36 -0400

> - adding subtags when canonicalizing is counter-intuitive (I would expect the canonical tag to be more compact).

Whereas I take "cmn" to be an abbreviation for "zh-cmn" that provides
more convenient processing in certain scenarios.

> - some legacy systems (CLDR, for example) have inherent
> structural/design reliance on the language portion being somewhat atomic

Then let them use the non-canonical form, as Google uses non-canonical
'iw' with no intention of changing.

> - the Lookup algorithm (i.e. locale-style negotiation) seems to be
> adversely affected by extlangs. Lookup produces better results when
> the language and extlang are "atomic". Having a canonical mapping to a
> primary subtag makes exactly that distinction. If I'm using Lookup, all
> I have to do is canonicalize the tags and then run lookup normally. The
> other way I have to change code (and would probably push hard for a
> 4647bis as a result).

You just need something different from canonicalization as currently
written.  I admit I am splitting hairs here.

I continue to think local filtering per RFC 2616 the most important use
case, more important than either mass filtering a la search engine or
resource lookup.

-- 
Where the wombat has walked,            John Cowan <[email-masked]>
it will inevitably walk again.          http://www.ccil.org/~cowan
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Kent Karlsson" <[email-masked]>
Date: Thu, 19 Jun 2008 21:16:28 +0200

> erroneous, as there are undoubtedly local variations.

But that wasn't my (semi)suggestion.

My (semi)suggestion was that sgn-US be preferred-value-mapped to
ase-US rather than to just ase (or sgn-ase) (and similarly for
the other sgn- grandfathered tags).

I.e. to keep the *already existing* region subtag. NOT that
sgn-ase-US would be "equivalent" to sgn-ase (or ase). I certainly
don't want to rule out ase-CA (sgn-ase-CA) and others. Just that
the grandfathered tag maybe should *keep* the region part in the
preferred value *the grandfathered sgn- tags map to*.

And... Hmm, sign languages aren't really spoken, they are signed...
But they can be written using mechanisms such as Sutton Sign
Writing (http://www.signwriting.org/).

	/kent k

---

From: John Cowan <[email-masked]>
Date: Thu, 19 Jun 2008 15:29:00 -0400

Kent Karlsson scripsit:

> Just that

For what purpose, given that the 'US' in the grandfathered form
has a meaning different from any 'US' in a productive form?

> Writing (http://www.signwriting.org/).

Quite so.

-- 
John Cowan  [email-masked]   http://ccil.org/~cowan
Promises become binding when there is a meeting of the minds and consideration
is exchanged. So it was at King's Bench in common law England; so it was
under the common law in the American colonies; so it was through more than
two centuries of jurisprudence in this country; and so it is today.
       --Specht v. Netscape
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Phillips, Addison" <[email-masked]>
Date: Thu, 19 Jun 2008 12:29:13 -0700

> Then let them use the non-canonical form, as Google uses non-
> canonical 'iw' with no intention of changing.

Architectural alignment is typically better, when it can be achieved.

> You just need something different from canonicalization as
> currently written.  I admit I am splitting hairs here.

Yes, but it is extremely convenient (not to mention somewhat compelling) when the existing algorithm works out-of-the-box one way but has to be modified the other.

Ultimately, this all circles back to the Ur-argument: do we want to say "tag anything Chinese as zh-something" or say "avoid tagging things zh-something".

> case, more important than either mass filtering a la search engine
> or resource lookup.

You have your applications and I have mine. ;-)

Local filtering is generally more important. It is, or will be, more common an operation to apply processing, styling, selection, etc. to items and text based on language than it is to select or filter from a collection of documents.

However, RFC 2616 lays out a case for resource lookup, as status code 300 just isn't that desirable. For that matter, 2616 dips its toe into Lookup when it says:

      Note: When making the choice of linguistic preference available to
      the user, we remind implementors of  the fact that users are not
      familiar with the details of language matching as described above,
      and should provide appropriate guidance. As an example, users
      might assume that on selecting "en-gb", they will be served any
      kind of English document if British English is not available. A
      user agent might suggest in such a case to add "en" to get the
      best matching behavior.

Best Regards,

Addison

Addison Phillips

---

From: "Kent Karlsson" <[email-masked]>
Date: Thu, 19 Jun 2008 21:48:39 +0200

> has a meaning different from any 'US' in a productive form?

To avoid the apparent region information loss, if nothing else.

	/kent k

---

From: John Cowan <[email-masked]>
Date: Thu, 19 Jun 2008 16:03:53 -0400

> To avoid the apparent region information loss, if nothing else.

Ah, but there is no actual (as opposed to apparent) region information
in the grandfathered tag "sgn-US", which encodes only a language --
one in use in the U.S., Canada, Guatemala, and various African nations
among others.  That being the case, it would be improper to supply region
information when deprecating it.

-- 
You are a child of the universe no less         John Cowan
than the trees and all other acyclic            http://www.ccil.org/~cowan
graphs; you have a right to be here.            [email-masked]
  --DeXiderata by Sean McGrath
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: Peter Constable <[email-masked]>
Date: Thu, 19 Jun 2008 21:53:32 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Kent Karlsson

> To avoid the apparent region information loss, if nothing else.

But it really is only *apparent* region information loss. To match the denotation exactly, the region ID should *not* be included.

Peter

---

From: Martin Duerst <[email-masked]>
Date: Fri, 20 Jun 2008 16:00:32 +0900

>"cmn" and "yue" the pre-deprecated ones.

[technical hat on] Seconded.       Regards,   Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]