Subject: [Ltru] Re: New scripts

From: "Doug Ewell" <[email-masked]>
Date: Tue, 27 Nov 2007 08:15:44 -0800

Martin Duerst <duerst at it dot aoyama dot ac dot jp> wrote:

>> BTW, I wonder how many people would have predicted, years ago when 
>> the basic rules for script subtags were formulated, that ISO 15924 
>> would eventually add code elements for "Mathematical notation" and 
>> "Symbols." Section 3.4 says they must be added to the Registry, no 
>> cherry-picking allowed, and that seems fine in principle, but these 
>> two are at least as irrelevant to tagging of natural-language text as 
>> are the region subtags 'AQ' and 'BV', or the language subtags 'mis' 
>> and 'und'.
>
> Well, there are a lot of people, especially here in Japan, using 
> little symbols or images in their emails/sms. If you want to tag 
> these, a script subtag for symbols may come in handy :-).

Is anyone really going to tag them separately from the surrounding text?

Anyway, after my rant about EU I'm certainly not going to argue in favor 
of changing the rules to prohibit these things.  But they will look 
somewhat ridiculous to LSR newcomers.

--
Doug Ewell  *  Fullerton, California, USA  *  RFC 4645  *  UTN #14
http://home.roadrunner.com/~dewell
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages  ˆ

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Frank Ellermann" <[email-masked]>
Date: Tue, 27 Nov 2007 17:48:33 +0100

Doug Ewell wrote:

> after my rant about EU I'm certainly not going to argue in favor 
> of changing the rules to prohibit these things.  But they will
> look somewhat ridiculous to LSR newcomers.

For Zsym and similar oddities it's not the fault of BCP 47, for
fonipa + fonupa it's less clear why there's no Lati and Latu if
Zmth and Latf are okay.  

For EU it would be the fault of the IETF, and EU isn't the worst
case in this zoo.  EA is in a certain sense a disputed territory,
IIRC DG is a subset of IO and was (or is) a court case in the UK
(aka GB on this list), IC might be another can of worms, and a
fourth fantasy island was exceptionally reserved by the UPU, do
they need language tags for their stamps ?

If Spain wants EA and IC as country codes let them, Debbie has
demonstrated that it is possible if necessary.  Besides EU does
not really help CLDR, they could use it anyway if they want it,
and even if they replace QU by EU they still have QO.  

Last but not least EU isn't stable, it's also not sure that the
EU will only grow, maybe some members like UK (GB) + DK + SE 
will split and start their own alliance.

 Frank

---

From: Peter Constable <[email-masked]>
Date: Tue, 27 Nov 2007 09:13:32 -0800

> From: Frank Ellermann [mailto:[email-masked]]

> will split and start their own alliance.

That does not rule out a possible scenario in which (e.g.) the EU defines spelling conventions for English for use in EU documents that do not exactly match usual conventions in the UK. Changes in the composition of the EU in either direction would not likely affect such spelling conventions.

If someone doesn't find "EU" useful for tagging documents, they don't have to use it. The inclusion of a subtag in the LSR is not of itself a recommendation that the subtag be used in any particular way; it merely creates the opportunity for use.

Peter

---

From: "Mark Davis" <[email-masked]>
Date: Tue, 27 Nov 2007 09:20:14 -0800

To repeat what I said earlier:

I firmly agree. BCP 47 is not only the source of language tags; it is
*the *reference
for stabilized, maintainable codes for languages, scripts, and regions.
Someone citing "regions as defined by BCP 47", even when not used in
language codes, has a stable, well-defined, maintainable code that also
encompasses UN M.49 regions. And this is a good thing, since it promotes the
use *and interworkability *with BCP 47, and software internationalization
overall.

By excluding edge cases like EU, we just make it harder for people to use
BCP 47 as is, and force the proliferation of private use codes (or worse
yet, non-BCP47 codes).

By being pragmatic, and handling just a few key cases that are important for
implementations, we make it far easier for people to adopt and use BCP 47.
Adding "EU" to 4646bis would be one of those key cases. It doesn't hurt to
have EU as a code in BCP 47, no more than it hurts to have BV as a code. And
the EU is a darn'd sight more important than a slew of the region codes we
already have -- whether in language codes or in interworking with them!

I agree with Addison's proposed text.

BCP 47 has many subtags that may not be of particularly high value to any
particular implementor in combination or even in isolation, such as BV or
AQ. The EU as a region code is far more useful than a great many other
region codes even purely in terms of languages; en-EU is more useful than
ja-AQ.

If we look at the language tags that are permitted, we have currently on the
order of
language:    490
script:    121
region:    281
not even counting variants, deprecated, grandfathered, and private use
codes.

That means that one can have 34,000 different tags for Japanese, for
example, most of which will never be necessary. A key feature of the
standard is that we give people the tools they need to make the distinctions
they need; it is up to them to avoid unnecessary distinctions for the tasks
at hand.

While we (speaking in any of my professional capacities) have no real need
for the new script codes, the meaning is reasonably clear, and I can well
understand that others may have some requirements for them: we should not
stand in the way. As Peter said earlier about EU:

Well, I'm not opposed to questions of what makes sense for an ecosystem of
closely related tagging schemes. Nobody would be served in the long run if
we had to have completely separate schemes and infrastructures for things
that overlap 95%, so it may make sense to allow a 2% compromise of scope in
one place or another to provide a single, coherent ecosystem of related
schemes.

Mark

On Nov 27, 2007 8:15 AM, Doug Ewell <[email-masked]> wrote:

> http://home.roadrunner.com/~dewell <http://home.roadrunner.com/%7Edewell>

-- 
Mark

---

From: "Frank Ellermann" <[email-masked]>
Date: Tue, 27 Nov 2007 19:09:04 +0100

Peter Constable wrote:
 
> That does not rule out a possible scenario in which (e.g.) the EU
> defines spelling conventions for English for use in EU documents
> that do not exactly match usual conventions in the UK.

Yes, but spelling conventions are better covered by a variant than
a region.  The tag en-EU is not necessarily what you describe, an
"official" EU spelling convention.
 
> If someone doesn't find "EU" useful for tagging documents, they
> don't have to use it.  The inclusion of a subtag in the LSR is not
> of itself a recommendation that the subtag be used in any 
> particular way; it merely creates the opportunity for use.

"Not useful" isn't the same as "potentially harmful", all here wen't
to great lengths to guarantee stability especially for region codes
after the various CS disasters.  If stability isn't relevant anymore
we can trim 4646 significantly, picking "whatever ISO 3166 currently
says", as it was the case for RFC 1766 + 3066.  No weird rules about
UN codes as fallbacks, just a short note "you get what you paid for
with region codes, there be dragons, they might bite after 50 years".

 Frank

---

From: Shawn Steele <[email-masked]>
Date: Tue, 27 Nov 2007 10:31:21 -0800

"...there be dragons, they might bite after 50 years".

I'm sure that regardless of the specifics, the dragons are going to bite people much sooner than that :)  There's too much instability in the geopolitical world.

- Shawn

---

From: "Mark Davis" <[email-masked]>
Date: Tue, 27 Nov 2007 10:54:07 -0800

I agree. We have and make no guarantees regarding the stability of
geographic areas associated with the territory codes.

If the Pacific states of the US secede and then join Canada, it would make
no difference in the codes we use, although there would be a major change in
territory. Any number of more realistic territorial changes could happen
over time; the EU might change somewhat more rapidly than most, but who
knows what the future will bring.

The reason that the political divisions are often useful in discriminating
among language variants is that there are often common linguistic features
that are associated with those political entities. And insofar as languages
form the basis for locales -- a major usage -- they are, of course also
important.

And even if we had some (bizarre) scheme for precise geographic location
(en-l-p3706-n9568-p3545-n ... for the language spoken within a polygon
starting at latitude 37.06 and longitude -95.68 and so on ...), we have no
assurances that people don't move over time, or that language variants don't
change or develop over time.

We live in an imperfect, changing world, and we have to be pragmatic in
considering what tools are useful, even when there is some degree of
imprecision.

Mark

On Nov 27, 2007 10:31 AM, Shawn Steele <[email-masked]> wrote:

> I'm sure that regardless of the specifics, the dragons are going to bite
> people much sooner than that :)  There's too much instability in the
> geopolitical world.
>
> - Shawn
>

---

From: Peter Constable <[email-masked]>
Date: Tue, 27 Nov 2007 11:17:19 -0800

> after the various CS disasters.

There's a categorical difference between stability of the semantic denotation of an ID like "CS", which are things like nation states, and the stability of the geographic region under the jurisdiction of entities such as nation states. If the US were to cede Maine to Canada, that would not require us to stop using "en-US" or "en-CA" of stability concerns. If "EU" were to mean "European Union" today but "Eugoslavia" twenty years from now, that would be a stability problem; but countries joining or leaving the European Union does not alter the identity of an entity known as the European Union, and language usage that is likely to correspond to it.

Peter

---

From: John Cowan <[email-masked]>
Date: Tue, 27 Nov 2007 14:34:29 -0500

Peter Constable scripsit:

> There's a categorical difference between stability of the semantic
> denotation of an ID like "CS", which are things like nation states,
> and the stability of the geographic region under the jurisdiction
> of entities such as nation states. If the US were to cede Maine to
> Canada, that would not require us to stop using "en-US" or "en-CA"
> of stability concerns. 

In the case of more radical realignments such as contemplated at
http://www.tbray.org/ongoing/When/200x/2004/11/04/NewNorthAmerica ,
M.49 can and does issue new numeric codes, and the 3166/MA can
change things accordingly.

> but countries joining or leaving the European Union does not alter
> the identity of an entity known as the European Union, and language
> usage that is likely to correspond to it.

*What* language usage that is likely to correspond to it?  That's
the issue.

-- 
At the end of the Metatarsal Age, the dinosaurs     John Cowan
abruptly vanished. The theory that a single         [email-masked]
catastrophic event may have been responsible        http://www.ccil.org/~cowan
has been strengthened by the recent discovery of
a worldwide layer of whipped cream marking the
Creosote-Tutelary boundary.             --Science Made Stupid

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 27 Nov 2007 11:52:37 -0800

Hi -

As a technical contributor...

> From: "John Cowan" <[email-masked]>
> To: "Peter Constable" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Tuesday, November 27, 2007 11:34 AM
> Subject: Re: [Ltru] Re: New scripts
...
> *What* language usage that is likely to correspond to it?  That's
...

The only example I recall was, for lack of a better word,
"bureaucratese," meaning the peculiar varieties of language
preferred by the governmental and quasi-governmental
officials associated with the EU.  I have heard it facetiously
called "mandarin English".

As a co-chair -

I don't know whether uch varieties really exist, or are
merely urban legends.  *That* would be a question for 
[email-masked], rather than this working group.

Our job here is merely to decide whether this sort of thing
is possible, and consequently should be supported by our
procedures.  Of course, someday someone might decide
that EU really should have a country code, and solve the
problem for us. It's just a question of which will happen first.

Randy

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 27 Nov 2007 22:52:13 +0100

On Tue, Nov 27, 2007 at 10:54:07AM -0800,
 Mark Davis <[email-masked]> wrote 
 a message of 87 lines which said:

> The reason that the political divisions are often useful in
> discriminating among language variants is that there are often
> common linguistic features that are associated with those political
> entities.

I suspect that it is true only for Europe. In Africa, at least,
countries are quite meaningless, linguistically speaking. It seems too
easy to find a lot of counter-examples where the country does not give
any indication on the variant spoken or written (what is es-US, for
instance?)

The idea to have countries in language tags (not in locales, where
things like currencies and holidays obviously depend on the country)
is questionable and seems to come from a traditional "Nation-State"
mind.

(Of course, in the name of stability, I do not suggest that we drop
them, just that we keep in mind that they are most often not a good
discriminant for languages.)

---

From: "Mark Davis" <[email-masked]>
Date: Tue, 27 Nov 2007 14:01:06 -0800

> ... we keep in mind that they are most often not a good

no disagreement.

---

From: John Cowan <[email-masked]>
Date: Tue, 27 Nov 2007 17:22:48 -0500

> is questionable and seems to come from a traditional "Nation-State"

It is indeed questionable.  However, we get them for free, and it
is often the case that orthographies (for languages less autonomous
than English) are set by national bodies of one sort or another,
or if not, then by national conventions like those which
distinguish en-US from en-GB from en-CA from en-AU ....

-- 
John Cowan  [email-masked]  http://ccil.org/~cowan
If he has seen farther than others,
        it is because he is standing on a stack of dwarves.
                --Mike Champion, describing Tim Berners-Lee (adapted)

---

From: "Frank Ellermann" <[email-masked]>
Date: Wed, 28 Nov 2007 15:47:23 +0100

Stephane Bortzmeyer wrote:
 
> Of course, in the name of stability, I do not suggest that we drop
> discriminant for languages.

"Keep in mind" could be a "SHOULD" in 4646bis, that would also remove
a reason for the the sudden rush to add codes of potentially disputed
territories such as EA to the *language* subtag registry.  Likely (?)
CLDR also doesn't need EA and similar codes.

 Frank

---

From: "Frank Ellermann" <[email-masked]>
Date: Wed, 28 Nov 2007 15:59:18 +0100

> called "mandarin English".

IIRC I used "eurocrat" as *example* here or on the review
list months ago, nobody ever tried to register a variant
for this.

> procedures.

Possible, supported by the 4646 rules (variant or official
3166 registration), no need to change this part of BCP 47.

Especially not disguised by a simultaneous introduction of 
exceptionally reserved codes, excluding the only arguably
relevant code UK.

 Frank