Subject: RE: [Ltru] Private Use Tags

From: Misha Wolf <[email-masked]>
Date: Thu, 07 Jul 2005 16:48:28 +0100

Dylan N. Pierce wrote:

> Leakage is going to happen. Sun Microsystems, for example, finds 
> the third subtag to be an excellent place to include currency 
> information ("es-ES-EURO"). Every copy of the JDK they've shipped 
> since 1.2 has gone out that way.

This sounds like a locale ID, not a language ID.  Does it use "-" 
separators or "_"?

Misha

-----------------------------------------------------------------
        Visit our Internet site at http://www.reuters.com

To find out more about Reuters Products and Services visit http://www.reuters.com/productinfo 

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Dylan N. Pierce" <[email-masked]>
Date: Thu, 07 Jul 2005 11:11:42 -0500

Misha,

It is, in fact, the locale ID. But it would be disingenuous to present 
that distinction as a significant difference. Effectively, the locale ID 
/is/ a language ID "plus." To whit, an "extended" or "modified" language 
tag, used for Sun's own purposes... which is exactly what we're talking 
about here.

According to Sun's own documentation, "The language argument is a valid 
ISO Language Code. These codes are the lower-case, two-letter codes as 
defined by ISO-639....  The country argument is a valid ISO Country 
Code. These codes are the upper-case, two-letter codes as defined by 
ISO-3166." And its purpose is so that applications can decide in what 
language to serve the documents (in this case, UI output). In other 
words, the difference between what they're doing and what we're doing 
is, at best, cosmetic; I imagine the Java programmers did it this way 
for the same reasons I have to take shortcuts in my own code: because 
existing language tags alone aren't sufficient for these purposes.

If this new draft takes these issues into consideration, we might be 
able to all get back on the same page again.

Dylan

---

From: "John.Cowan" <[email-masked]>
Date: Thu, 7 Jul 2005 14:52:27 -0400

Dylan N. Pierce scripsit:

> is, at best, cosmetic; 

In fact not; the difference is deep and conceptual.  The country subtag
in a locale ID serves an entirely different purpose from the one in a
language tag.  Rather than meaning "The variant of the language commonly
used in country X", it means "Using the various notational and cultural
conventions of country X".

For example, the Mohawk language (ISO 639-2 code 'moh') is spoken on
both sides of the U.S.-Canadian border, but does not come in national
variants "moh-US" and "moh-CA"; though well-formed and valid per RFC
3066, these tags are not useful.  However, for someone using a computer
in Mohawk, the locale IDs "moh_US" and "moh_CA" are eminently useful:
the first signals a desire for U.S. dollars and Fred Flintstone units,
the second a desire for Canadian dollars and metric units.

Similarly, "en-DE" does not make much sense, as there is no German
national variant of the English language (jokes about "Ve haf
vays!" aside), but "en_DE" represents using the English language in a
German national context, with euros and metric units and German-style
postal addresses.

Indeed, in principle there is no reason why a locale ID like "en-us_DE"
can't be useful:  this says that the context is still euros, etc.,
but the dictionary now contains specifically American English.

-- 
Newbies always ask:                             John Cowan
  "Elements or attributes?                      http://www.ccil.org/~cowan
Which will serve me best?"                      http://www.reutershealth.com
  Those who know roar like lions;               [email-masked]
  Wise hackers smile like tigers.                   --a tanka, or extended haiku

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 7 Jul 2005 17:24:31 -0700

Hi -

> From: "John.Cowan" <[email-masked]>
> To: "Dylan N. Pierce" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Thursday, July 07, 2005 11:52 AM
> Subject: Re: [Ltru] Private Use Tags
...

This discussion of how, despite surface similarities, language
tags and locale identifiers are very different things, though
educational, is diverting us from a much more important question
that Dylan raised when he wrote:

| But given what you've explained to me, I declare very firmly: we should
| stake out the "policy question" explicitly. Either we /do/ want to
| include private use tags, at which point we should set up a mechanism to
| ensure that the private use tags themselves do (as you put it) "promote
| open access, promote consistent tagging and promote interoperability"
| (by handling leakage and providing a way to know which private tag is
| which)... or we do /not/ really want to encourage private use tags,
| which we could do more consistently by simply saying, "These tags cover
| language, dialect and script, and nothing more. Anything over and above
| this should be handled outside the tag," and eliminate private use tags
| completely.

So, unless there is a specific proposal for text that will further clarify that
language tags should not be confused with locales, let's get back to
Dylan's question so we can wrap things up.

Randy, ltru co-chair

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 7 Jul 2005 21:29:01 -0700

I do not think that this line is at all as clear as you would think. The
notion of a locale is very fuzzy: most of the items that people associate
with it, other than language, are merely guesses. The currency of a
transaction is often completely independent of the user's location; I could
be buying something on the web and pay euros, or pounds, or dollars, no
matter where I live. The same goes for timezones, even units. (We all
remember the rocket that failed because two people in the same country were
interpreting a number as being in different units). It is a fairly rough
approximation to assume the values of these from seeing en_UK (or en-UK). Of
course, that may be all that the information that someone has, but you can't
read into it more than a good guess. And other areas get fuzzy as well. Is
sort order a language or a locale issue?

The language is clearly a core part of what people mean by 'locale', and
anything beyond that is pretty speculative. A real locale id shades off into
something more like a bundle of preferences.

But this is all pretty off-topic; I really hope we can focus on just the
items we need to finish off to get this thing to last call.

‎Mark

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 7 Jul 2005 21:59:23 -0700

Hi -

PLEASE take off-topic discussions elsewhere.  As you yourself write:
"I really hope we can focus on just the items we need to finish off
to get this thing to last call."

Randy, ltru co-chair

----- Original Message ----- 
From: "Mark Davis" <[email-masked]>
To: "John.Cowan" <[email-masked]>; "Dylan N. Pierce" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, July 07, 2005 9:29 PM
Subject: Re: [Ltru] Private Use Tags

?Mark

---

From: "Dylan N. Pierce" <[email-masked]>
Date: Fri, 08 Jul 2005 00:46:51 -0500

[Randy Presuhn writes:]
> Dylan's question so we can wrap things up.

To keep us on topic:

Given the suggestion that originally brought me here, and the issues 
that unfolded from it, I'd have to agree that Randy is correct, and this 
is what it boils down to. We're trying to nail down a specification here 
that, in addition to fulfilling the mandate given, will also have the 
added benefit of actually being /used/ by the people who need it.

I'm sympathetic to a mandate, and I certainly do not want to derail the 
move toward final call (remember, I /want/ this specification to pass). 
But this specification is either doing, "Just this," or it's doing, 
"This, and more." And I'm happy either way...

But if it's doing "Just this," then let's put our money where our mouths 
are and say, "Just this." We don't need private use tags at all if we 
are in agreement that corollary information, no matter how closely 
related, should be handled outside the tag and, as Randy notes, we 
shouldn't encourage the twisting of the tag by saying, "Here's your 
scribble area." Cripes, all of XML is the scribble area; we can afford 
to keep the tag clean.

On the other hand, if we're doing "This, and more," then I would believe 
we have a responsibility to ensure that the "more" proceeds in an 
orderly fashion--to be read, "open, consistent and interoperable."

As I said, I'm happy either way, but I don't think it's a non-issue. I 
suppose part of the question becomes, "How much work do we want to have 
to do /after/ the specification is in place?" If we eliminate the 
scribble area, the answer is none. If we implement a namespace, the 
answer is, "It becomes the registrar's problem." And if we leave a 
wide-open private use area, the answer becomes, "Well, it depends on 
what people actually do... and how much they step on each other's 
toes... until they come back to us demanding a solution."

Needless to say, I favor one of the first two options.

Dylan

---

From: "Randy Presuhn" <[email-masked]>
Date: Fri, 8 Jul 2005 00:52:54 -0700

Hi -

Note that I've added an issue number(https://rt.psg.com/ user& password ietf)

(as a technical contributor)

> From: "Dylan N. Pierce" <[email-masked]>
> To: "Randy Presuhn" <[email-masked]>; <[email-masked]>
> Sent: Thursday, July 07, 2005 10:46 PM
...
...

Eliminating them completely might be hard for some to swallow. (I'm guessing.
If others can confirm one way or the other, please do so.)

Section 4.5 of
http://www.ietf.org/internet-drafts/draft-ietf-ltru-registry-08.txt
currently says:
   Private-use subtags require private agreement between the parties
   that intend to use or exchange language tags that use them and great
   caution SHOULD be used in employing them in content or protocols
   intended for general use.  Private-use subtags are simply useless for
   information exchange without prior arrangement.

Would a stronger version of this (or the following paragraphs address the
concerns?  If so, proposed text would be appreciated.

Randy

---

From: "Dylan N. Pierce" <[email-masked]>
Date: Fri, 08 Jul 2005 04:10:42 -0500

> concerns?  If so, proposed text would be appreciated.

Another way of looking at the concern. Reading from 4.5:

"For example, the region subtags 'AA', 'ZZ' and in the ranges
'QM'-'QZ' and 'XA'-'XZ' (derived from ISO 3166 private use codes) MAY
be used to form a language tag.  A tag such as "zh-Hans-XQ" conveys a
great deal of public, interchangeable information about the language
material (that it is Chinese in the simplified Chinese script and is
suitable for some geographic region 'XQ').  While the precise
geographic region is not known outside of private agreement, the tag
conveys far more information than an opaque tag such as "x-someLang",
which contains no information about the language subtag or script
subtag outside of the private agreement."

The tag "zh-Hans-XQ" provides enough information that parsing agents 
know what they don't know: the region. The legitimate-but-opaque tag 
"zh-Hans-x-whoKnows" might contain a privately-defined region... but if 
so, why not use the first option? It also might contain something that's 
completely outside the intended scope of the language tag. If we're 
serious about what these tags are meant to represent, why make it 
possible to work against it? This returns to the "scribble space" issue.

I'm not sure a stronger version of the warning itself would clarify 
this. It would have to say something like:

"Private-use subtags require private agreement between the parties
that intend to use or exchange language tags that use them and they
MUST NOT be used in content or protocols intended for general use
unless it can be guaranteed that they will not inadvertently show up
in parsing agents used by other parties with private-use agreements
that coincedentally employ the same alphanumeric sequence."

Which sounds ridiculous. Okay, yes, I intentionally made it sound 
ridiculous, but this is the point: if we're documenting language, 
region, script and variant, it's easy enough to indicate "unknown 
region" when encountering a private-use region subtag. If you encounter 
a pure private-use "x," all you know is that it is either an unknown 
language, script, region or variant that doesn't want to admit it, or an 
unknown something else that's outside the scope of the language tag 
completely.

What does allowing an unknown "something else" gain in clarity or 
interoperability? Returning to Addison's original reply to my first 
proposal (now withdrawn, but for this very reason), if the "something 
else" really is legitimate, then a new RFC can be written to make use of 
the reserved tags. If not, then it's in the wrong place, and if you've 
ever tried to keep an office organized, you know that allowing folks to 
put things in the wrong place is an excellent way to ensure that later 
on, no one knows where to look for them (except, perhaps, by "private 
agreement," which is how I keep from losing my coffee cup).

So I guess at this point, what I am looking at isn't proposed new text, 
or proposed changes in text, but a proposed elimination of text, to 
whit, the text defining the private-use subtag "x". My question: is the 
gain in "swallowing comfort" won by unregulated, unparsable private-use 
tags worth the corresponding loss in clarity and interoperability?

If the answer is an honest resounding "Yes" from everyone else with an 
interest in this issue, I'll concede the point and go away. But if we're 
doing it just because we're just not willing to take that final step of 
saying, "/This/ is what these tags define, and nothing else," because 
we're nervous about the commitment, then I have a feeling we might be 
making a mistake.

Dylan

---

From: r&d afrac <[email-masked]>
Date: Fri, 08 Jul 2005 13:24:48 +0200

At 09:52 08/07/2005, Randy Presuhn wrote:
>Hi -Eliminating them completely might be hard for some to swallow. (I'm 
>guessing.

Let be candid, this is another way to reduce the scope of the Draft without 
telling it plainly. The followin text:

    Private-use subtags require private agreement between the parties
    that intend to use or exchange language tags that use them. Great
    caution SHOULD be used in employing them in content or protocols
    intended for general use. Private-use subtags SHOULD respect the
    general format contraints. Even used in a private context, general
    libraries are legitimate in only using the part of each subtag which
    matches the general ABNF [or in disregarding the subtags wich
    does not match the general ABNF]. Private-use subtags are
    simply useless for information exchange without prior arrangement.

Would be OK for us (even with the [or ...] being used). This provides an 
escape sequence from the Draft. Not the best and most elegant, but 
workable: this is what we will do anyway.

jfc

---

From: r&d afrac <[email-masked]>
Date: Fri, 08 Jul 2005 13:02:24 +0200

>items we need to finish off to get this thing to last call.

Mark,
the whole issue is that a langtag is to identify something. These 
somethings are not defined in your Draft. The Last Call will identify that 
among others, and will fail again. Unless the game consists for you too, in 
accumulating the failed Last Call and draining competences from all over 
the world in a waste of time, please identify:

- each subtag you use
- their correlations into the subtag you propose - which becomes more and 
more different from the lose catch all RFC 3066 lack of definition. RFC 
3066 was general and never said as you keep adding "this of off-topic". All 
what I ask is you say what is on topic.

This at least would help Randy's mails about "stay on topic" clearer.
Or in 2007 we will still be discussing about the 8th Last Call. Am I the 
only one to truely want to see something matching the charter being delivered.

The charter does not say to _narrow_ the scope but to address the problems, 
which are that the scope is not large enough and not structured enough to 
support all the encountered needs.

jfc

>â€ŽMark

---

From: r&d afrac <[email-masked]>
Date: Fri, 08 Jul 2005 13:37:25 +0200

At 11:10 08/07/2005, Dylan N. Pierce wrote:
>Another way of looking at the concern.

is to respect the charter and retrocompatibility with RFC 3066. We 
extensively use and plan to use "x-tags".

The way we see (from our perspectibe) it is that x-tags are the standard 
format:
- "x-"    is the language tag indicating sequence and we experimentaly use 
<x-en> to signify that the text enters/finish English </x-en>
- "xn--" is an extended name (punycoded name) (we experimentaly use <xn--> 
and </xn--> to mean that the text was punny coded and to be restored)
The Draft sequence are default special cases.

Obviously if "x-" retrocompabilty was not assured, we would have no more 
problem in introducing "x-tags" as a separate taging system.
jfc

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 8 Jul 2005 15:32:55 -0700

Private use tags serve a purpose; they do not need to be associated with a
region. I see no reason to make any change, nor do I see a reason to revisit
all of this over and over again.

‎Mark

----- Original Message ----- 
From: "Dylan N. Pierce" <[email-masked]>
To: "Randy Presuhn" <[email-masked]>; <[email-masked]>
Sent: Friday, July 08, 2005 02:10
Subject: Re: [Ltru] [psg.com #1061] eliminate (or proscribe) Private Use
Tags

---

From: "Randy Presuhn" <[email-masked]>
Date: Fri, 8 Jul 2005 15:42:55 -0700

Hi -

> To: "Dylan N. Pierce" <[email-masked]>; "Randy Presuhn" <[email-masked]>; <[email-masked]>
> Sent: Friday, July 08, 2005 3:32 PM
> Subject: Re: [Ltru] [psg.com #1061] eliminate (or proscribe) Private Use Tags
>

...

Rather than rehashing endlessly, please see
http://www1.ietf.org/mail-archive/web/ltru/current/msg02617.html
where I suggested we further narrow the debate in hopes of
getting a resolution.

Randy