Subject: [Ltru] Private Use Tags

From: "Dylan N. Pierce" <[email-masked]>
Date: Tue, 05 Jul 2005 17:04:04 -0500

(This is a re-send of an e-mail I originally sent to the authors of a 
previous draft; I have since been educated as to the proper way to comment.)

Dear Mr. Phillips and Mr. Davis,

First, please forgive me if I'm not following proper procedure in 
commenting on this draft; while I do have a strong programmer's interest 
in this standard, I admit that I'm not typically a participant in these 
procedures and haven't thoroughly educated myself on the policies for 
submitting comments.

I would like to recommend an addition to this draft, for which I think I 
can make a rather compelling case based on hypothetical but quite 
reasonable scenarios. Personally, I hope very much that your draft 
becomes a standard, as the problems with a canonical parsing of current 
RFC 3066 language tags are well-known and bothersome to developers 
everywhere. Your draft strikes me as an excellent way to finally 
standardize the practice in a way which will be accessible to all 
developers without having to investigate thirty different standards and 
documents from ten different organizations.

Regarding Section 3.4 on extensions and extension namespace: You already 
have here a mechanism in place for extending this specification. I would 
like to suggest an extension which should probably be incorporated into 
the main specification. I believe you should define an "organization 
convention" extension for use by private companies and organizations for 
their own purposes.

I realize that a "private use" extension is already defined in section 
2.2.7. However, I maintain that the private use extension is not 
sufficient for potential development and interdevelopment among 
important organizations, as there is no way a parsing agent could assume 
anything significant about the tags which follow. And yet, the 
registration of 3.4 extensions is also insufficient because, frankly, 
you'll rapidly run out of letters if you make a sincere effort to define 
namespace for private companies and organizations.

Let's take a concrete example. Let's say that the American Library 
Association (ALA) decides to define an extension to help them classify 
books by reading level. As your specification stands, they have two 
choices: they can register a 3.4 extension (we'll say they register "L") 
and then use their subtags as follows:

en-US-L-g6: A book written in English as spoken in the United States at 
the sixth-grade reading level.

The ALA would have excellent reasons for wanting such a tag, as it would 
greatly facilitate the database querying and transfer of material to 
public schools.

However, we see the first problem: the ALA has their tag, which many 
schools would use. Then, Associated Press would want their tag to 
indicate regional assumptions. We'll give them "P" (for "press"):

en-US-P-ky: An article written in English as spoken in the United States 
which assumes readers are already familiar with names, cities, politics, 
etc., in Kentucky. (They would use this to distribute versions to 
Kentucky press where they don't have to explain that Frankfurt is the 
capital, distinguishing them from national or international versions 
which would make no such assumption and explicitly specify that 
Frankfurt is the capital.)

If we keep up like this, as I mentioned, we'll rapidly run out of 
singleton letters. Everyone will want one, some for valid reasons, 
others for silly reasons, and then your registration authority would be 
in the unenviable position of having to make value judgments regarding 
what is valid and what is silly, given such limited real estate.

Furthermore, you'll be putting the organizations themselves in a 
difficult position. For example, if the ALA decides to modify their 
convention, this is something that is only of interest to them and the 
people who use their specification. However, in order to make their own 
internal changes, they will technically have to go through the entire 
process of revising a stable specification through the registration 
authority (according to 3.4, which requires stability and canonical 
representation), something which is never recommendable.

And finally, parsing agents which have no interest in the ALA's tag 
(which will be most of them) will nonetheless have the burden of 
checking conformance.

If we take the other approach, and say, "We have the 'x' tag for private 
use. The ALA and AP can take that tag and follow it up however they 
want," then we're creating another problem. All of the parsing agents 
which do have an interest in those tags cannot be guaranteed that they 
mean what they think they mean.

For example, if the ALA decides to go with:

en-US-x-ala-g6

But subsequently the Associate Press decides that their private tag 
"x-ala" means articles of interest to Alabamans, then what's the ALA do 
to when they want to classify articles written by the AP? The problem is 
that parsing user agents will be unable to assume anything about the tag 
that follows, and once a conflict occurs, both tags become either 
useless, or subject to the type of interpretation that a human might 
perform easily but a machine cannot.

The solution is simply to define an organizational namespace. We take a 
random tag--we'll say "P" for private--and then allow companies and 
organizations to register their own namespace. Everything that follows 
their namespace tag is then interpreted according to their standard, 
whatever that may be. For example, the ALA would register "ala," the AP 
would register "ap," Microsoft would register "mcrsoft," Adobe would 
register "adobe" and so on.

Then, anyone seeing a tag like this:

en-US-P-ala-g6

could know unambiguously that whatever follows the P-ala is to be 
interpreted by the ALA's own convention, whatever that might be. Each 
registering organization could then be responsible for the stability and 
canonical representations of their own namespace without affecting the 
stability of the specification as a whole.

Parsing agents which are not interested in the AP's tags simply knows to 
ignore anything after the "P" tag that isn't an organization in which it 
has an interest. Parsing agents that are interested can now know with 
assurance that the information is what they're looking for. Companies 
and organizations can establish their own standards which can easily 
evolve to suit their needs. Private companies can establish 
compatibility standards between themselves which won't affect the 
specification as a whole.

This could be infinitely extensible merely by setting aside one of the 
organizational tags to mean "check the next set." For example, if the 
American Library association registers "ala" as above, and then later 
the Association of Libertarians and Anarchists shows up, finds that all 
the mnemonic representations of their name are already used and there's 
not much space left on the registery (and with 368 alphanumeric 
possibilities, that's not likely, but let's pretend), they could define 
their namespace as "set2-ala" (assuming we've already decided that 
"set2" is the tag when means "check the next set").

This allows all companies and organizations which have a need to define 
their own namespaces and then use them as the needs of their particular 
domain indicate in a way that is nonetheless unambiguously established 
for parsing agents which can then make error-free decisions about 
whether or not the information which follows is useful to their needs, 
all done without sacrificing the stability of the main specification.

This is the extent of my speculation on the issue. I did consider the 
possibility of using Java-package-name-like identifiers tied to domain 
registration, so that Microsoft could have the "com-microsoft" tag and 
the ALA could have the "org-ala" tag, but this would end up violating 
the eight-character rule and allow just any yahoo with a website to 
include whatever he sees fit (en-US-com-sexychicks-38D comes to mind), 
which I don't think is a desirable solution at all.

If you have found this comment at all useful, I would appreciate hearing 
back.

Sincerely,
Dylan N. Pierce
IT Coordinator, TykeTek

TykeTek/Diapositivas Gloria
Salvador Quevedo y Zubieta #821 Int. 6
Col. la Perla
C.P. 44360 Guadalajara, Jal.
MEXICO

E-Mail: [email-masked]
Telephone: +52 (33) 3617.3660
Cellular: +52 (33) 1149.7057

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: r&d afrac <[email-masked]>
Date: Wed, 06 Jul 2005 01:18:59 +0200

Dear Dylan,
this request makes a lot of sense. There are many issues there. A few ones.

1. you characterise in a way the use of a document. I am not sure this 
directly fits with the characterisation of a language. But it characterise 
a relation channel. I mean by that the way the document is intended to be 
received, send or exchanged, and from there classified. Today the proposed 
Draft leaves this undefined in at least two ways:

- first paragraph. It says "Human beings on our planet have, past and 
present, used a number of languages.  There are many reasons why one would 
want to identify the language used when presenting or requesting 
information.". One could say that "used" may relate to exchanged, 
"presenting" to send and "requesting" to received, with some variation 
because for example requesting does no mean that it was received.

- part 2. "The language tag always defines a language as used (which 
includes being  spoken, written, signed, or otherwise signaled) by 
human   beings for communication of information to other human beings. 
Computer languages such as programming languages are explicitly excluded."

The problem here is that a language is not defined (what it is? how is it 
identified? etc.) however the langtag is normative of that something. The 
usage of the proposed langtags can only be subjective (the perception of 
the users) and the discussed language to be rather undefined concepts. Your 
proposition creates languages _values_ (the instanciation of the Kentucky 
press). I am not sure you can really qualify it within the Draft framework. 
This is because it is filtered by a media (Kentucky press) and not by a 
speakers community (unless you mean the readers  - or the authors? - of the 
Kentucky press). Please recall that they do not want to accept man/computer 
and computer/computer languages. This obviously creates a classification 
problem with StarWars, what is H2D2 speaking? and Yoda who is not a human 
being? The Japanese Fair of Robotics this years, shown Droïds interrelating 
in Japanese or in English. There is also a vacuum for computer generated 
texts, alarms, etc. Would you introduce "r" for Robots? but that would 
oppose the spirit of the Draft?

2. you want to permit organisations and persons to define their personal 
name space (cf. John Klensin recent Draft on IANA) and define their own 
format. This was recently proposed and denied. The first problem you meet 
with this is size of the namespace you need and its structure. You consider 
that Microsoft would register "mcrsoft", why that? Mr. Sungil Yoon who owns 
McrSoft.com has the right to use it. You will say that "Microsoft" is 
longer than 8alpha. Right, but RFC 1766 said that you cannot change that 
and is to be respected by consensus of this WG. Obviously you can object to 
this consensus, I will too, others will probably too and this will not be 
anymore a consensus. But short of that, only user owning universal rights 
(every class, every country) can claim a tag (like "mercedes" or 
"cocacola") otherwise we have conflicts.

I note that RFC 2860 can also create a problem for the Draft. Your naming 
part is by essence an ICANN part of the IANA: the Registrar and Examiner 
must be designated by the ICANN BoD and appeals probably subject to GAC. 
This should be reflected in the Draft. Is this really what you want?

Another problem you may not have investigated is that Microsoft could have 
different branches and needs, for example "microsoft.corp", "microsoft.us", 
etc. the languages spoken in its different branches being certainly 
different. This is why we have introduced three warnings you can find on 
http://rfc3066.org:

- there is no subtag size limit in the x-tags part
- the "." and the ":" are accepted characters, "." introducing a comment or 
an additional part and ":" permitting to use URNs? For us 
"x-en.microsoft.us-Latn-de" qualifies the language of a Mr. Gates visiting 
Germany.

But please note that, if this proposition, initially presented by F. 
Charles, cannot conflict with any previous format since it is a private 
area, it is not supported by this WG, what is deemed to have consensually 
opposed, (one or two objections).

3. you consider the notions of referent (ex.: commonly accepted reading 
level -L-6) and context (to know about the Kentucky cities and life). These 
are two levels which are very important to the support of a relation 
(together with their dates - as per ISO 11179 - to know which version is to 
be used). Other referents can be Dictionaries, Grammars, publishers, etc. 
Other contexts can be style, mimics, accents, etc. These notions are most 
probably too complex for the Draft and can be multiplied and need 
priorities in case of conflicts when two referential systems have different 
descriptions. Please accept that the Draft only supports one single mode 
(script) and has no provision (yet) to support other modes.

All this can/should certainly be supported. But this would call for a 
general framework introduction of language support (BCP 47) within the 
Internet architecture, as a continuation/extension of the RFC 3066. In this 
case the Draft would be an application of this framework. The sentence 
"This document replaces RFC 3066" should then be replaced by "this document 
complements RFC 3066":  this is a part of the debate over the Charter, this 
WG consensus does seem to want to engage.

4. you say you do not consider that using domain names would be adequate, 
but you do not document it. This is one of the solutions to support 
individual/avatars and contexts grids. I would therefore be interested you 
document your position. This is a point which is hotly debated in some ISO 
committees, and belongs to what is qualified as the "pulverisation" of a 
user-centric Internet (i.e. its ultimate granularity). Work currently 
carried one coreboxes and OPES (WG-OPES) go down to this degree and even 
below (the individual relation level and context: the way you speak when 
you are with someone else specific, under some identified circumstances. 
ex: the language you use with a cop who stopped you on the road).

Thank you for this interesting thinking.
jfc

At 00:04 06/07/2005, Dylan N. Pierce wrote:
>reasonable scenarios. Personally, I hope very much that your draft becomes 
>a standard, as the problems with a canonical parsing of current RFC 3066 
>language tags are well-known and bothersome to developers everywhere. Your 
>draft strikes me as an excellent way to finally standardize the practice 
>in a way which will be accessible to all developers without having to 
>investigate thirty different standards and documents from ten different 
>organizations.
>
>sufficient for potential development and interdevelopment among important 
>organizations, as there is no way a parsing agent could assume anything 
>significant about the tags which follow. And yet, the registration of 3.4 
>extensions is also insufficient because, frankly, you'll rapidly run out 
>of letters if you make a sincere effort to define namespace for private 
>companies and organizations.
>
>schools would use. Then, Associated Press would want their tag to indicate 
>regional assumptions. We'll give them "P" (for "press"):
>
>etc., in Kentucky. (They would use this to distribute versions to Kentucky 
>press where they don't have to explain that Frankfurt is the capital, 
>distinguishing them from national or international versions which would 
>make no such assumption and explicitly specify that Frankfurt is the capital.)
>
>singleton letters. Everyone will want one, some for valid reasons, others 
>for silly reasons, and then your registration authority would be in the 
>unenviable position of having to make value judgments regarding what is 
>valid and what is silly, given such limited real estate.
>
>Furthermore, you'll be putting the organizations themselves in a difficult 
>position. For example, if the ALA decides to modify their convention, this 
>is something that is only of interest to them and the people who use their 
>specification. However, in order to make their own internal changes, they 
>will technically have to go through the entire process of revising a 
>stable specification through the registration authority (according to 3.4, 
>which requires stability and canonical representation), something which is 
>never recommendable.
>
>And finally, parsing agents which have no interest in the ALA's tag (which 
>will be most of them) will nonetheless have the burden of checking conformance.
>
>use. The ALA and AP can take that tag and follow it up however they want," 
>then we're creating another problem. All of the parsing agents which do 
>have an interest in those tags cannot be guaranteed that they mean what 
>they think they mean.
>
>"x-ala" means articles of interest to Alabamans, then what's the ALA do to 
>when they want to classify articles written by the AP? The problem is that 
>parsing user agents will be unable to assume anything about the tag that 
>follows, and once a conflict occurs, both tags become either useless, or 
>subject to the type of interpretation that a human might perform easily 
>but a machine cannot.
>
>assurance that the information is what they're looking for. Companies and 
>organizations can establish their own standards which can easily evolve to 
>suit their needs. Private companies can establish compatibility standards 
>between themselves which won't affect the specification as a whole.
>
>American Library association registers "ala" as above, and then later the 
>Association of Libertarians and Anarchists shows up, finds that all the 
>mnemonic representations of their name are already used and there's not 
>much space left on the registery (and with 368 alphanumeric possibilities, 
>that's not likely, but let's pretend), they could define their namespace 
>as "set2-ala" (assuming we've already decided that "set2" is the tag when 
>means "check the next set").
>
>domain indicate in a way that is nonetheless unambiguously established for 
>parsing agents which can then make error-free decisions about whether or 
>not the information which follows is useful to their needs, all done 
>without sacrificing the stability of the main specification.
>
>registration, so that Microsoft could have the "com-microsoft" tag and the 
>ALA could have the "org-ala" tag, but this would end up violating the 
>eight-character rule and allow just any yahoo with a website to include 
>whatever he sees fit (en-US-com-sexychicks-38D comes to mind), which I 
>don't think is a desirable solution at all.
>
>If you have found this comment at all useful, I would appreciate hearing back.
>
>Sincerely,
>Dylan N. Pierce
>IT Coordinator, TykeTek
>

---

From: "Dylan N. Pierce" <[email-masked]>
Date: Wed, 06 Jul 2005 11:48:34 -0500

In response to jfc (forgive me if I don't yet understand how to ensure 
that my e-mail appears in the thread below the appropriate post; I 
freely admit this is my first time participating in a procedure such as 
this).

Whenever I read a standard, for better or for worse, the question I am 
asking myself isn't, "What does this mean?" or "What purpose does this 
serve?" I am always asking myself, "How do I write a program that does 
this?" This is why RFC 3066, for all that it's a BCP, simply is 
inadequate; I am interested enough in this working group to come here 
and express a solid support for the current direction because having a 
language tag which is parsable according to constructable rules greatly 
reduces the amount of work any programmer has to do when developing for 
compliance.

As such, I read your first point, regarding characterizing the use of a 
document, and it tells me something interesting about myself. The fact 
is, for all its irony, I'm not typically even remotely interested in how 
a document is used; I tend to focus on how a document is /filed/ by the 
people who use it. If a client tells me, "I want a list of all documents 
sorted alphabetically by the third sentence of the second chapter," I 
might ask, "Why? Are you sure?" but if the client insists, I'll 
dutifully begin writing the appropriate algorithsm.

I admit that issues of defining "What is a language?" and its 
philosophical correlates are perhaps of coffee-table interest to me, but 
not of professional interest as a programmer. Instead I merely want to 
know how I serve to any random end-user the appropriate document 
following whatever language /he/ thinks he's speaking. This means I need 
my language tags to be /descriptive,/ not proscriptive, and they have to 
be extensible in a logical way.

For better or for worse, if we are describing human languages, we must 
deal with the reality that human beings /do/ invent languages. It /is/ 
possible to find websites written in Klingon and poetry written in 
"Yodish." A bit disconcerting, to be sure, but possible. Certainly, if 
someone in my living room was trying to speak to me in Klingon, I'd 
probably request that he get a life, but /personally,/ I can do that. 
Professionally I can't; I can't use the fact that a man who speaks 
Esperanto, an equally artificial language, is more likely to have a 
girlfriend than a man who speaks Klingon as a justification for design 
limitations. Again, human beings /do/ invent languages and any tagging 
standard which does not account for this reality is inadequate for the 
task of classifying human languages.

Effectively, I look at the language tag in this fashion: if, for any two 
random given people, I must use two different syntaxes to say the same 
thing, then I need a different tag. en-US and en-GB are different not 
just because the British like throwing in superfluous u's, but also 
because the word "fanny" gets me in more trouble in one place than in 
the other. Same with the word "mantequilla" en es-MX versus es-AR. 
Anyone interested in providing global content must be able to navigate 
these differences /and/ similar unpredictable differences which arise in 
the future. What happens, for example, when significant language-use 
differences are based on social class in the exact same region in the 
exact same tongue? No existing tag accounts for this; can we be sure 
that 35 possibilities for extension protect us against future social and 
creative inventions? Extensibility and modularity must be incorporated 
into the system from its inception or the system will fail. Human 
creativity and pop culture move altogether faster than specification 
revision committees.

Ultimately, these tags are not, and /cannot/ be, proscriptive for how 
people are /allowed/ to classify languages; you can't program humans 
like a computer. Instead, they need to be descriptive of how human 
beings /use/ language--"use" it here in both senses, of how they 
actually speak, write or signal it, and also in how they already 
classify it for their own purposes of transmission, and that 
descriptiveness must share the same capacity for growth as the objects 
it describes. In other words, the tags used to describe languages must 
themselves be like languages: if the language changes from region to 
region, so must the tags. If languages divide or combine over time, so 
must the tags. And if languages can spring wholesale from the minds of 
hack science-fiction screenwriters, so must the tags.

Further, if languages can be analyzed for factors important to one 
organization but irrelevant to another, so must the tags. The 
reading-level example, for instance, is intrinsically part of how 
language is used within a culture; the very educational institutions 
teaching the language divide material in this fashion. The regional 
press example points to how material is requested and provided, still 
however analyzed based on sheer linguistic--word choice and level of 
abstractness--factors. And since it would be daunting for a registration 
body to make any attempt at trying to track and describe the myriad of 
human possibilities for interpreting a language, best we put that charge 
directly in the hands of the people who do it for a living. Certainly 
this means that corporations will also use their namespace for less 
germaine reasons. And fortunately, parsing agents can ignore their tags 
and still remain completely in compliance: a small price to pay for 
effectively making the entire world a de facto but organized 
registration authority for how language is used worldwide.

I've been informed both here and privately that perhaps a more 
appropriate approach would be to wait until this document becomes a 
standard and then propose organizational namespace as a new Internet 
Draft. Certainly, you guys know better than I do what we're up against 
and I'll defer to your best judgment. But the entire reason I so 
strongly support this project is because we undeniably /need/ a parsable 
internationalization architecture (as a programmer, /I/ need it, and I 
have yet to speak to a colleage who accuses RFC 3066 of being 
sufficient) and it needs to speak to all the ways in which languages are 
used, distributed, selected, and even (perhaps I'm making enemies here) 
invented.

Extensibility can be anything from a lifesaver to a mere buzzword. For 
any descriptor to be extensible in a way which has value, it must be 
extensible in exactly the same ways in which the object it describes is 
extensible. If it is not, time and human creativity will obsolete it.

Sincerely,
Dylan N. Pierce

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 6 Jul 2005 11:45:43 -0700

Hi Dylan -

If I might summarize your message, I think you have four concerns:
   1) are tags descriptive or prescriptive?
   2) how well does the specification cope with new languages?
   3) support for tagging other linguistics aspects of a document
   4) is the space defined by the extension mechanism sufficiently large?

As a technical contributor, I think your first concern can be definitively
answered: "Language tags are, from a linguistic perspective, descriptive."
Would it be helpful to include this sentence in the i-d?  The only person
who has argued otherwise, as far as I know, is Mr. Morfin.

On your second concern, the support for novel or artificial languages, the
draft is quite clear that if the language is used by humans for communication
with humans, it is within scope.  (Section 2)  Since, as you affirm, Klingon is
in use for human-to-human communication, it has been registered.  I think
this is a pretty convincing demonstration that the registry can cope with novel
languages.

Regarding your third concern, tagging other linguistic aspects of a document,
as "social class" or "preferred spelling dictionary", I think you'd find some
lack of agreement on whether these belong in a language tag or whether
they should be regarded as distinct attributes.  The working group does not
need to resolve this question to meet the terms of its charter.  My personal
view is that this is a good thing; the universe of all possible attributes of
linguistic interest is far to large to permit completion of our work in a
reasonable time.  I think using a language tag to encode all the
linguistic attributes of a document makes as much sense as trying to
boil the ocean.  There's a big difference between identifying the language
of a document and cataloging all its linguistic attributes.

On your fourth concern, whether the space defined by the syntax of the
extension mechanisms is sufficiently large, depends in large part on the
extent to which you try to use language tags to address your third concern.
If the only tool you have is a hammer, then every problem is a nail.  :-)

Randy

----- Original Message ----- 
> From: "Dylan N. Pierce" <[email-masked]>
> To: <[email-masked]>
> Sent: Wednesday, July 06, 2005 9:48 AM
> Subject: Re: [Ltru] Private Use Tags
>

---

From: "Dylan N. Pierce" <[email-masked]>
Date: Wed, 06 Jul 2005 14:32:50 -0500

Randy,

Sounds pretty accurate to me. Remembering that my point of view is 
generally always tainted by use-case scenarios, you'll have to forgive 
my tendency to focus on an infinitude of "What if?" hypotheticals when 
everyone around me merely wants to ship a finished product; that happens 
at my home company, as well.

Regarding point one, I imagine it's a case of being too close to the 
issue. Of /course/ we know they're descriptive. But these things are 
designed to be used by actual people, and people can be picky. I'm sure 
Mr. Morfin is not unique in that regard. If something as simple as 
specifying, "These tags are descriptive, not prescriptive, and do not 
represent a limiting factor in languages or variations an application 
may support," could head off a perceived slight by the unrepresented, 
that doesn't seem to be an excessively high cost.

Regarding two, actually, I'm in complete agreement with you; I didn't 
mean my side trip down that road to be taken as a point of contention 
with the current draft. Rather, it was more a direct response to jfc's 
concerns about ever-spiraling artificial languages, to serve as a sort 
of affirmation that yes, the number of artificial languages will most 
likely continue to grow, and our job isn't to do anything about that 
other than design a mechanism which can describe it. I have no argument 
that the current mechanism seems to fulfill that need.

The third point is the interesting one (and, as you noted, the fourth 
point merely follows from it). Perhaps my mistake was including 
examples; those are too easy to take as straw men. My position is most 
definitely /not/ that the language tags should be used to catalog other 
linguistic aspects of the document. Making the specification that 
precise would, in fact, be rather /against/ the direction I want to go; 
it would simply be eliminating more possibilities by defining canonical 
ones.

My suggestion, instead, is an attempt to refine the private-use tag 
system to include organizational namespaces so that organizations /that 
would be using private tags anyway/ for whatever purposes they felt 
legitimate--perhaps including but certainly not limited to other 
linguistic features--can have an added guarantee that, when 
intercommunicating with other organizations who agree with their tags, 
their respective specialized parsing agents can definitively know that 
the particular private tag they are looking at is, in fact, the one of 
interest to them and not someone else's private tag that happens to 
share the same letters. Exactly /what/ this would be used for is, as you 
have rightly noted, quite beyond the scope of this project and would 
probably bore us all to death anyway (as if I'm not doing that already).

So no, please don't misunderstand me; I'm not campaigning for a complete 
linguistic analysis in the tag. I'm campaigning for increased 
compatibility by having a clear, machine-readable way of informing those 
people who are already taking advantage of private-use tags that the 
particular private tag they are looking at is in fact the one they meant 
to look at. That kind of clarity and assurance, I believe, would benefit 
everyone.

Sincerely,
Dylan N. Pierce

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 6 Jul 2005 13:38:49 -0700

Hi -

(as a technical contributor)

> To: "Randy Presuhn" <[email-masked]>; <[email-masked]>
> Sent: Wednesday, July 06, 2005 12:32 PM
...
(much agreement elided)
...
> to look at. That kind of clarity and assurance, I believe, would benefit
...

Thanks for the clarification.  I think there's really a policy question
underneath this: how strongly do we want to encourage the registration
of subtags?  If the point of having a registry is to provide open access
to information that allows someone to figure out what a tag signifies,
and to promote consistent tagging of data, motivated by a desire to promote
interoperability, then I think it's important to emphasize the "last resort"
nature of private-use tags.

As you note, the need to structure a space of private use tags arises
when there is a risk that they would "leak".  I such a case, the use
is no longer "private", and properly registered tags should be
used instead.  I think this is especially true if there is any reason
anyone else would be interested in material in the language in question.

An example of a mechanism that could provide extreme extensibility
would be something like http://www.iana.org/assignments/enterprise-numbers.
We certainly had plenty of experience with that in the IETF's Operations
and Management area.  Some of the problems with such a discipline (or any
mechanism that encourages the use of (sub)tags which are not publicly
registered) include:
    1.  there may be multiple ways to say the same thing
        (especially bad for matching, but also a problem for those
        tagging content)
    2.  there is no common way to find out what a given (sub)tag means
    3.  when different organizations have their own registries, the
        rigor with which the registrations are maintained varies wildly.
    4.  when an organization dissolves or is aborbed by another organization,
        its registry often is lost in the shuffle, but the data lives on

These are the kind of real problems the SNMP and CMIP communities have
experienced.  In the case of language tagging, I believe the need for
vendor- or organization-specific (sub)tags is even lower, and the value
of a common set of registrations is even higher.

Consequently, even though I think there is value in permitting private use
extensions, I think we should do nothing to encourage their proliferation.

Randy

---

From: r&d afrac <[email-masked]>
Date: Thu, 07 Jul 2005 02:17:45 +0200

>who has argued otherwise, as far as I know, is Mr. Morfin.

I am afraid this may be a mis-representation of my position. I do not want 
it to be said "descriptive", I wand it to be defined. This being said, I 
would certainly welcome the wording proposed by Dylan: I am not sure wat 
your "linguistic perspective" want to say, all the more than languages are 
not defined.

>they should be regarded as distinct attributes.

Referent and context (or style) defines versions of a language.

>The working group does not
>need to resolve this question to meet the terms of its charter.

I disagree. Not on the integration of ISO codes. But on scalability.

>   My personal
>reasonable time.

No. It is sufficient to say that referent and context subtags (particulars 
of a language, and particulars to its mode of usage) are given a format and 
to be registered. This only calls for the use of ".". 
en.oxford.casual-Latn-US is OK with me. Or en.IBM.admin-Latn-US. Parsers 
need only to stop at the first "." in a subtag.

>  I think using a language tag to encode all the

True. This is why the idea to formally attach non related attributes is an 
odd idea. Also to document it via an RFC and to make it a IANA registry. 
But if you really want it, it is to be engaged correctly. This means it 
must scale.

>If the only tool you have is a hammer, then every problem is a nail.  :-)

Scalability is the limit. What means there is no limit.
jfc

---

From: r&d afrac <[email-masked]>
Date: Thu, 07 Jul 2005 03:36:04 +0200

Dear Dylan,
thank you for your response. I think we are in agreement on many things. 
But - you are a programmer - you know that from a concept to a program 
there must be an analysis and a development. Also, there is a added 
complexity which is that we are the IETF and not Open Source or Unicode. 
This means that we have to deal with networks, what means asynchronous 
interactions in real time with unknown final agents. There are simple rules 
from experience which help through this complexity which have been 
summarized by the present IETF Chair in RFC 1958. It is really worth 
reading and try to think that way and if possible improve it. Obviously you 
add your experience. I see you are following this.

There are three rules which are basic for me: (a) except the rule which 
says that, everything can change (b) kiss, keep it simple, stable, stupid, 
etc. (c) scalability which means that you must be able to apply the theory 
of that rule everywhere it can apply and it will work, a correlative one is 
that you should not solve two things which may become similar in two 
different ways. In a nusthell this means open consistency: open, everything 
is possible, consistency, one single idea should apply because sometime, 
somewhere you will have a conflict. This is basically what you say everywhere.

There is however a difference between what you say about programing and 
what I say about architecture: just think a step further, because here we 
are normating. So you _are_ to ask "why?" and not "are you sure", but "if 
we want that, we all have to do it that way for greatest efficiency": you 
aree like the client of your client. You build the envirnonment he will use 
to ask you to develop an application.

At 18:48 06/07/2005, Dylan N. Pierce wrote:
>that my e-mail appears in the thread below the appropriate post; I freely 
>admit this is my first time participating in a procedure such as this).
>
>inadequate; I am interested enough in this working group to come here and 
>express a solid support for the current direction because having a 

The problem specific to this group was decscribed by Addison. He says and 
we all agree: RFC 3066 is too imprecise. There are two responses to that:

1. people like me who say (for my applications) "great, I am free"
2. people like Addison (for his XML or Unicode, etc.) and you for you 
applications who say "gee, I am lost".

The role of the Draft is to address (2) while protecting (1). Now, you say: 
(3) "great, but I want more", and the role of the Draft should be to 
provide more. Now, you see we have a problem, because the Draft is expected 
to provide (1), (2) and (3) and provides only (2).

What the Internet community (charter) expects from a BCP 47 is to address 
the problems of RFC 3066, which only provided (1). This can be done in two 
ways:

1. in writing a language tags framework for the Internet, of which the 
Draft will support (2) and will be ready to support as many (3) as you need.
2. because much work has been spent on (2) and that x-tags permit to 
support other formats and explore (3), to keep (1) and add (2) in just 
changing "replace RFC 3066" by "complement RFC 3066". Then we can all 
freely work.

>document, and it tells me something interesting about myself. The fact is, 
>for all its irony, I'm not typically even remotely interested in how a 
>document is used; I tend to focus on how a document is /filed/ by the 
>people who use it.

Then you are more a programer (an author) than a networker. However, please 
note that you use the terms "people who use it". You do not say "people it 
is intended for". So you consider some action by the "non authors".

>If a client tells me, "I want a list of all documents sorted 
>alphabetically by the third sentence of the second chapter," I might ask, 
>"Why? Are you sure?" but if the client insists, I'll dutifully begin 
>writing the appropriate algorithsm.
>
>not of professional interest as a programmer.

I am not sure of that !!! Identfying a language is a very complex bit a 
programing. The language will be defined by statistical rules, etc. Look 
for example the orthographic correction of Word, which support 
multilanguage text and considers the language of the paragraph. This means 
a very interesting piece of code to decide that a paragraph is French and 
the next one is English. But, mainly,as a networker you see a language is 
an interaction protocol. No basic conceptual difference between http and 
English and C. But they can be used in many different modes and over many 
different media. Now, as a programer you become all the sudden very 
interested when due to that the language is not going to be the same 
depending on the mode or the media because the mode or the media is going 
to interfere with the restitution or because you will be able to compact 
exchanges (what you do for example with an error message).

>Instead I merely want to know how I serve to any random end-user the 
>appropriate document following whatever language /he/ thinks he's 
>speaking. This means I need my language tags to be /descriptive,/ not 
>proscriptive, and they have to be extensible in a logical way.

Descriptive/prescriptive have a meaning only in a context of use and by 
which partner of the exchange at which point of the exchange. If I describe 
what is necessary it becomes prescriptive.

Full agreement with the extensibility requirement.

>deal with the reality that human beings /do/ invent languages.

Yes all the time. And the worst of it is they invent it for machines too. 
If I tell you printf("you %s\n", "right"). What is it?

>  It /is/ possible to find websites written in Klingon and poetry written 
> in "Yodish." A bit disconcerting, to be sure, but possible. Certainly, if 
> Professionally I can't;

Why? If your customer is Lucas, I bet you will do what he wants and you 
will send your bill. Open consistency, or scalability.

>I can't use the fact that a man who speaks Esperanto, an equally 
>artificial language, is more likely to have a girlfriend than a man who 
>speaks Klingon as a justification for design limitations. Again, human 
>beings /do/ invent languages and any tagging standard which does not 
>account for this reality is inadequate for the task of classifying human 
>languages.

Right!

>thing, then I need a different tag. en-US and en-GB are different not just 
>because the British like throwing in superfluous u's, but also because the 
>word "fanny" gets me in more trouble in one place than in the other.

Right! But this may be also true with people who have lived something 
good/bad in common a word alludes to.

>Same with the word "mantequilla" en es-MX versus es-AR. Anyone interested 
>in providing global content must be able to navigate these differences 
>/and/ similar unpredictable differences which arise in the future. What 
>happens, for example, when significant language-use differences are based 
>on social class in the exact same region in the exact same tongue?

100% agreement.

>No existing tag accounts for this; can we be sure that 35 possibilities 
>for extension protect us against future social and creative inventions?

We can be sure of not. And why would pentatridecimal be a limit (why 35 BTW?).

>Extensibility and modularity must be incorporated into the system from its 
>inception or the system will fail. Human creativity and pop culture move 
>altogether faster than specification revision committees.

100% agreement.
But remember you are a programer. So you must consider how to implement it. 
The work we carry is the following/

1. two or more people dialoging/polyloging together establish a space of 
exchanges. This space has common references, we name referents (never mind 
at this stage what they are).
2. in so doing the establish relations. Each relation has its own context 
which modify (enrich, reduce, modify, etc.) the referent.
3. the exhanges are carried in using person to person interintelligibility 
protocols named "languages" supported by the end to end interoperation.
4. the referents and contexts are supported by "CRC" (common reference 
centers) where all the protocols elements (which extend far further than 
the pure language dscussed here, but will include the DNS, the LADP 
Directories being uses, etc.) can be seeked. For example through the DNS, 
OPES, etc.
5. one of the first thing people will do will be to negotiate the 
environment of the exchange and of the relation. For example, they will 
start negotiating the protocol: http/ftp?, English, French, Spanish;;; what 
is the most adequate to general exchange and to specific relations? You see 
that for example I use English here, but French would be better for me and 
I could try Spanish with you. At a given time you negotiate a Spanish 
context when you quote "mantequilla" (as a French speaker es-MX and es-AR 
are the same). You see that depending on the language we negotiated at a 
given time in the exchange or relation, if we quote a Web site, it will be 
different and if we call an LDAP directory the result may be different if 
it is multilingual.

>people are /allowed/ to classify languages; you can't program humans like 
>a computer. Instead, they need to be descriptive of how human beings /use/ 
>language--"use" it here in both senses, of how they actually speak, write 
>or signal it, and also in how they already classify it for their own 
>purposes of transmission, and that descriptiveness must share the same 
>capacity for growth as the objects it describes. In other words, the tags 
>used to describe languages must themselves be like languages: if the 
>language changes from region to region, so must the tags. If languages 
>divide or combine over time, so must the tags. And if languages can spring 
>wholesale from the minds of hack science-fiction screenwriters, so must 
>the tags.

Right. And the way this is done on a distributed network is subsidiarity. 
This means that one (several?) semantics are to be defined (for mutual 
understanding, parsing, filtering, etc.) - as this Draft does for one - and 
people can use it the way they want. When you registered "megared" or 
"dylanpierce" the rule was first come fist serve. You were free.

>reading-level example, for instance, is intrinsically part of how language 
>is used within a culture; the very educational institutions teaching the 
>language divide material in this fashion. The regional press example 
>points to how material is requested and provided, still however analyzed 
>based on sheer linguistic--word choice and level of abstractness--factors. 
>And since it would be daunting for a registration body to make any attempt 
>at trying to track and describe the myriad of human possibilities for 
>interpreting a language, best we put that charge directly in the hands of 
>the people who do it for a living.

"who do it for a living" or "who live with it"?

>Certainly this means that corporations will also use their namespace for 
>less germaine reasons. And fortunately, parsing agents can ignore their 
>tags and still remain completely in compliance: a small price to pay for 
>effectively making the entire world a de facto but organized registration 
>authority for how language is used worldwide.

people empowerment is usually the best solution when you deal with people 
realtions. However you run into conflict easly because people tend to want 
to do the same thing. You usually have three ways to address this: the 
centralised hierarchy, which quickly lead to control (this is the system 
the Draft creates with strict reference to ISO), the decentralised system 
with needs some kind of technical oligarchy and also has some lose ends 
(this is the current RFC 3066 system), the distributed system where you do 
not have registration hierachichal trees but forests (everyone can create 
its own registry).

Dealing with centralisation is quite sipmlifying. And a temptation. But it 
does not work, because as you describe it the world is centralised, 
decentralised and distributed at the same time.

>I've been informed both here and privately that perhaps a more appropriate 
>approach would be to wait until this document becomes a standard and then 
>propose organizational namespace as a new Internet Draft.

This is not possible as long as this Draft wants to replace RFC 3066. This 
is no problem if it complements it. Then there are two solutions:

- either you proposition is general and you propose a framework for the 
Internet language idenfication which will replace RFC 3066 as BCP 47.
- or your proposition only specifies one additional semantic in parallel of 
the current Draft semantic and you refer to RFC 3066

>  Certainly, you guys know better than I do what we're up against and I'll 
> defer to your best judgment.

Up to you to decide. If the Draft replaces RFC 3066 your extensions will 
have to obey the Draft not RFC 3066. Look at what it implies. IMHO if you 
want to intriduce "p-" you are far better off now than starting a new 
document. Simply because they want the document to go through and you could 
block it without "p-" if your "p-" gathers support. Afterward, everyone 
will tell you "wait for experience" "we have decided no", etc.

We have the IDN experience. IDNA (Internationalised Domain Names 
applications) has been accepted by WG Members on the ground that it would 
be experimental and if it did not work we could change it. It does not 
work. M$ do not intend to implement it, yet after two years and China has 
split from the main Internet with its own co-root on the issue to correct 
it. IMHO (and we dispute on that, the same as we disputed over IDNs) this 
Draft if accepted will do the same. So it is very important to give it all 
the possible flexibility to give it a chance.

>But the entire reason I so strongly support this project is because we 
>undeniably /need/ a parsable internationalization architecture (as a 
>programmer, /I/ need it, and I have yet to speak to a colleage who accuses 
>RFC 3066 of being sufficient) and it needs to speak to all the ways in 
>which languages are used, distributed, selected, and even (perhaps I'm 
>making enemies here) invented.

Full agreement.
But I do not think anyone will tell that RFC 3066 is sufficient: RFC 3066 
is less restrictive. Actually what is wrong is the "langtag" by itself 
instead of supporting all the tags people really needs, whithout tying them 
into rigid supertags.

>Extensibility can be anything from a lifesaver to a mere buzzword. For any 
>descriptor to be extensible in a way which has value, it must be 

Amen.
jfc

---

From: r&d afrac <[email-masked]>
Date: Thu, 07 Jul 2005 02:30:53 +0200

At 21:32 06/07/2005, Dylan N. Pierce wrote:
>generally always tainted by use-case scenarios, you'll have to forgive my 
>tendency to focus on an infinitude of "What if?" hypotheticals when 

Seems you are a good guy for QA.

>represent a limiting factor in languages or variations an application may 
>support," could head off a perceived slight by the unrepresented, that 
>doesn't seem to be an excessively high cost.

Good start. I think it could do more. But good thing.

>Regarding two, actually, I'm in complete agreement with you; I didn't mean 
>my side trip down that road to be taken as a point of contention with the 
>current draft. Rather, it was more a direct response to jfc's concerns 
>about ever-spiraling artificial languages, to serve as a sort of 
>affirmation that yes, the number of artificial languages will most likely 
>continue to grow, and our job isn't to do anything about that other than 
>design a mechanism which can describe it. I have no argument that the 
>current mechanism seems to fulfill that need.

Here I object.
- it is for human/human only. I do not know what it means nowadays.
- please read at the practical limitations imposed via ISO

>to look at. That kind of clarity and assurance, I believe, would benefit 

This is called a semantic + vocabulary. This is not complex to achieve and 
there are many ways. But we are here engaged into the RFC 3066 format. The 
Draft wants a more strict semantic. A more strict semantic means more 
possible conflicts. You then reduce the risk in reducing the scope (your 
initial definition). Now, you want to extend the scope in adding reduced 
scopes: this is good as it gives more possibilities while not enlarging the 
risks of conflicts. But you are limited by the format ...

Our approach to that is to say (it was suggested here) we take "x-", reduce 
its constraints and document it. If developers do not support our extension 
it will not work, what is not a problem since our system does not want to 
be compatible in the "x-" part.

We could probably find a compromise. For everyone position to be supported. 
See my other mail.
jfc

---

From: "Dylan N. Pierce" <[email-masked]>
Date: Thu, 07 Jul 2005 10:32:58 -0500

> of subtags?

I'd have to agree with this. Probably this entire side discussion can be 
put to bed by taking a firm stand one way or the other on the answer to 
this question.

> If the point of having a registry is to provide open access

Eliminating them completely would be the way to achieve these things 
with certainty. Having them as sort of a "wild, wild west" area where 
there is no law and anyone can do whatever they want actually does the 
opposite of the three things you mention here (provide open access, 
promote consistent tagging, promote interoperability).

What I was proposing was, perhaps, a kind of middle-ground for these two 
extremes. Something in between complete, non-extensible regimentation 
and "may God have mercy on your soul." I'm beginning to see what the 
objections are about, and I wonder if perhaps including private tags in 
the first place wasn't a concession that shouldn't have been made.

> anyone else would be interested in material in the language in question.

Leakage is going to happen. Sun Microsystems, for example, finds the 
third subtag to be an excellent place to include currency information 
("es-ES-EURO"). Every copy of the JDK they've shipped since 1.2 has gone 
out that way. So on the one hand, they're hardly keeping it "private" 
and on the other hand, I haven't seen any petition for a registration of 
currency information in the third subtag, and I'd probably oppose it if 
I did (currency is even /less/ a part of language than reading level, in 
my opinion).

Point being that, if people are already doing it, and it seems 
reasonable (as it does to me, but I'm hardly representative) that people 
will continue doing it, we can either /act/ like they're going to do it, 
and respond to those needs, or we can clearly tell them, "Just don't do 
it /here/."

 >[some skippage of your text]
 >

Regarding the part I didn't copy over, I begin to see the problems. It 
makes me want to reverse my position to the point of saying, "Let's 
eliminate private tags completely; if someone uses something of their 
own, it's officially no longer in compliance with the specification."

My belief is that if something is there, it will be used (and perhaps in 
ways we do not foresee). If something is not to be used, it should not 
be there. I'm uncomfortable with the idea of "discouraged extensibility."

Just to head off misinterpretation, let me go on record as stating that 
if I were called upon to vote for this draft as it stands, if my vote 
would matter, I would vote to pass it. It's certainly better than its 
predecessor and that's why I took an interest in these proceedings in 
the first place. I don't want the perfect to be the enemy of the good, 
so for all intents and purposes, I support the draft.

But given what you've explained to me, I declare very firmly: we should 
stake out the "policy question" explicitly. Either we /do/ want to 
include private use tags, at which point we should set up a mechanism to 
ensure that the private use tags themselves do (as you put it) "promote 
open access, promote consistent tagging and promote interoperability" 
(by handling leakage and providing a way to know which private tag is 
which)... or we do /not/ really want to encourage private use tags, 
which we could do more consistently by simply saying, "These tags cover 
language, dialect and script, and nothing more. Anything over and above 
this should be handled outside the tag," and eliminate private use tags 
completely.

Dylan