Subject: [Ltru] Re: Question

From: "Doug Ewell" <[email-masked]>
Date: Thu, 24 Mar 2005 22:41:27 -0800

Frank Ellermann <nobody at xyzzy dot claranet dot de> wrote:

>> Are you aware that 'jw' was not documented as an error by
>> ISO 639 RA/JAC until 2002?
>
> No, it wasn't clear whether this was a typo fixed 1988 or a
> typo fixed 2001, or maybe an old typo fixed long ago but now
> officially documented.

It doesn't matter; the result is the same.

> The in, iw, etc. cases are different, they were no typos.

It still doesn't matter.  If these codes were ever made available to
users, they were probably used.

> Actually it's not very important, do whatever you like, but
> don't use it to justify FQ or RH later when we come to the
> region codes. ;-)

It's not a matter of whatever *I* like.  Please don't go there.  This is
supposed to be a group effort.  I sure wish more people would speak up
on this so it wasn't just a matter of "what Frank wants" versus "what
Doug wants."

And yes, I will be consistent and describe the same approach for both
language and country codes, at least until there is some rough consensus
for doing otherwise.  Right now it's just Frank and Doug bickering about
it on the list.

>> Frank, please write a counterproposal.  I don't think it's
>> accomplishing anything to continue saying "Or we could
>> totally change the way country codes are handled" without a
>> document that the WG can discuss.
>
> I've already done that using the worst case CS as an example
> how it would work for a 3066bis-my-way introduced about 1989.
> Addison added the 3066bis-your-way version for the same case.
>
> IMHO these examples were clear, in a 3066bis-your-way all
> obsolete ISO 3166-1 codes are preserved using UN numbers for
> some new regions, in a 3066-bis-my-way ISO 3166-1 is used as
> is, if somebody wants a stable tag (guaranteed for centuries)
> he's forced to use UN numbers instead of ISO 3166-1.

All right, let's make it a more-or-less formal...

QUESTION:  Should the draft implement region-code stability: (a) by
preserving all assigned ISO 3166 codes with their original meaning,
except those that have already been reassigned, and using UN numeric
codes in the case of future reassignments; (b) by allowing UN codes in
parallel with ISO 3166 codes, as a preferred alternative, and
deprecating the ISO codes; or (c) same as (b), but NOT deprecating the
ISO codes?

I won't bother writing a pro and con for this one.  I think everyone
interested will be able to judge for themselves.

> Let's hope that "they" don't deprecate any country code while
> we are desperately trying to find a compromise.  We still want
> compatibility with anything that was valid under 1766 / 3066.

I couldn't agree more.  See my earlier remark about a temporary
moratorium.

-Doug Ewell
 Fullerton, California
 http://users.adelphia.net/~dewell/

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: John Cowan <[email-masked]>
Date: Fri, 25 Mar 2005 02:27:10 -0500

Here's my take on Doug Ewell's question.  "Date A" means 1995-03-01,
the publication date of RFC 1766.  "Date B" means the publication date
of the new BCP that replaces RFC 3066.  These dates could be changed
without affecting my judgments very much.

> preserving all assigned ISO 3166 codes with their original meaning,

ISO codes terminated before Date A and not reassigned have never been
valid in 1766 or 3066 langtags, and should not be permitted.

ISO codes terminated after Date A and not reassigned before Date B
should be permitted and marked as deprecated, with the nearest equivalent
documented in the registry.

ISO codes in effect as of Date B, whether or not they were once assigned
to a different meaning, should be permitted in their meaning as of Date B.

> codes in the case of future reassignments;

UN codes that encode countries should only be allowed in the case of
the reassignment of an ISO 3166-1 code after Date B.

UN codes that encode regions of the world should be permitted.

> (b) by allowing UN codes in parallel with ISO 3166 codes, as a preferred
> alternative, and deprecating the ISO codes; or (c) same as (b), but
> NOT deprecating the ISO codes?

Having two codes for the same thing is always a bad idea and should not
be allowed unless a code reassignment after Date A but before Date B
makes it necessary.

-- 
After fixing the Y2K bug in an application:     John Cowan
        WELCOME TO <censored>                   [email-masked]
        DATE: MONDAK, JANUARK 1, 1900           http://www.ccil.org/~cowan

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 25 Mar 2005 08:21:01 -0800

I feel like this conversation is going off in the weeds. For backwards
compatibility, the goal should be at least:

(z) Every generated language tag that was valid under 1766 or 3066 remains
valid, with the same semantic. The only exceptions are cases where the code
changed to mean an unrelated entity (eg CS).

Your principle violates that.

> valid in 1766 or 3066 langtags, and should not be permitted.

This is precisely the problem. They *are* permitted. According to
http://www.ietf.org/rfc/rfc1766.txt, all and only the codes in
    [ISO 639]
         ISO 639:1988 (E/F) - Code for the representation of names of
         languages - The International Organization for
         Standardization, 1st edition, 1988 17 pages Prepared by
         ISO/TC 37 - Terminology (principles and coordination).

    [ISO 3166]
         ISO 3166:1988 (E/F) - Codes for the representation of names
         of countries - The International Organization for
         Standardization, 3rd edition, 1988-08-15.are valid. Note that
*nowhere* in 1766 does it say that a code later deprecated in ISO 639 or
3166 is not valid, nor does it even say that earlier deprecated codes are
invalid.‎Mark----- Original Message ----- 
From: "John Cowan" <[email-masked]>
To: "Doug Ewell" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, March 24, 2005 23:27
Subject: Re: [Ltru] Re: Question

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 25 Mar 2005 20:32:58 +0100

Mark Davis wrote:

> This is precisely the problem. They *are* permitted.

The obsolete RfC 1766 said:

| ISO 639 defines a registration authority for additions to and
| changes in the list of languages in ISO 639. This authority
| is:
[...]

...not an obscure reference in an obsolete RfC.  Same idea for
ISO 3166-1.  The actual RfC 3066 states:

| All 2-letter subtags are interpreted as ISO 3166 alpha-2
| country codes from [ISO 3166], or subsequently assigned by
| the ISO 3166 maintenance agency or governing standardization
| bodies, denoting the area to which this language variant
| relates.

Later 3066 speculates about the chances of an ISO 639-1:2000
published in 2001, and forbids the user assigned codes AA, ZZ,
QM-QZ, and XA-XZ, they were allowed in RfC 1766.

SNAFU.  It's the purpose of 3066bis to cleanup this mess, not
to preserve it.  Apparently nobody has a copy of ISO 3166:1988.
Doug is forced to guess using private Web pages what this old
stuff is about.  We found RH, BQ, FQ, and NQ, these codes are
irrelevant and disgusting.  Nobody recognizes these codes, and
nobody needs any ?Q for fantastic languages in the Antarctic.

> *nowhere* in 1766 does it say that a code later deprecated in
> ISO 639 or 3166 is not valid

"additions to and changes in the list".  Country codes like NH,
BQ, FQ, and NQ are shown as _unassigned_ on the 3166-1 page.
Maybe _deprecated_ in ISO 639 is a different issue, but 3166-1
has a clear list of states:

officially assigned       - examples: AQ, AX, CS
user assigned             - examples: AA. QQ, XX
exceptionally reserved    - examples: AC, EU, IM
transitionally reserved   - examples: BU, NT, SF
indeterminately reserved  - examples: DY. FL, RH
not used at present stage - examples: AP, BX, EF
unassigned                - examples: BQ, DD, NH

"Transitionally reserved" is roughly the same as "deprecated".
"Not used at present stage" is plain nonsense not related to
any country codes.  "Unassigned" is "free / available".

"Exceptioally reserved" are the missing ccTLDs and some others.
"Indeterminately reserved" is apparently used for car-plates.

BTW, Doug has RH as Southern Rhodesia, the ISO 3166-1 Web page
has RH as Haiti (indeterminately reserved).

                            Bye, Frank

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 25 Mar 2005 12:23:21 -0800

People presumed that 3066 was backwards compatible with 1766, and data was
generated on that basis. The cost of supporting full backwards compatibility
with 1766 and 3066 (for all but the ISO codes that were reused) is extremely
low, just a few additional codes. We then don't disturb the stability of
tags that were generated in good faith, using codes that at some point in
time qualified as 'assigned' in the base standards.

On that basis, I think it's a no-brainer.

You are the only one I hear saying that we don't care about backward
compability with codes that were valid under 1766 and 3066. Anyone else want
to speak up for non-backwards compatibility?

‎Mark

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 25 Mar 2005 22:15:27 +0100

Mark Davis wrote:
 
> People presumed that 3066 was backwards compatible with 1766,
> and data was generated on that basis.

You're talking about country codes valid 1995 (RfC 1766 and
later).  I'm talking about country codes invalid 1994 (before
RfC 1766), "unassigned" and similar cases like RH, NH, FQ, DD.

                        Bye, Frank

---

From: John Cowan <[email-masked]>
Date: Fri, 25 Mar 2005 16:40:02 -0500

> to speak up for non-backwards compatibility?

I deny that 3166 codes obsoleted before 1995-03-01 were ever valid in
1766 or 3066, so I agree with your statement de dicto, but I disagree
with it de re.  I believe that Addison agrees.

-- 
Yes, chili in the eye is bad, but so is your    John Cowan
ear.  However, I would suggest you wash your    [email-masked]
hands thoroughly before going to the toilet.    http://www.reutershealth.com
        --gadicath                              http://www.ccil.org/~cowan

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 25 Mar 2005 14:49:56 -0800

Joe Blow was deciding whether his code should allow "en-BU" as a valid code
according to 1766. He looks at the text and finds:
   In the first subtag:    -    All 2-letter codes are interpreted as ISO
3166 alpha-2
         country codes denoting the area in which the language is
         used.In the references, he sees:

    [ISO 3166]
         ISO 3166:1988 (E/F) - Codes for the representation of names
         of countries - The International Organization for
         Standardization, 3rd edition, 1988-08-15.

So he looks at ISO 3166:1988. There he finds that BU was perfectly valid.
And since 'en' was also, and the rest of the syntax was fine, it must be a
valid language tag. That is a perfectly reasonable interpretation on Joe's
part. Part of the problem, of course, *was* that 1766 and 3066 were not very
specific as to validity and stability. That is part of the reason that we
are being very careful about this in 3066bis!

Now, if we were talking about a large percentage of the codespace here, I
might agree that it is worth sacrificing backwards compatibility for these
codes. But since there are only a few, let's just include the darned things
for backwards compatibility and move on to do something more productive.

‎Mark

---

From: John Cowan <[email-masked]>
Date: Fri, 25 Mar 2005 18:22:29 -0500

> So he looks at ISO 3166:1988. There he finds that BU was perfectly valid.
 
Does he not also find a statement that the code list in ISO 3166:1988
is subject to revision by the 3166/MA?  That's a serious question;
I have never seen a copy of 3166:1988.

Jesuitical it may be, but it seems to me that discontinued 3166 codes
are not part of 3166.
 
> for backwards compatibility and move on to do something more productive.
 
The reason it matters is that there's a good chance some of those codes
will be used again someday, in which case it's going to be awfully lame
to say "Well, that code was used for the French claims to Antarctica half
a century or more ago, so you can't have it."  That's taking stability
to an insane level, particularly as there are no sensible xx-FQ language
tags and never will be.

-- 
While staying with the Asonu, I met a man from      John Cowan
the Candensian plane, which is very much like       [email-masked]
ours, only more of it consists of Toronto.          http://:www.ccil.org/~cowan
        --Ursula K. Le Guin, Changing Planes

---

From: Tex Texin <[email-masked]>
Date: Fri, 25 Mar 2005 15:22:10 -0800

The question of speaking up for no-BC is a bit loaded when asked in that
way.

I would not mind forgoing backward compatibility if there were
compensating benefits.

I would need to believe that the benefits were real, and also that there
weren't a number of hidden dangers that might be more problematic than
dealing with the cost of legacy.

But given the ambiguities in the legacy of language tags, and its
different usages by different technologies, I can certainly see creating
a new scheme. In fact would want it to look different so it didnt suffer
the confusion and baggage of the current system.

I said this a few years ago as well when we discussed solving the
problems I raised in the locales paper for the Unicode conference. I
would still welcome a new approach. I don't see however, that we have a
way to articulate a clean solution.

The problem I see is not in architecting the scheme for tagging, but
determining what a language is, in a useful way for software developers,
so that they can understand which language tag to use, and not an
identification which only linguists recognize or criteria which are not
clearly expressed.

So yes, forget compatibility if I can have a scheme that works.
On the other hand, if we maintain compatibility and make the current
system more complex and no more usable, I don't need that either.

I would welcome a clear set of rules for choosing a tag for content, and
rules for finding content with known tags.

tex

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 25 Mar 2005 17:06:16 -0800

Look, we regularly get beaten up by various IETF people for the backwards
compatibility of the Unicode Standard. I hate to see IETF take a much more
lackadaisical approach to language tags. I'm ok with excluding codes that no
reasonable person would believe is was valid at some time according to the
text of 1766 and 3066.

And if we start down the road of judging each individual case according to
whether we think it was "lame" or not, we will just tie ourselves up in
knots. We have a bright line; let's use it. We should simply not break
backward compatibility, when the only benefit is having a very small number
of additional codes.

‎Mark

---

From: Frank Ellermann <[email-masked]>
Date: Sat, 26 Mar 2005 12:38:33 +0100

Mark Davis wrote:

> Joe Blow was deciding whether his code should allow "en-BU"

That's a sad story, like Jane Blog's adventure in cyberspace:

Seeing RH on a car in Haiti she checks the ISO 3166-1 Web
page and finds something about "RH Haiti" as expected,  So
she tags a Creole text with fr-RH, and later she's surprised
when her job application is returned by the ZW embassy.

> let's just include the darned things for backwards
> compatibility and move on to do something more productive.

The most serious problem with the actual draft is, that it
forces us to choose between an erroneous concept of backwards
compatibility and compatibility with future country codes for
the known list of "invalid" country codes.  At least for the
decade 1995 - 2005 we can get one unambiguous idea of "valid".

                          Bye, Frank

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Sat, 26 Mar 2005 14:27:35 +0100

>decade 1995 - 2005 we can get one unambiguous idea of "valid".

I do not understand this problem. Are versioning and conflicts not 
supported by CVS for example?
UDC just choses to update its change on Jan 1st every year as did Internet 
when created.
Every military publication in the world, every legal texts, every 
commercial catalog, etc. has the same problem.

if AI was valid until 31/12/77 there are three entries in the tables:

AI77    262
AI      660
DJ      262
= 18 bytes

262     Djibouti
660     Anguilla
= 22 bytes

The application will use the date of the file to know the proper value of 
"AI".
This covers years -32.000 to + 32,000 .
The requirement is only that a datetag is provided or obtained from the 
file directory.
jfc

J-F C. MORFIN
UTEL, President

---

From: Tex Texin <[email-masked]>
Date: Fri, 25 Mar 2005 15:08:35 -0800

tex