Subject: [Ltru] Re: "Obsolete" region subtags

From: "Doug Ewell" <[email-masked]>
Date: Sun, 13 Mar 2005 21:54:09 -0800

Frank Ellermann <nobody at xyzzy dot claranet dot de> wrote:

>> a fair and unbiased (I hope) set of pro and con arguments:
>
> Yes, within the constraints of the draft.  It's a problem in
> the design for some cases:
>
> 1 - Region known as XX splits into XX and YY, one part inherits
>     the CC.  Old XX tags could be nonsense under the new XX,
>     example sl-YU
>
> 2 - Regions known as AA and BB join to a new AA, the union
>     inherits the CC of one of its former parts.  The registry
>     still supports old BB tags, but there's no way to identify
>     old AA tags.  Example: dsb-DD vs. fy-DE.
>
> 3 - Same case, the registry wouldn't identify AA as canonical
>     value for BB, beause it has not the same UN number.
>     Example:  YD (720) plus YE (886) are now YE (887).

Frank, I know you have additional problems with the use of ISO 3166-1
country codes in language tags.  I did summarize each of them for the
authors, each with its own set of pro and con arguments, and I can
present those to the group if desired.

I'd like to focus on one issue at a time here, and at the moment we were
talking about setting a cutoff date to restrict the use of withdrawn
country codes as region subtags.

I'd also like to draw attention to the charter, which refers
specifically to the goal of updating RFC 3066 within the general
framework of RFC 3066.  It does not provide for this group to overturn
the basic principles of RFCs 1766 and 3066, which include the use of ISO
3166-1 country codes to narrow the scope of a language.  Everyone agrees
that there are drawbacks to using these codes, but they are entrenched
in current usage, and they have mnemonic value (as you pointed out in
your other arguments).  It is not our mission to render untold numbers
of existing language tags obsolete.

There are several other issues that various people have had regarding
language tags, which are also not within the scope of this group or its
charter.  I suspect these will come out before long.

> 4 - Sometimes regions have an UN number and no CC, but one or
>     more ccTLDs.  Examples:  830 (GG), 830 (JE), 833 (IM).

Again, let me reiterate:  ISO 3166-1 country codes used to qualify
language tags are *not* the same as ccTLDs.  They do not fulfill the
same purpose.  They should not be confused for one another.

> Or the UN numbers are the solution, and ISO 3166 CCs and IANA
> ccTLDs simply mean whatever they mean "now", allowing old CCs
> in their old meaning temporarily (with a canonical UN number),
> until they get a new meaning.

Earlier, you stated that the old FQ should not be reserved in the
language subtag registry, because if ISO 3166/MA ever reassigns FQ to a
new country, its residents "will hate the obscure UN number and you."
Now you say the numbers are better than the alpha-2 codes?

Please, let's stick to the work at hand, which is to update and enhance
RFC 3066, not to replace it with a new mechanism altogether.

-Doug Ewell
 Fullerton, California
 http://users.adelphia.net/~dewell/

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Doug Ewell" <[email-masked]>
Date: Sun, 13 Mar 2005 23:59:01 -0800

Randy Presuhn <randy underscore presuhn at mindspring dot com> wrote:

> I think we have several alternatives available for initializing the
> registry:
>    1) pointer to external file/web site (current proposal in
>       -registry-00.txt
>    2) reference to WG internet-draft handed to IANA
>    3) reference to WG RFC containing initial contents
>    4) embed initialization data into appendix of registry document
>
> Did I miss some?  My biggest concern about 2-4 is that the quirks
> of i-d posting and rfc publication  formats would require us to spend
> time on aspects of registry entries that we might otherwise not
> have to spend.

I suppose I should clarify that I'm willing to bend and twist the
proposed registry into pretty much any shape the WG desires.  XML,
comma-quote, Excel, Frank's tabular format, whatever.

My main concern is the one Randy sort of alluded to, that we might get
bogged down in the details of formatting when we have bigger issues to
tackle.

I should also point out that the format of the registry has been
discussed before, on ietf-languages.  The draft originally specified
that the records were to be semicolon-delimited, but this posed a
problem with ISO 639 descriptions which contained semicolons, such as
"Catalan; Valencian".  Although the descriptions are not normative, we
didn't feel it was a great idea to modify or truncate them simply to fit
the file format, and enclosing such names in quotes would have been
heavier-weight than simply changing the file format to bar-delimited to
avoid the problem.

IIRC, we did briefly discuss XML, but I don't recall the details of that
discussion.

I guess the only reasonable plain-text format I would really object to
is tab-delimited.  Although some Unix developers are really fond of this
format, it doesn't seem to hold up well in certain Internet and mail
environments where tabs get expanded to spaces.  This might be a major
problem since the registry will be available online, and probably will
be mirrored unofficially on numerous sites.

-Doug Ewell
 Fullerton, California

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 14 Mar 2005 09:33:23 +0100

Doug Ewell wrote:

> I'd like to focus on one issue at a time here, and at the
> moment we were talking about setting a cutoff date to
> restrict the use of withdrawn country codes as region
> subtags.

Okay, we can discuss it in this order, but we can also discuss
it simultaneously.  The cutoff date could solve most immediate
problems, but not the general problem.  So if we'd decide that
there is a general problem _and_ fix it somehow, then we would
not need to discuss a cutoff date.

> specifically to the goal of updating RFC 3066 within the
> general framework of RFC 3066.

It apparently allows to keep the 1766 / 3066 idea as is, which
is by definition backwards compatible, and to use UN M 49 codes
for persistent tags.  I could use en-GB for something that is
at least valid for one day plus five years - the old ISO 3166
grace period, but you said that they plan to extend this to 50
years, and of course GB won't cease to exist tomorrow.

I could also use en-826 when en-GB isn't stable enough for my
purposes.  If it's against the charter of this WG to discuss
this idea, then I'm really lost and probably wasting our time.

> It is not our mission to render untold numbers of existing
> language tags obsolete.

Sure, but existing 3066 and future 3066bis tags could coexist.

If en-833 is allowed (actual draft) and doesn't cause havoc for
existing 3066 implementations, then the same would be true for
en-826.  And en-GB could stay what it always was, no absolutely
guaranteed persistence, but good enough as long as GB is a
country code plus 5 or plus 50 years, whatever ISO decides.

Plus the time until they really decide to recycle it for some
new GB.  Most probably the meaning of "en" is so far from
today's "en" if and when that happens, that the stability of
GB in an old en-GB is the least problem with this tag.

> Again, let me reiterate:  ISO 3166-1 country codes used to
> qualify language tags are *not* the same as ccTLDs.  They do
> not fulfill the same purpose.

Yes, they are not exactly the same, but the differences are
limited to a few odd cases.  And we're talking about a future
IANA registry here, they already have a registry for ccTLDs
based on ISO 3166.  I trust that IANA gets it right no matter
what ISO 3166 does.

And I'm convinced that normal Internet users will try to use
their ccTLD in language tags no matter what RfC 3066bis says.
Why not sanction this expected behaviour instead of trying to
forbid it ?

> Now you say the numbers are better than the alpha-2 codes?

It's an idea to support stable tags with numbers and less than
perfect ccTLD tags in the same draft simultaneously.  With full
backwards compatibility for all tags valid when RfC 1766 was
published, but without guaranteed stability.  For guaranteed
stability 3066bis could offer the UN M 49 numbers as a feature.

This could solve all remaining problems I had with draft 10.

No ??-FQ or ???-FQ tags exist today, and when FQ is introduced
it would be what the users expect it to be in a language tag of
their century.  Or their millennium in the case of a future FQ.

                          Bye, Frank

---

From: John Cowan <[email-masked]>
Date: Mon, 14 Mar 2005 08:37:40 -0500

> comma-quote, Excel, Frank's tabular format, whatever.

"Record-jar" format as described at
http://www.catb.org/~esr/writings/taoup/html/ch05s02.html#id2906931
might be a good choice.  Essentially, it uses RFC 2822 style headers
(with foldable whitespace) for each entry in the registry, and a line
containing "%" (more robust than a blank line) to separate entries.
It allows fairly arbitrary data in a tall and narrow layout.

-- 
While staying with the Asonu, I met a man from      John Cowan
the Candensian plane, which is very much like       [email-masked]
ours, only more of it consists of Toronto.          http://:www.ccil.org/~cowan
        --Ursula K. Le Guin, Changing Planes

---

From: John Cowan <[email-masked]>
Date: Mon, 14 Mar 2005 08:47:34 -0500

Frank Ellermann scripsit:

> purposes.  

We don't want to allow both en-GB and en-826 at the same time.
That would be very bad for interoperability.
Indeed, the latter would never become valid, even if the ISO 3166 MA
were to reassign "GB".

> en-826.

Syntactically speaking, yes.  Semantically, it would be a disaster.
The reason en-833 is not a problem is that it refers to a variety
variety of English that is geographically restricted in a way
that does not correspond to any ISO 3166 code.

> based on ISO 3166.  I trust that IANA gets it right no matter

Historically IANA has slavishly followed the whims of the ISO 3166 MA.

> Why not sanction this expected behaviour instead of trying to

Because it's wrong and not backward compatible.

-- 
John Cowan  [email-masked]  www.reutershealth.com  www.ccil.org/~cowan
Heckler: "Go on, Al, tell 'em all you know.  It won't take long."
Al Smith: "I'll tell 'em all we *both* know.  It won't take any longer."

---

From: "Randy Presuhn" <[email-masked]>
Date: Mon, 14 Mar 2005 11:47:42 -0800

Hi -

(co-chair hat ON)

> From: "Doug Ewell" <[email-masked]>
> To: "LTRU Working Group" <[email-masked]>
> Cc: "Frank Ellermann" <[email-masked]>
> Sent: Sunday, March 13, 2005 9:54 PM
> Subject: [Ltru] Re: "Obsolete" region subtags
...
> present those to the group if desired.
...

Let's keep technical discussions (within the scope of our charter) on
the WG mailing list.  In particular, since Addison Phillips and Mark Davis
are serving as *editors* of the registry i-d, they now take their direction
from the WG, even as we recognize their contribution as authors of
the material that went into that -00 version, and as contributors to the
technical discussion.

Randy

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 15 Mar 2005 05:03:34 +0100

John Cowan wrote:

> We don't want to allow both en-GB and en-826 at the same
> time.  That would be very bad for interoperability.

There's no problem with interoperability, if all CCs have a
canonical and stable UN number as long as the CC exists.

The draft already has many deprecated entries with pointers
to new and stable canonical entries.

For the big CS mess the registry would have been:

region | CS | ... | 200
region | YU | ... | 890

Snapshot after the split of CS:

region |  CS | ... | 200 | deprecated
region |  CZ | ... | 203
region |  SK | ... | 703
region |  YU | ... | 890
region | 200 | ... | | deprecated, use 203 or 703

Snapshot after the split of YU:

region |  BA | ... | 070
region |  CS | ... | 200 | deprecated
region |  CZ | ... | 203
region |  HR | ... | 191
region |  MK | ... | 807
region |  SI | ... | 705
region |  SK | ... | 703
region |  YU | ... | 891 | note change from 890
region | 200 | ... | | deprecated, use 203 or 703
region | 890 | ... | | deprecated, use 070, 191, 705, 807, 891

Snapshot after the premature recycling of CS:

region |  BA | ... | 070
region |  CS | ... | 891
region |  CZ | ... | 203
region |  HR | ... | 191
region |  MK | ... | 807
region |  SI | ... | 705
region |  SK | ... | 703
region |  YU | ... | 891 | deprecated
region | 200 | ... | | deprecated, use 203 or 703
region | 890 | ... | | deprecated, use 070, 191, 705, 807, 891

As you see the canonical UN number never changes, even if it's
deprecated.  And the CC is only an alias of a canonical value,
always reflecting the official ISO 3166 meaning, where it
exists.  Kept as deprecated alias like e.g. CS until ISO 3166
assigned a new official value.

All the tag reviewer has to do, is to figure out the UN number.

 [GB]
> the latter would never become valid, even if the ISO 3166 MA
> were to reassign "GB".

You're talking about the draft as it is, but I claim that it's
broken, and propose a fix.  With this fix GB could be assigned
to a new region when necessary (after a long transition period
like at least 50 years).
                            Bye, Frank