Subject: RE: [Ltru] Re: [psg.com #954] all aliases must be equally support ed

From: Misha Wolf <[email-masked]>
Date: Wed, 18 May 2005 20:54:24 +0100

Ira McDonald wrote:

> First, aliases are _bad_ for the reliability of any protocol
> or interface.  The so-called 'preferred MIME name' label on
> aliases in the IANA Charset Registry is useless and dangerous.

We must insist that those creating tags use canonical values.
Any other approach will lead to a broken Internet.

Misha

-----------------------------------------------------------------
        Visit our Internet site at http://www.reuters.com

To find out more about Reuters Products and Services visit http://www.reuters.com/productinfo 

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Addison Phillips" <[email-masked]>
Date: Thu, 19 May 2005 11:02:24 -0700

Not true. Here is the governing text:

# For fields of type 'language', 'extlang', 'script', 'region', and 'variant', a canonical mapping of this record to a subtag record of the same 'Type'.
# For fields of type 'grandfathered' and 'redundant', a canonical mapping to a complete language tag.

"no-nyn" is of type "grandfathered", thus it is a complete language tag and thus "nn" is of type "language".

Addison

Addison P. Phillips
Globalization Architect, Quest Software
Chair, W3C Internationalization Core Working Group

Internationalization is not a feature.
It is an architecture. 

> -----Original Message-----
> From: John Cowan [mailto:[email-masked]]
> Sent: jeudi 19 mai 2005 11:00
> To: Addison Phillips
> Cc: [email-masked]
> Subject: Re: [Ltru] Re: [psg.com #954] all aliases must be equally support
> ed
> 
> Addison Phillips scripsit:
> 
> >    <t>Redundant or grandfathered tags that have a Canonical-Value
> mapping
> >       in the IANA registry (see <xref target="ianaformat"/>) SHOULD
> (MUST??)
> >       be replaced with their mapped value. These items are either
> >       deprecated mappings created before the adoption of this document
> >       (such as the mapping of "no-nyn" to 'nn' or "i-klingon" to 'tlh')
> >       or are the result of later registrations or additions to this
> >       document (for example, "zh-gouyu" might be mapped to a
> >       language-extlang combination such as "zh-cmn" by some future
> >       update of this document).</t>
> 
> This still suffers from the problem that we don't know what the Canonical-
> Value
> is used for.  For example, there is no indication whether "no-nyn" is
> mapped
> to the *language* tag 'nn' or to the *region* tag 'nn' (not currently
> defined,
> but could be).  A subtag will be canonicalized to a subtag of the
> same type, but a grandfathered tag could be canonicalized to any kind of
> subtag or a combination of subtags.  We need a provision for this.
> 
> (Credit: Doug has been making this point repeatedly.)
> 
> --
> Don't be so humble.  You're not that great.             John Cowan
>         --Golda Meir
> [email-masked]

---

From: "Addison Phillips" <[email-masked]>
Date: Thu, 19 May 2005 16:33:59 -0700

Based on this and what I know of other's positions, I'm guessing that we are near to a consensus (at least, on how to proceed towards a resolution). Because this will a number of changes to the text which may be difficult to unwind later, I'm prepping draft-02 for submission tonight so that we have the items resolved in it "bookmarked". (So draft-03 will contain an attempt at resolving this largish issue)

Addison

Addison P. Phillips

> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Mark Davis
> Sent: 2005年5月19日 16:03
> To: John Cowan
> Cc: [email-masked]
> ok, I can live with that for the grandfathered stuff.
> 
> ‎Mark
> 
> ----- Original Message -----
> From: "John Cowan" <[email-masked]>
> To: "Mark Davis" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Thursday, May 19, 2005 13:37
> > Mark Davis scripsit:
> > > I have to look this over in detail, but a crucial requirement is to
> > > guarantee the invariant:
> > >
> > > if isValid(tag, date1), then canonicalize(tag, date1) =
> canonicalize(tag,
> > > date2), for all date2 > date1
> > >
> > > Does this still preserve that?
> >
> > It does not preserve that invariant for grandfathered tags: they remain
> > always valid but not necessarily always canonical.
> >
> > I think we should be able to live with that, as the alternative is very
> ugly:
> > we will be stuck with some linguistic objects that can never have
> generative
> > names.
> >
> > --
> > John Cowan  [email-masked]  www.ccil.org/~cowan
> www.reutershealth.com
> > Any sufficiently-complicated C or Fortran program contains an ad-hoc,
> > informally-specified bug-ridden slow implementation of half of Common
> Lisp.
> >         --Greenspun's Tenth Rule of Programming (rules 1-9 are unknown)
> >
> >
> 
> 
>

---

From: "Addison Phillips" <[email-masked]>
Date: Fri, 20 May 2005 09:59:25 -0700

I think the problem here is that when regions change identity and get a new code, they generally *are* somehow different. The "old country" in some sense no longer exists, even if the new entity shares the same borders. Does Zaire exist? Does Rhodesia? 

So the "Use" value is not really a canonicality problem. We can weaken the recommendation even more to be more accurate:

> >    <t>Subtags of type 'Region' that have a Canonical-Value mapping
> >       in the IANA registry (see <xref target="ianaformat"/>) MAY be
> >       replaced with their mapped value when performing comparison
          of tags. It should be noted that this may change the meaning of
          the tag.</t>

Addison P. Phillips

> Behalf Of McDonald, Ira
> Sent: 2005年5月20日 9:53
> To: 'Doug Ewell'; LTRU Working Group
> Subject: RE: [Ltru] Re: [psg.com #954] all aliases must be equally support
> ed
> 
> Hi,
> 
> Doug's right - we need to find concensus here and move on.
> 
> I think I've been guilty of a mis-conception here.  I admit my
> comments were colored by having worked on specifications and
> software based on the IANA Charset Registry - which has a 10-to-1
> ratio of aliases to base charset tags - the use of charset tags
> in MIME content has been a thorough failure.
> 
> If indeed language subtag aliases are _rare_, then the requirement
> for support of all aliases is fine.
> 
> Canonicalization of langugage tags (by exercise of an algorithm,
> not by reference to a copy of the IANA Language Subtag Registry)
> should be a SHOULD for content generation and content queries
> (because it's likely to reduce processing errors by less than
> perfect software).
> 
> I'm perfectly happy to lose the problematic 'Canonical:' field
> in the registry and replace it with 'Use:'.  The idea of 'once
> canonical, forever canonical' is wrapping us all around the
> axle, but not visibly helping.
> 
> The problem we're wrestling with is that our ISO/UN sources are
> _not_ entirely stable and we intuitively think we should try to
> achieve better stability in the IANA Language Subtag Registry.
> We're probably catching at moonbeams...
> 
> Cheers,
> - Ira
> 
> Ira McDonald (Musician / Software Architect)
> Blue Roof Music / High North Inc
> PO Box 221  Grand Marais, MI  49839
> phone: +1-906-494-2434
> email: [email-masked]
> 
> > From: [email-masked]
> > [mailto:[email-masked]]On
> > Behalf Of Doug Ewell
> > Sent: Friday, May 20, 2005 2:47 AM
> > To: LTRU Working Group
> > Subject: [Ltru] Re: [psg.com #954] all aliases must be
> > equally supported
> >
> >
> > McDonald, Ira <imcdonald at sharplabs dot com> wrote:
> >
> > > Whenever possible, the language tag registry draft should not
> > > encourage the proliferation of aliases for tags or subtags.
> >
> > Misha Wolf <Misha dot Wolf at reuters dot com> wrote:
> >
> > and yet Frank Ellermann <nobody at xyzzy dot claranet dot de> says:
> >
> > > The complete concept of "Canonical:" appears to be
> > > wrong, if you even think about any "let's forget the rest".
> > > ...
> > > Supporting only an obsolete idea of "Canonical:" as stated in
> > > an obscure IANA registry not reflecting the state of the art
> > > in the source standards after date-B was a worse idea.
> >
> > IMHO, this is the single biggest technical stumbling block in the
> > project.  The draft has always taken the middle road by saying that
> > there were canonical subtags, which would not change dammit, and then
> > there were aliases which could be used (but shouldn't), and must be
> > recognized as equivalent to the canonical.
> >
> > Now we have some saying there should be no aliases at all, only
> > canonicals, and others saying the whole notion of canonical
> > stability is
> > "an obsolete idea."  Folks, if we don't find a way to resolve this, we
> > are nowhere.
> >
> > --
> > Doug Ewell
> > Fullerton, California
> > http://users.adelphia.net/~dewell/
> >
> >
> >

---

From: "Addison Phillips" <[email-masked]>
Date: Sat, 21 May 2005 10:46:57 -0700

That turns out not to be quite true. Mark and I had a chat last night about this issue, which, it turns out, hinges on what canonicalization of a language tag means.

What we discussed was basically that the region subtag one chooses ("Choice of Language Tag") does evolve along the lines of the "Use" concept. It makes eminent sense and the compromise I suggested earlier in the week is that way to embody that in the draft.

Then there is the question of canonicalization. Canonicalization for what? For comparison and matching, of course. If you look at the way the Use or Canonical-Value concept is set up, there is a date (in the Deprecated field) associated with each mapping. This means that:

1. For any given tag, you can construct what the comparable tag would have been for any given earlier date.

2. For any given tag, you can construct the set of comparable tags that are valid at any given date (up to the date of the registry).

3. The purpose for doing this is to compare tags with ranges. In other words, this text should appear in the matching draft. (Whether it appears in the registry draft is a separate matter, please note).

This means that the Use/Canonical-Value setup as described in the email thread during this past week *does* meet Mark's requirements for "canonicalization" because the data for supporting the relationships he cares about is in the registry. The next step is to incorporate the text into the draft, which I will set about doing over this weekend, so we can see what it looks like.

Any thoughts on whether to keep the canonicalization section in the registry draft or just have it in matching?

Addison

Addison P. Phillips

> Behalf Of Frank Ellermann
> Sent: 2005?5?21? 0:23
> To: [email-masked]
> Subject: [Ltru] Re: [psg.com #954] all aliases must be equally support ed
> 
> McDonald, Ira wrote:
> 
> > The problem we're wrestling with is that our ISO/UN sources
> > are _not_ entirely stable and we intuitively think we should
> > try to achieve better stability in the IANA Language Subtag
> > Registry.
> 
> My intuition was and is "mirror the sources as long as possible
> under the rule of stability".  Plus an emergency exit for cases
> like CS etc.
> 
> ...some cosmic event like the CS disaster, yes.  It's rare.
> The "Use:" concept is IMHO a pretty simple way to survive it.
> But "Use:" is dynamic, not static, and Mark hates it.  Bye.
> 
> 
>

---

From: "Addison Phillips" <[email-masked]>
Date: Sun, 22 May 2005 10:26:01 -0700

~sigh~ We're all in raging agreement here.

The proposal is to have "canonical" mappings that track ISO 3166. Subtags that are deprecated (i.e. withdrawn by ISO 3166) remain valid in language tags (in ISO 3166 they disappear, but in language tags they remain in the registry with a notation showing the date they were withdrawn by ISO 3166). 

Language tags that used (notice the tense) the old code remain valid and in "canonical form" (whatever that means). There is an ability to figure out from the registry what the "modern code" is. Notice that sometimes when a country changes its code there is also a real, practical change in the national entity. And sometimes it is just a code change. How close or far this falls from national identity and emotional connection issues depends on each case.

Let's do the text and see how well that suits everyone's needs.

Addison

Addison P. Phillips

> Sent: 2005年5月21日 23:13
> To: LTRU Working Group; Frank Ellermann
> I was trying to describe a compromise approach. I can't make any sense of
> what you are trying to say below, but as I said, it doesn't much matter.
> We
> can support both formulations, a stable canonical form and an unstable
> (pick
> latest) one, the one you want.
> 
> ‎Mark
> 
> From: "Frank Ellermann" <[email-masked]>
> To: <[email-masked]>
> Sent: Saturday, May 21, 2005 13:06
> > Mark Davis wrote:
> > > For Frank (and Ira?), not having the latest codes be in the
> > > canonical form is (I guess) not fashionable (ok, I'm being a
> > > bit flip, but I really don't know what their rationale is).
> >
> > User1: My cc just changed from GE to XX...
> > M.D. : Don't panic, GE is still valid.
> > User1: But the new government is very XX.
> > M.D. : Forget governments.  This is the Internet.  Not a zoo.
> > User1: But ISO 3166 also says XX.
> > M.D. : ISO says many things.  IANA knows what's good for you.
> > User2 (interrupting): IANA has GG, can I use it ?
> > M.D. : That's a ccTLD and no country code.
> > User1: So I should use GE instead of XX ?
> > M.D. : Yes.
> > User2: I want GG.
> > M.D. : I'm only talking with User1, *plonk*
> > User1: Thanks, and there won't be a problem with GE ?
> > M.D. : Of course not, it's stable and canonical forever.
> >
> > [some years later, Georgia finds that GE is free and grabs it]
> >
> > User3: What are these crazy XX-bastards doing, are they mad ?
> > User2 (interrupting): I want GG.
> > User1: GE is XX.
> > User3: No, GE is Georgia.  Idiot.
> > User2: It's not GG.
> > User3: GE is Georgia. *plonk*
> > User1: No, it's XX.
> > M.D. : Wait a moment.
> > User3: We waited 50 years, that's long enough.
> > User1: No, GE is XX, I was 20 years in prison for saying GE.
> > User3: I've already told you that you're an idiot.
> >
> >                          Bye, Frank
> >
> >
> >