Subject: RE: [Ltru] Re: [psg.com #954] all aliases must be equally supported

From: "Addison Phillips" <[email-masked]>
Date: Wed, 11 May 2005 09:33:17 -0700

This is very hard to follow. I think that it would be better to spell out the principles and enshrine those in the text. Basically this is the problem we've been using NH->VU to illustrate.

Again, I see several options.

1. The draft currently says that "once canonical/always canonical" and spells out how to determine whether to add a new ISO 3166 code or not. In this set of rules, if the NH->VU transition occurred today then the registry would look like:

Subtag: NH
Description: Vanuatu

2. The registry could do as Frank suggests and track changes to ISO 3166, making codes non-canonical as country names evolve. In this case the registry would look like:

Subtag: NH
Description: New Hebrides
Deprecated: yyyy-mm-dd
Canonical: VU
%%
Subtag: VU
Description: Vanuatu

This has the effect of "invalidating" tags in existing content in canonical processors.

3. The registry could also do as I've suggested and deprecate but not map.

Subtag: NH
Description: New Hebrides
Deprecated: yyyy-mm-dd
%%
Subtag: VU
Description: Vanuatu

4. The registry, finally, could deprecate and map, but not using the canonical mechanism (once canonical, always canonical):

Subtag: NH
Description: New Hebrides
Deprecated: yyyy-mm-dd
Use: VU
%%
Subtag: VU
Description: Vanuatu

Personally, I don't think there is much to recommend #1 aside for absolute stability of codes. I don't care for #2 because I *do* want codes to remain canonical for their given meaning. Either #3 or #4 seem like good choices to me. (Frank has previously argued that #3 is not a good choice because there is no machine readable mechanism for indicating what to use)

Some comments below.

Addison P. Phillips
Globalization Architect, Quest Software
Chair, W3C Internationalization Core Working Group

Internationalization is not a feature.
It is an architecture. 

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Frank Ellermann
> Sent: 2005?5?10? 23:39
> To: [email-masked]
> Subject: [Ltru] Re: [psg.com #954] all aliases must be equally supported
> 
> Randy Presuhn wrote:
> 
> > Could you suggest some specific replacement text that would
> > address your concerns?
> 
> It was point (4) of five on my ToDo list in:
> 
> <http://mid.gmane.org/[email-masked]>
> 
> For point (1) Doug and Addison apparently prefer to get an
> RfC for the registry draft _before_ discussing the final
> draft registry (i.e. determine more / other Suppress-Scripts).
[Addison Phillips] 

Speaking for myself: I would prefer to see the draft registry track text in the registry draft and not the other way around. Because the rules in the text above are all "after Date B" rules, this particular item doesn't affect the registry (although one might apply these rules to existing deprecated codes from the post-Date A era)
> 
> That strategy surprised me, but it could be a good idea for
> my point (5) in:
> 
> <http://mid.gmane.org/[email-masked]>
> 
> Point (2) is actually something for you, I wanted a SHOULD
> with rationale for the Suppress-Script issue, you found that
> what I want is in fact a "MUST NOT unless" with rationale,
> and that's still missing in draft -01.
[Addison Phillips] 

You refer to the text:

<q>
Suppress-Script contains a script subtag that SHOULD NOT be used to form language tags with the associated primary language subtag. This field MUST only appear in records whose 'Type' field-value is 'language'. See Section 4.1 (Choice of Language Tag).
</q>

But, in fact, the rationale appears slightly lower in the document:

<q>
The field 'Suppress-Script' MUST only appear in records whose 'Type' field-value is 'language'. This field may appear at most one time in a record. This field indicates a script used to write the overwhelming majority of documents for the given language and which therefore adds no distinguishing information to a language tag. For example, virtually all Icelandic documents are written in the Latin script, making the subtag 'Latn' redundant in the tag "is-Latn".
</q>

Additional explanatory text about Suppress-Script could be added, but this is really a separate item from issue #954.
> 
> That leaves point (4) "canonical", and admittedly I was too
> lazy to find all occurences of "canonical" in the text so
> far.  An incomplete attempt:
> 
> Line 512, stick to canonical alpha-3 tag even if an alpha-2
> is added => no problem, that only tries to avoid unnecessary
> aliases.
> 
> Line 630, UN region numbers are the final canonical value
> for any given region with "ambiguous" alpha-2 codes => okay.
> 
> Line 1088, never add Canonical => delete
> Line 1089, never modify Canonical => delete
> 
> That's the paragraph defining the problematic "canonical
> forever" concept, it should be deleted.
> 
> Line 1095: get rid of "STRONGLY" (fourth reminder, IIRC)
> 
> Line 1112:
> - Although valid in language tags, subtags and tags with a 'Deprecated'
> - field are deprecated and validating processors SHOULD NOT generate
> - these subtags.  Note that a record that contains a 'Deprecated' field
> - and no corresponding 'Canonical' field has no replacement mapping.
> +
> + Although valid in language tags, subtags and tags with a 'Deprecated'
> + field SHOULD NOT be generated by validating processor.  Note hat a
> + record that contains a 'Deprecated' field and no corresponding
> + 'Canonical' field has no replacement mapping.
> +
> + Future updates of the source standards may replace former canonical
> + tags by new tags.  In that case the old tag is marked as 'Deprecated'
> + with a pointer to the the new tag in the 'Canonical' field.  Pointers
> + to the old canonical value in other tags are updated.  Validating
> + processors SHOULD use the the most recent registry.
> 
> The latter SHOULD is no new restriction, it was always implied.
> 
> Line 1242:
> 
> | Values in the fields 'Type', 'Subtag', 'Tag', 'Added' and
> | 'Canonical' MUST NOT be changed and are guaranteed to be
> | stable over time.
> 
> s/'Added' and 'Canonical'/and 'Added'/
> In theory we hope that that 'Deprecated' is guaranteed to be
> stable, but in practice not mentioning this allows Murphy to
> rule and SNAFU on the side of the source standards.
[Addison Phillips] 

Good catch.
> 
> Line 1277 - 1297:  Modify the procedures for withdrawn / added
> tags to be reflected as deprecated / canonical in the registry.
> 
> Remove IM example (it could be still correct dependig on how
> we want to handle the introduction of alpha-2 codes for regions
> formerly only known by their number, but it's no example for
> "canonical reflects the sources").  BTW, if GG and JE would be
> added this cries for a judgement call.  Insisting on 830 as a
> canonical value for both GG and JE would be ridiculous.  And
> not reflect Doug's undocumented "same piece of land" concept.
> 
> Etc., e.g. line 1355, there are more lies, where the "canonical
> forever" concept is just wrong, forcing users to consider an
> obsolete tag for their region as "canonical" will never fly.
> 
> I would never follow this idea no matter what any "least common
> practice" states.  It's ZW, not RH.  If I look into the source
> standard I want a decent chance to guess what the canonical tag
> might be without unnecessary surprises in the 3066bis registry.
> 
>                              Bye, Frank
> 
> 
> 
> _______________________________________________
> Ltru mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ltru

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 11 May 2005 19:49:15 +0200

Addison Phillips wrote:

> I think that it would be better to spell out the principles
> and enshrine those in the text.

ACK.

> Basically this is the problem we've been using NH->VU to
> illustrate.

Also fine, nothing so exotic as CS.

 [scenario 1]
> then the registry would look like:

> Description: Vanuatu

Why didn't you add VU as in (4) ?  In 3.3 "rule 9" you say
"add" if there's no conflict.  So scenario (1) reflecting
draft -01 should be:

| Subtag: NH
| Description: Vanuatu
| %%
| Subtag: VU
| Description: Vanuatu
| Canonical: NH

"Rule 9" is the item with example IM.  Read NH for 833, and
VU for IM.

 [scenario 2]
> Description: New Hebrides

Yes, that's what I'd prefer.  Updating any old Canonical: NH
elsewhere to Canonical: VU, because the registry is no lesson
in graph theory.

> content in canonical processors.

No, not "invalidating", NH is still valid, in all scenarios.

Maybe "decanonalizing", but a validating processor using an
obsoleted registry is always in trouble, not only for this
scenario, also for completely new entries.

 [scenario 3]
> Description: New Hebrides

That's almost the same as (2), but no pointer from NH to VU
or v.v., so that would be worse than (1) or (2).  All aliases
should have exactly one canonical value.

 [scenario 4]
> Description: New Hebrides

Same problem as (3), unless "Use: VU" is a new kind of pointer
in addition to "Canonical:".  Then it's almost the same (2).

I don't see any advantage in comparison with (2), but (4) is
better than (1).  The "validating processors need a 'canonical
forever' value" argument is IMHO wrong, a validating processor
always works on its own old, potentially obsolete, view of the
registry.  And if it's updated with a new registry it can as
well integrate the complete diif, not only new records.

What you lose in (2) is "once tagged canonically as long as
possible canonical".  I don't see where that's a problem.  If
it really is a problem, (4) with an ersatz-canonical "Use: VU"
could solve it.

> I don't care for #2 because I *do* want codes to remain
> canonical for their given meaning.

But why ?  Who needs this, it's only confusing if "canonical"
differs from the source standard, and otherwise irrelevant,
because NH -> VU or VU -> NH establishs the "alias" relation.

> Either #3 or #4 seem like good choices to me. (Frank has
> previously argued that #3 is not a good choice because there
> is no machine readable mechanism for indicating what to use)

(3) is the only scenario without a documented alias relation,
therefore it's the worst scenario,

(2) is more or less what we do for all existing records before
the final date-B.  If it is what we want before date-B I don't
see why we should stop to want it after date-B.

 [missing MUST NOT Suppress-Script and rationale]
> the rationale appears slightly lower in the document:

The reason for this ugly kludge is backwards compatibility.

That reason is not stated in what you have.  It's not only
Doug who sees that it's ugly like hell, I see it too.  But
unfortunately it's necessary.  Otherwise the effects with
legacy tools and/or for legacy content are hard to predict.

                        Bye, Frank