Subject: [Ltru] Can't add a Preferred-Value to a redundant tag

From: "Doug Ewell" <[email-masked]>
Date: Mon, 11 Sep 2006 23:48:18 -0700

I've been working through the ISO 639-3 code elements for sign 
languages, converting them into extlang subtags and correlating them 
with the existing "sgn-XX" redundant tags, and have run into a problem.

As I understand it, the plan is to deprecate the "sgn-XX" tags in favor 
of the new "sgn-xxx" language-extlang combinations, meaning that we 
would give "sgn-XX" a Preferred-Value of "sgn-xxx".  (Remember that the 
Preferred-Value for a tag must be a tag, not a subtag.)  However, 
according to Section 3.1 we cannot do this:

    The field 'Preferred-Value' MUST NOT be modified once created in the
    registry.  The field MAY be added to records of type "grandfathered"
    and "region" according to the rules in Section 3.3.  Otherwise the
    field MUST NOT be added to any record already in the registry.

I don't know why we allowed grandfathered tags to be assigned a 
Preferred-Value at a later date, but did not allow the same for 
redundant tags.

As a reminder, these 'sgn-XX" tags are the only redundant tags that were 
*already* redundant at the time they were registered under RFC 3066. 
The others contained script subtags, or some other components which were 
not allowable RFC 3066 subtags.

We seem to have three options (using American Sign Language as our 
example):

1.  Change the wording in Section 3.1 to allow redundant tags to be 
assigned a Preferred-Value.  This way, "sgn-US" could be deprecated and 
assigned a Preferred-Value of "sgn-ase".  The disadvantage is that there 
might have been a perfectly good reason for disallowing this in the 
first place.

2,  Do not change the wording, and add "ase" as a deprecated-as-birth 
extlang subtag.  We cannot assign it a Preferred-Value of "sgn-US" as we 
might like, since the P-V for any subtag must be a subtag of the same 
type.  This would mean two tagging possibilities for the same 
Description, one of which would be deprecated but would not point to the 
other.

3.  Do not change the wording, and do not add "ase" at all.  The 
disadvantage of this is that coding for sign languages becomes 
haphazard; American Sign Language would be "sgn-US" but Cuban Sign 
Language would be "sgn-csf".  We would have to justify which 639-3 sign 
languages are added on the basis of which redundant tags exist.

Three guesses which solution I prefer.

--
Doug Ewell
Fullerton, California, USA
http://users.adelphia.net/~dewell/
RFC 4645  *  UTN #14

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: Addison Phillips <[email-masked]>
Date: Tue, 12 Sep 2006 08:49:40 -0700

> Preferred-Value at a later date, but did not allow the same for 

Because redundant tags are, well, redundant. That is, "sgn-BE" will 
still be both "well-formed" and "valid" regardless of what we do with 
extlangs. In fact, it expresses an entirely different value than 
"sgn-bvs" does. One is "Belgian Sign Language", a specific language. The 
other is "sign language(s) as used in Belgium".

> Description, one of which would be deprecated but would not point to the 

We will have to decide about this particular point.

> languages are added on the basis of which redundant tags exist.

I don't know about "justification". We have here a problem with a 
registered tag "arriving first". However... I do note that we have an 
out for the registered-but-outmoded stuff. We can change the Description 
field to reflect the more generic meaning (remember: *broadening* of 
meaning is allowed!).

Thus:

Tag: sgn-US
Description: Sign Language as used in the USA
Comments: originally registered for ASL. "sgn-ase" is recommended
   for that meaning.

Addison
-- 
Addison Phillips
Globalization Architect -- Yahoo! Inc.

Internationalization is an architecture.
It is not a feature.

---

From: "Doug Ewell" <[email-masked]>
Date: Tue, 12 Sep 2006 20:05:18 -0700

Addison Phillips <addison at yahoo dash inc dot com> wrote:

> "sgn-bvs" does. One is "Belgian Sign Language", a specific language. 
> The other is "sign language(s) as used in Belgium".

That was my objection all along to this scheme.

>> ... This would mean two tagging possibilities for the same 
>> Description, one of which would be deprecated but would not point to 
>> the other.
>

I'd really prefer not to have any more deprecated-at-birth subtags than 
absolutely necessary.

> However... I do note that we have an out for the 
> registered-but-outmoded stuff. We can change the Description field to 
> reflect the more generic meaning (remember: *broadening* of meaning is 
> allowed!).
>
> Thus:
>
> Tag: sgn-US

I can live with that, notably because it doesn't require the Comments 
field to contain critical, normative information.  I'd suggest spelling 
out "ASL" since we would be doing this for several entries:

Comments: originally registered for American Sign Language; use
  "sgn-ase" instead

Overall, though, if you can get the Reviewer to buy off on this 
approach, I am fine with it.

---

From: John Cowan <[email-masked]>
Date: Wed, 13 Sep 2006 00:24:27 -0400

> other is "sign language(s) as used in Belgium".

Actually, it doesn't.  If you go back to RFC 3066, it specifically allows
ietf-languages to register tags like "sgn-BE" that look compositional
but don't mean what they say.  Paragraph 2 of section 3:

   This procedure MAY also be used to register information with the IANA
   about a tag defined by this document, for instance if one wishes to
   make publicly available a reference to the definition for a language
   such as sgn-US (American Sign Language).

So RFC 3066 and its registry between them explicitly say that sgn-US does
not mean "sign languages spoken in the US", but the specific language
called "American Sign Language" (which, BTW, is also used in Canada and
perhaps elsewhere).  Due to grandfathering, this is still true under
RFC 4646.

Since "sgn-US" does not have compositional meaning, by deprecating it
in favor of "sgn-ase" we free it up to mean what it appears to mean.

> Comments: originally registered for ASL. "sgn-ase" is recommended

That's as much as to say the original meaning is deprecated.  So
let's say so openly, and supply the new tag for that meaning using
the mechanism designed for that, namely Preferred-Value.

-- 
John Cowan    <[email-masked]>     http://www.ccil.org/~cowan
But no living man am I!  You look upon a woman.  Eowyn I am, Eomund's daughter.
You stand between me and my lord and kin.  Begone, if you be not deathless.
For living or dark undead, I will smite you if you touch him.

---

From: John Cowan <[email-masked]>
Date: Wed, 13 Sep 2006 01:27:24 -0400

> might have been a perfectly good reason for disallowing this in the 

+1

-- 
John Cowan  [email-masked]    http://ccil.org/~cowan
Big as a house, much bigger than a house, it looked to [Sam], a grey-clad
moving hill.  Fear and wonder, maybe, enlarged him in the hobbit's eyes,
but the Mumak of Harad was indeed a beast of vast bulk, and the like of him
does not walk now in Middle-earth; his kin that live still in latter days are
but memories of his girth and his majesty.  --"Of Herbs and Stewed Rabbit"

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 13 Sep 2006 16:55:19 +0200

John Cowan wrote:

 [3066 sgn-US]
> Due to grandfathering, this is still true under RFC 4646.

Okay, we got it as syntactically "redundant", but in fact
it's semantically "grandfathered".  We should say so in a
new "sgn" paragraph of the section explaining "redundant"
vs. "grandfathered".

> That's as much as to say the original meaning is deprecated.
> So let's say so openly, and supply the new tag for that
> meaning using the mechanism designed for that, namely
> Preferred-Value.

+1.  Some options:

1: Reclassify sgn-XX as exceptionally "grandfathered".
2: Modify the "MUST NOT add Preferred-Value" rule for this
   special case wannabe-redundant sgn-XX.
3: Modify the "MUST NOT add Preferred-Value" rule for any
   deprecation triggered by the source standards.

I'd like option 3 and a note about sgn-XX in the "redundant"
vs. "grandfathered" section.

Frank

---

From: Addison Phillips <[email-masked]>
Date: Wed, 13 Sep 2006 08:41:20 -0700

> That was my objection all along to this scheme.

John points out that these tags have meanings defined by the 3066 
registration process, which is true. I don't have a problem with 
deprecating the tags and it is probably best to include said deprecation 
in the registry so that users will migrate to the new tags. However... 
it *does* mean that we'll be breaking our own precedent and 
(potentially) breaking someone's tags.

+1

---

From: Addison Phillips <[email-masked]>
Date: Wed, 13 Sep 2006 08:58:47 -0700

Frank Ellermann wrote:
> +1.  

+1

> Some options:

I don't like (1). I'd prefer (3).
-- 
Addison Phillips
Globalization Architect -- Yahoo! Inc.

---

From: "Mark Davis" <[email-masked]>
Date: Wed, 13 Sep 2006 10:01:08 -0600

We already put a mechanism in place for this -- semantic widening.

sgn-US should simply mean "sign language as used in the US", which is
a widening of ASL. To perform that action, actually just dropping from
the registry would work. Every tag that was well-formed and valid
beforehand is also afterwards; and the semantic is broadened. I'd have
to read the rules again to see if that is permitted, however.

Mark

---

From: John Cowan <[email-masked]>
Date: Wed, 13 Sep 2006 12:12:13 -0400

Mark Davis scripsit:

> a widening of ASL. 

Well, but does it then exclude documents (videos, e.g.) that are
Canadian in origin?

> To perform that action, actually just dropping from
> the registry would work. 

If we were going to drop redundant tags from the registry, we would
have avoided putting them in there in the first place.  The point about
"sgn-US" is that it's syntactically redundant but semantically grandfathered.
It is not our business to change its meaning, only to mark it deprecated.

-- 
If you have ever wondered if you are in hell,         John Cowan
it has been said, then you are on a well-traveled     http://www.ccil.org/~cowan
road of spiritual inquiry.  If you are absolutely     [email-masked]
sure you are in hell, however, then you must be
on the Cross Bronx Expressway.          --Alan Feuer, NYTimes, 2002-09-20

---

From: Addison Phillips <[email-masked]>
Date: Wed, 13 Sep 2006 09:14:07 -0700

Dropping redundant registrations is not permitted. Making them truly 
redundant, however, is permitted. I'd support both widening the 
redundant registration and providing a preferred-value (true deprecation).

Addison

---

From: Peter Constable <[email-masked]>
Date: Wed, 13 Sep 2006 19:55:49 -0700

From: John Cowan [mailto:[email-masked]] 

> Well, but does it then exclude documents (videos, e.g.) that are

Is "Canadian in origin" the relevant attribute? I could sit in Toronto
and create content that is thus Cdn in origin but that uses en-US
conventions and so should be tagged en-US; I regularly sit in Seattle
and create content that is thus US in origin but that uses en-CA
conventions and so should be tagged en-CA.

Of course, what matters (which I know you know) is what language variety
is reflected in the document, and that potentially there is a Cdn
variant of ASL that might be used in some content, so redefining sgn-US
from "ASL" (which was a correct attribute of that content) to "signed
language as used in the US" (which is not a correct attribute of that
content) is resulting in the content becoming inappropriately tagged. 

Peter Constable

---

From: "Doug Ewell" <[email-masked]>
Date: Thu, 14 Sep 2006 06:26:22 -0700

Addison Phillips <addison at yahoo dash inc dot com> wrote:

> deprecating the tags and it is probably best to include said 
> deprecation in the registry so that users will migrate to the new 
> tags. However... it *does* mean that we'll be breaking our own 
> precedent and (potentially) breaking someone's tags.

The only action I can see that would break anything would be to redefine 
the registered meaning of "sgn-US" away from ASL and back to its 
generative meaning.  And because you pointed out the clause about 
"broadening the meaning" of Registry entries, I'm not even sure that is 
at issue.

We should be able to use the mechanisms we designed (in this case, 
deprecation and Preferred-Value) to solve the problems we face.  This 
thread sounds as though someone else made the rules, and we are walking 
around with a screwdriver trying to figure out how to work around them.

---

From: "Doug Ewell" <[email-masked]>
Date: Thu, 14 Sep 2006 06:38:04 -0700

> to read the rules again to see if that is permitted, however.

It's only a question of *which* rule we want to relax: the one in 
Section 3.3 about not removing entries from the set of grandfathered and 
redundant tags, or the one in Section 3.1 about not adding a 
Preferred-Value to an existing entry that is not "grandfathered" or 
"region".

What caused all this trouble was the practice of registering tags under 
RFC 3066 that were syntactically valid under RFC 3066, but carried a 
different meaning, and adding a clause to RFC 3066 that sanctioned this 
practice on the basis of "register[ing] information with the IANA about 
a tag defined by this document."  We now have a set of "redundant" tags 
that are NOT truly redundant: they do NOT mean the same thing as a whole 
as by analyzing the pieces separately.  This makes them fundamentally 
different from truly "redundant" tags such as "yi-latn" and "es-419".

---

From: Addison Phillips <[email-masked]>
Date: Thu, 14 Sep 2006 08:38:30 -0700

Doug Ewell wrote:
> redundant tags, 

Absolutely no way we should relax this rule. It makes a mockery of even 
having redundant tags. GF and redundant are supposed to be a complete 
set of the registered values from 3066... so that people can see exactly 
what happened to the tags. If we want to deprecate a tag, let's 
deprecate it. But hiding it is bad.

or the one in Section 3.1 about not adding a
> Preferred-Value to an existing entry that is not "grandfathered" or 

Currently, there is no way for us to deprecate language, extlang, 
script, or variant subtags, as well as redundant tags. The question we 
have to ask is whether we did this intelligently. Our thinking, 
expressed in the introduction to Section 3.4 (Stability), is "once 
canonical, always canonical for that meaning" and we had language 
subtags in particular in mind. Regions (countries, if we mean ISO 3166) 
do actually change over time, so included preferred-value registration 
there to accommodate it.

But... RFC 4646 contains Section 3.4, item 14, which suggests that we 
artificially created a contradiction in the rules:

--
Codes assigned by ISO 639, ISO 15924, or ISO 3166 that are withdrawn by 
their respective maintenance or registration authority remain valid in 
language tags. A 'Deprecated' field containing the date of withdrawal 
MUST be added to the record. If a new record of the same type is added 
that represents a replacement value, then a 'Preferred-Value' field MAY 
also be added.
--

By removing the prohibition in Section 3.1, we reduce the number of 
rules overall and put the rules in one place. Preferred-Values can be 
added to any record, which reduces our need to review and oversee ISO's 
decisions to cases in which they recycle codes.

> different from truly "redundant" tags such as "yi-latn" and "es-419".

Nothing says this in RFC 4646. If we intend to persist the registered 
meaning, we really should say so (and back out those records to 
grandfathered). But ultimately, I think the best course of action is:

1. Allow the registered meaning to remain, keeping the description (even 
though it could be broadened). Consider it historical.

2. Deprecate the redundant tag and provide the Preferred-Value (tell 
people what to do).

3. Relax in the knowledge that these tags are rare in practice and that 
we haven't hurt anyone (or anyone's content) in the process.

Addison

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 14 Sep 2006 09:49:01 -0600

Very sensible; I agree.

Mark

---

From: Addison Phillips <[email-masked]>
Date: Thu, 14 Sep 2006 08:49:10 -0700

> around with a screwdriver trying to figure out how to work around them.

(laughing) No, it is just that we probably know the rules better than 
others and can use them to achieve a useful end. In my opinion, like 
some of the other registrations pre-4646, the sgn-* registrations are 
ill-considered junk. If we had "i-asl" we wouldn't be having this 
conversation: we'd have a grandfathered tag and we'd deprecate the heck 
out of it. But instead we have these 
"well-formed-but-not-meaning-what-they-say" tags......

So deprecate them, but allow the registered meaning to persist for those 
that pay attention to such things.

Addison

---

From: "Doug Ewell" <[email-masked]>
Date: Thu, 14 Sep 2006 19:38:36 -0700

Addison Phillips <addison at yahoo dash inc dot com> wrote:

>> Section 3.3 about not removing entries from the set of grandfathered 
>> and redundant tags,
>
> Absolutely no way we should relax this rule. It makes a mockery of 
> even having redundant tags. GF and redundant are supposed to be a 
> complete set of the registered values from 3066... so that people can 
> see exactly what happened to the tags. If we want to deprecate a tag, 
> let's deprecate it. But hiding it is bad.

I agree 100%.  I had to bring it up as "an" option, since Mark had 
suggested it, but it's not one I prefer.

> artificially created a contradiction in the rules:

This actually doesn't surprise me, since these were among the most 
complex rules in one of the most bureaucratic parts of the document.

> added to any record, which reduces our need to review and oversee 
> ISO's decisions to cases in which they recycle codes.

Sounds good, especially the part about putting the rules in one place.

> grandfathered).

Hold on.  They are semantically grandfathered, but syntactically 
redundant.  They consist wholly of subtags that could be combined to 
form a valid tag.  That is central to the concept of "grandfathered" and 
we should not change this definition.

> But ultimately, I think the best course of action is:
>
> 1. Allow the registered meaning to remain, keeping the description 
> (even though it could be broadened). Consider it historical.

Figuratively, of course.  There is no such formal category in the 
Registry (like "Deprecated") and I assume you are not proposing one.

> 3. Relax in the knowledge that these tags are rare in practice and 
> that we haven't hurt anyone (or anyone's content) in the process.

I agree with all of this.

---

From: Addison Phillips <[email-masked]>
Date: Fri, 15 Sep 2006 09:06:16 -0700

> Registry (like "Deprecated") and I assume you are not proposing one.

Yes, of course. "Type: redundant" is pretty much "historical" anyway.