Subject: Re: [Ltru] Uniqueness of variant subtags (Was: BCP47 Appeals process

From: "Doug Ewell" <[email-masked]>
Date: Tue, 23 Sep 2008 06:53:47 -0600

Stephane Bortzmeyer <bortzmeyer at nic dot fr> wrote:

> We are in WGLC so I won't suggest a change to 4646bis but I should 
> have noticed earlier that uniqueness of variant subtags could have 
> been expressed more clearly.

I agree that the text should specify that two subtags of the same type 
shall not have the same value.  Hopefully this is a quick fix and not 
something that will generate a lengthy debate and tie up the WG Last 
Call.  There's not much to debate about.

--
Doug Ewell  *  Thornton, Colorado, USA  *  RFC 4645  *  UTN #14
http://www.ewellic.org
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages  ˆ

_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Phillips, Addison" <[email-masked]>
Date: Tue, 23 Sep 2008 09:55:26 -0700

Agreed.

I created a new editor's copy, that I will put online in a bit. In this copy, to address this issue, I put this paragraph into the section on the registration project just following the sentence " Variant registration requests SHOULD include at least one 'Prefix' field in the registration form":

--
<t>Requests to assign an additional record of a given type with an existing subtag value MUST be rejected. For example, the variant subtag 'rozaj' already exists in the registry, so adding a second record of type 'variant' with the subtag 'rozaj' is prohibited.</t>
--

Addison Phillips
Globalization Architect -- Lab126

Internationalization is not a feature.
It is an architecture.

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Doug Ewell
> Sent: Tuesday, September 23, 2008 5:54 AM
> To: LTRU Working Group
> Subject: Re: [Ltru] Uniqueness of variant subtags (Was: BCP47
> Appeals process
>

---

From: Martin Duerst <[email-masked]>
Date: Tue, 30 Sep 2008 13:42:59 +0900

I seem to vaguely remember that last time this issue came up
for discussion, we didn't go as far as the new text because
of cases such as year numbers, where it would be very strange
to say you can't have any other variant subtag '1901' because
this one is already used for German.

As said above, this is just a vague recollection, but I'd like
those with better memories to cross-check/confirm/deny.

Regards,    Martin.

At 01:55 08/09/24, Phillips, Addison wrote:
>Agreed.
>
>I created a new editor's copy, that I will put online in a bit. In this 
>copy, to address this issue, I put this paragraph into the section on the 
>registration project just following the sentence " Variant registration 
>requests SHOULD include at least one 'Prefix' field in the registration form":
>
>--
><t>Requests to assign an additional record of a given type with an existing 
>subtag value MUST be rejected. For example, the variant subtag 'rozaj' 
>already exists in the registry, so adding a second record of type 'variant' 
>with the subtag 'rozaj' is prohibited.</t>
>--
>
>Addison Phillips
>> http://www.alvestrand.no/mailman/listinfo/ietf-languages  $B%R+(B
>> 
>_______________________________________________ >Ltru mailing 
>list >[email-masked] >https://www.ietf.org/mailman/listinfo/ltr

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: "Phillips, Addison" <[email-masked]>
Date: Mon, 29 Sep 2008 22:03:00 -0700

Actually, I think we (I, at least) said the opposite at the time. The problem with "naked" year numbers is that there are a finite number of them and we are bound to have events happen to languages in the same Gregorian year at some point--however, we should not permit "modal meanings" from being registered. Luckily, it seems that the current trend is to append some awful cryptic gunk to the year to make it unique (1959akad, 1958hpin).

The draft already said that adding prefixes which compromised the meaning of a subtag would/SHOULD be rejected. This just prevents to records from having the same value with different definitions (something I'd define as a bug--two records with the same Primary Key).

Addison

Addison Phillips

> From: Martin Duerst [mailto:[email-masked]]
> Sent: Monday, September 29, 2008 9:43 PM
> To: Phillips, Addison; Doug Ewell; LTRU Working Group
> >> I agree that the text should specify that two subtags of the
> same
> >> type
> >> shall not have the same value.  Hopefully this is a quick fix
> and
> >> not
> >> http://www.alvestrand.no/mailman/listinfo/ietf-languages  ヒ・
> >>

---

From: "Mark Davis" <[email-masked]>
Date: Tue, 30 Sep 2008 10:24:34 +0200

Somehow this got bounced from LTRU. Trying again.
Mark

---------- Forwarded message ----------
From: Mark Davis <[email-masked]>
Date: Tue, Sep 30, 2008 at 8:33 AM
Subject: Re: [Ltru] Uniqueness of variant subtags (Was: BCP47 Appeals
process
To: "Phillips, Addison" <[email-masked]>
Cc: Martin Duerst <[email-masked]>, Doug Ewell <[email-masked]>,
LTRU Working Group <[email-masked]>

I see it differently; we added some pointless digits to a string of text to
make it unique.

I see no problem at all with having variant subtag that have a unique
meaning without having a unique denotation, subtags can be used meaningfully
in combination with others where useful for producing a specific denotation.
A variant name like "northern" would be fine, and could be usefully and
meaningfully combined with language/script/region subtags where appropriate,
AND matches the way that we do actually name variants in many cases, by
geographic extent (just look at how many such cases there are in ISO 639-3).

I view that as far superior to bizarre obfuscations of the terms, jumping
through hoops for no purpose. The real problem is the inconsistency. There
was no need to obfuscate "fonipa" by having "fonipade", "fonipaen",
"fontipfr", and so on. There is no need to obfuscate "western", "pinyin", or
"academi"(c). What we have ended up with is just a muddle that sloshes
between the extremes with little rhyme or reason.

Mark

On Tue, Sep 30, 2008 at 7:03 AM, Phillips, Addison <[email-masked]>
wrote:
> Actually, I think we (I, at least) said the opposite at the time. The
problem with "naked" year numbers is that there are a finite number of them
and we are bound to have events happen to languages in the same Gregorian
year at some point--however, we should not permit "modal meanings" from
being registered. Luckily, it seems that the current trend is to append some
awful cryptic gunk to the year to make it unique (1959akad, 1958hpin).
>
> The draft already said that adding prefixes which compromised the meaning
of a subtag would/SHOULD be rejected. This just prevents to records from
having the same value with different definitions (something I'd define as a
bug--two records with the same Primary Key).

---

From: "Doug Ewell" <[email-masked]>
Date: Tue, 30 Sep 2008 07:05:14 -0600

Martin Duerst <duerst at it dot aoyama dot ac dot jp> wrote:

> I seem to vaguely remember that last time this issue came up for 
> discussion, we didn't go as far as the new text because of cases such 
> as year numbers, where it would be very strange to say you can't have 
> any other variant subtag '1901' because this one is already used for 
> German.

To me this is a better argument for avoiding bare year numbers than for 
allowing duplicate subtag values.

---

From: "Phillips, Addison" <[email-masked]>
Date: Tue, 30 Sep 2008 08:06:53 -0700

But ‘northern’ wouldn’t meaning anything different. It is one thing to allow a generic subtag to mutate meaning differently in different contexts. It is another thing to have a totally different meaning accrete to a subtag. If ‘1901’ meant both the German orthography and, say, some Uzbek transliteration scheme, you couldn’t tell which was meant.

Addison

Addison Phillips

From: [email-masked] [mailto:[email-masked]] On Behalf Of Mark Davis
Sent: Monday, September 29, 2008 11:33 PM
To: Phillips, Addison
Cc: Martin Duerst; Doug Ewell; LTRU Working Group
Subject: Re: [Ltru] Uniqueness of variant subtags (Was: BCP47 Appeals process

I see it differently; we added some pointless digits to a string of text to make it unique.

I see no problem at all with having variant subtag that have a unique meaning without having a unique denotation, subtags can be used meaningfully in combination with others where useful for producing a specific denotation. A variant name like "northern" would be fine, and could be usefully and meaningfully combined with language/script/region subtags where appropriate, AND matches the way that we do actually name variants in many cases, by geographic extent (just look at how many such cases there are in ISO 639-3).

I view that as far superior to bizarre obfuscations of the terms, jumping through hoops for no purpose. The real problem is the inconsistency. There was no need to obfuscate "fonipa" by having "fonipade", "fonipaen", "fontipfr", and so on. There is no need to obfuscate "western", "pinyin", or "academi"(c). What we have ended up with is just a muddle that sloshes between the extremes with little rhyme or reason.

Mark

On Tue, Sep 30, 2008 at 7:03 AM, Phillips, Addison <[email-masked]<mailto:[email-masked]>> wrote:
>> From: Martin Duerst [mailto:[email-masked]<mailto:[email-masked]>]
>> >> From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On
>> >> Behalf Of Doug Ewell
>> >> [email-masked]<mailto:[email-masked]>
>> >list >[email-masked]<mailto:[email-masked]> >https://www.ietf.org/mailman/listinfo/ltr
>>
>>
>> #-#-#  http://www.sw.it.aoyama.ac.jp
>> mailto:[email-masked]<mailto:[email-masked]>
>
> [email-masked]<mailto:[email-masked]>

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 30 Sep 2008 17:20:04 +0200

On Tue, Sep 30, 2008 at 08:06:53AM -0700,
 Phillips, Addison <[email-masked]> wrote 
 a message of 367 lines which said:

> If "1901" meant both the German orthography and, say, some Uzbek
> transliteration scheme, you couldn't tell which was meant.

Yes, you can, if there is a Prefix. Why not saying that a variant
subtag MUST be unique _unless_ there is a (different) Prefix on all
the variants sharing the same subtag? (Warning: this may break the
current implementations which, while it is not clear in RFC 4646, may
assume that variant subtags are unique.)

I understand the rationale (behind the uniqueness of variant subtags)
but I regret it. "northern" is a good example. If there is one
northern dialect of French and one of Uzbek, it is a pity we cannot
register them without adding letters that will make less obvious.

---

From: "Phillips, Addison" <[email-masked]>
Date: Tue, 30 Sep 2008 08:39:02 -0700

Let me retry expressing this. Consider this record:

Type: variant
Subtag: 1901
Description: German orthography of 1901
Description: Uzbek transliteration of 1901
Prefix: de
Prefix: uz
Comments: Good luck figuring this out.

This is very different from:

Type: variant
Subtag: northern
Description: Northern direction variation of a language

Noting that "en-ZA-northern" is far to the south of virtually all "southern" variations.

In other words, if you register a subtag, it should have some kind of specified meaning and not be subject to competing interpretations. "es-CO-1901" is probably still nonsense, but at least you know what sort of nonsense :-P.

I also note that no truly generic variants (such as northern) have been accepted. Both fonipa and fonupa refer to something concrete that could be applied to (m)any languages. 

Addison

Addison Phillips

> From: Stephane Bortzmeyer [mailto:[email-masked]]
> Sent: Tuesday, September 30, 2008 8:20 AM
> To: Phillips, Addison
> Cc: Mark Davis; LTRU Working Group; Doug Ewell
> Subject: Re: Uniqueness of variant subtags (Was: BCP47 Appeals
> process
>

---

From: "Phillips, Addison" <[email-masked]>
Date: Tue, 30 Sep 2008 08:40:36 -0700

(editor hat ON)

Since there is no consensus on my proposed change, unless directed by the chairs otherwise, I will removed my proposed text from the editor's copy before incorporating any other changes.

Addison

Addison Phillips

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 30 Sep 2008 13:43:36 -0700

Hi -

As a technical contributor...

> From: "Phillips, Addison" <[email-masked]>
> To: "Stephane Bortzmeyer" <[email-masked]>
> Cc: "LTRU Working Group" <[email-masked]>; "Ewell" <[email-masked]>; <[email-masked]>
> Sent: Tuesday, September 30, 2008 8:39 AM

This is precisely how the [email-masked] list appears
to proposing that "pinyin" be handled.

Consider, on the other hand...

Type: variant
Subtag: 1901
Description: German orthography of 1901
Prefix: de
Comments: Should be clear.

Type: variant
Subtag: 1901
Description: Uzbek transliteration of 1901
Prefix: uz
Comments: Should be clear.
 
We should be clear about whether we want to permit this or not.
I think there is no problem when the prefix is used.  The problem
is figuring out what it means when one of these variants is used
with any other prefix.  (It's also a problem if someone has used the
subtag as a database key.)

Randy

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 30 Sep 2008 22:39:57 +0200

On Tue, Sep 30, 2008 at 08:40:36AM -0700,
 Phillips, Addison <[email-masked]> wrote 
 a message of 41 lines which said:

> Since there is no consensus on my proposed change, unless directed
> by the chairs otherwise, I will removed my proposed text from the
> editor's copy before incorporating any other changes.

True, there is no consensus yet but the issue is a real one, no? RFC
4646 does not provide clear guidance (although ietf-languages'
practice has always been to refuse identical variant
subtags). Shouldn't we delay the I-D a bit for that?
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 30 Sep 2008 23:00:06 +0200

On Tue, Sep 30, 2008 at 01:43:36PM -0700,
 Randy Presuhn <[email-masked]> wrote 

> Consider, on the other hand...

The second one, with two records, has my preference.
 
> We should be clear about whether we want to permit this or not.

Yes. Today, we are in a limbo, it seems.

> (It's also a problem if someone has used the subtag as a database
> key.)

Yes. It seems most developers assumed that identical variant subtags
were forbidden and so allowing your solution would break
compatibility.

Also, for all other types, duplication of a subtag is forbidden. It
may be difficult to explain why variants are different.

---

From: "Phillips, Addison" <[email-masked]>
Date: Tue, 30 Sep 2008 14:59:42 -0700

> may be difficult to explain why variants are different.

I am wholly and utterly opposed to allowing two records of the same type with the same subtag.

Addison

---

From: John Cowan <[email-masked]>
Date: Tue, 30 Sep 2008 18:21:24 -0400

> type with the same subtag.

And so am I.

-- 
John Cowan  [email-masked]  http://ccil.org/~cowan
Linguistics is arguably the most hotly contested property in the academic
realm. It is soaked with the blood of poets, theologians, philosophers,
philologists, psychologists, biologists and neurologists, along with
whatever blood can be got out of grammarians. - Russ Rymer
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Doug Ewell" <[email-masked]>
Date: Tue, 30 Sep 2008 18:43:04 -0600

Phillips, Addison wrote:

> But ‘northern’ wouldn’t meaning anything different. It is one thing to 
> allow a generic subtag to mutate meaning differently in different 
> contexts. It is another thing to have a totally different meaning 
> accrete to a subtag. If ‘1901’ meant both the German orthography and, 
> say, some Uzbek transliteration scheme, you couldn’t tell which was 
> meant.

I agree.  I can see 'pinyin' being used for slightly different pinyins, 
but not when the two meanings are altogether different.

> Comments: Good luck figuring this out.

I agree, this would be bad.  But even worse would be:

Type: variant
Subtag: 1901
Description: German orthography of 1901
Prefix: de
Comments: Don't mix this up with the other 1901.
%%

Type: variant
Subtag: 1901
Description: Uzbek transliteration of 1901
Prefix: uz
Comments: Has your database committed seppuku yet?

> I also note that no truly generic variants (such as northern) have 
> been accepted. Both fonipa and fonupa refer to something concrete that 
> could be applied to (m)any languages.

We've traditionally used "generic" to mean "applicable to all 
(languages)" and not "abstract."

Stephane Bortzmeyer <bortzmeyer at nic dot fr> wrote:

>> Since there is no consensus on my proposed change, unless directed by 
>> the chairs otherwise, I will removed my proposed text from the 
> practice has always been to refuse identical variant subtags). 
> Shouldn't we delay the I-D a bit for that?

I agree that we ought to discuss it and try to reach a resolution before 
IETF Last Call.  I would hope this would not entail any major delay.  I 
know others have opposing views on this (and I got burned yesterday for 
ignoring that fact) but to me it is obvious which choice we should make.

---

From: "Phillips, Addison" <[email-masked]>
Date: Tue, 30 Sep 2008 18:02:25 -0700

> (languages)" and not "abstract."

"Abstract" would be a better word in this context. So: we have not registered any abstract generic variants ;-)

I would say, though, that generic variants are not *necessarily* applicable to 'all' languages. It is that the list of languages is non-specific. One could possibly create an enumeration, but that list might be quite large and it isn't known in advance.

Addison

---

From: "Mark Davis" <[email-masked]>
Date: Wed, 1 Oct 2008 08:25:12 +0200

As am I.

I'm not opposed to

de-1901
uz-1901
and so on. These are meaningful and easy for people to understand.

I think that can be encapsulated as:

Type: variant
Subtag: 2003
Description: Denotes a change in orthography, transliteration, or other
usage associated with the publication of a significant standard or other
specification in 2003
Prefix: de
Prefix: uz
Prefix: be-ungegn

We can then add prefixes that are known to have such cases over time.

Mark

On Wed, Oct 1, 2008 at 12:21 AM, John Cowan <[email-masked]> wrote:

> John Cowan  [email-masked]  http://ccil.org/~cowan<http://ccil.org/%7Ecowan>

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 30 Sep 2008 23:52:17 -0700

Hi -

> From: "Mark Davis" <[email-masked]>
> To: "John Cowan" <[email-masked]>
> Cc: "LTRU Working Group" <[email-masked]>
> Sent: Tuesday, September 30, 2008 11:25 PM

I'm worried about "fuzziness" creeping in  in cases like this.
A registration request might provide a very precise definition,
identifying a particular national or international standard
for a particular prefix.  The variant record, lacking this
information, might consequently result in the subtag being
interpreted much more broadly in the case of that particular
prefix.  (That's the crux of my concern with respect to
"pinyin" on the [email-masked] list, especially
when understood for extended pinyin-like schemes, since
its entirely conceivable that more than one might exist
for languages for which pinyin wasn't originally designed.)

The net result, then, is that yet another variant becomes
necessary for the strictly defined variant, bringing with
it potential user confusion.  (E.g., if it's really
Hanyu Pinyin, tag it zh-cmn-Latn-pinyin-hpinyin?)

Randy

---

From: "Mark Davis" <[email-masked]>
Date: Wed, 1 Oct 2008 09:49:51 +0200

All subtags are vague to some degree; we have to accept that. I don't speak
precisely the same as, say, Ken Whistler, although we grew up only 50 miles
apart. And when I lived in Switzerland, each neighboring village around
where I lived had its own marked differences. Yet broad terms like gsw are
useful. The question is whether the range of dialects (or ideolects) that
subtags encompass are meaningful to distinguish as a group, and correspond
to some umbrella concept that people understand. The latter part is crucial.
That's why I'm ok with pinyin if it is defined as 'an official romanization
according to the Chinese government'. People understand what that means.

Some artificial, after-the-fact principle about which letters are given
particular phonetic values doesn't cut it, because it doesn't correspond to
natural grouping that people would think of. If there was some romanization
of Armenian that used 'j' , 'c', in the same way as Hanyu Pinyin, nobody
would understand our using 'pinyin' as a subtag for that -- it would be
bizarre.

Mark