Subject: [Ltru] Re: Serbo-Croatian Deprecations and Variants

From: "Doug Ewell" <[email-masked]>
Date: Mon, 6 Aug 2007 20:05:11 -0700

<Karen underscore Broome at spe dot sony dot com> wrote:

> What should I do with films that have been categorized as 
> Serbo-Croatian in the past? The language isn't suddenly different 
> because the political climate changed since the films were first made.

As Addison said, you don't actually have to change anything; that's why 
we made sure that "deprecated" subtags were still valid, and not removed 
from the Registry.

> If I wish to maintain classifications previously made against this 
> deprecated language that is now back in business in 639-3, would I 
> extend the deprecated 639-1 prefix and include the 639-3 code;

You don't mean "sh-sr" or "sr-hr", right?  That's not well-formed.

> use one of the deprecated 639-2 codes;

Which are those?

> or just use the new 639-3 code alone? Which of these options is most 
> "wise"?

You can leave everything alone, or you can (more or less arbitrarily) 
choose "sr" or "hr", both of which are valid 639-1 code elements.

> I'm only considering this question in terms of spoken content and I 
> realize we don't have the answer for this yet. But I figure answers 
> start with questions (despite my studio's affiliation with 
> "Jeopardy").

--
Doug Ewell  *  Fullerton, California, USA  *  RFC 4645  *  UTN #14
("Jeopardy!" wannabe)
http://users.adelphia.net/~dewell/
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: [email-masked]
Date: Tue, 7 Aug 2007 09:41:49 -0700

Doug,

I understand that deprecated tags are still valid. I wouldn't use hr or sr 
because that's a more narrow semantic than Serbo-Croatian. I guess my 
question is this: Is it "wiser" to use the deprecated ISO 639-1 tag, or 
the ISO 639-3 tag, which is not deprecated and is a macrolanguage. Assume 
a clean slate and that any tagging applied to date can be changed with a 
SQL update, but individual language classifications cannot be reassigned.

Should the draft offer guidance on this? I don't know which tag is 
preferred -- sh or hbs. (I initially thought that perhaps the sh tag might 
be combined with the hbs tag, but now I realize that's silly -- I was 
thinking of the Chinese variant examples from the day before.)  I'm not 
really using it in a macrolanguage sense, but I'd rather not use any 
deprecated tagging, whether it's valid or not. Note: This language tag 
will not be attached to a region.

Regards,

Karen Broome
Metadata Systems Designer
Sony Pictures Entertainment
310.244.4384

"Doug Ewell" <[email-masked]> 
08/06/2007 08:05 PM

To
"LTRU Working Group" <[email-masked]>
cc
[email-masked]
Subject
[Ltru] Re: Serbo-Croatian Deprecations and Variants

Which are those?

---

From: John Cowan <[email-masked]>
Date: Tue, 7 Aug 2007 15:42:54 -0400

[email-masked] scripsit:
> the ISO 639-3 tag, which is not deprecated and is a macrolanguage.  

No 639-1 or -2 tag will ever be replaced by a 639-3 tag.

-- 
Here lies the Christian,                        John Cowan
        judge, and poet Peter,                  http://www.ccil.org/~cowan
Who broke the laws of God                       [email-masked]
        and man and metre.

---

From: "Doug Ewell" <[email-masked]>
Date: Tue, 7 Aug 2007 23:21:25 -0700

<Karen underscore Broome at spe dot sony dot com> wrote:

> I understand that deprecated tags are still valid. I wouldn't use hr 
> or sr because that's a more narrow semantic than Serbo-Croatian. I 
> guess my question is this: Is it "wiser" to use the deprecated ISO 
> 639-1 tag, or the ISO 639-3 tag, which is not deprecated and is a 
> macrolanguage. Assume a clean slate and that any tagging applied to 
> date can be changed with a SQL update, but individual language 
> classifications cannot be reassigned.

You can't use 'hbs' in any case, because it is defined in ISO 639-3 as 
equivalent to 'sh' (see 
http://www.sil.org/iso639-3/documentation.asp?id=hbs).  Whenever an 
alpha-2 and an alpha-3 code element exist for the same language, only 
the alpha-2 can be used in a language tag.  That has been the rule since 
RFC 3066 first allowed alpha-3 subtags, and in the 4646 era we implement 
it by not putting the alpha-3 in the Registry.

What makes Serbo-Croatian unusual in this regard is that there was a 
code element for it in ISO 639-1 (sh) and there is now one in 639-3 
(hbs), but apparently never in ISO 639-2.  Furthermore, and strangely, 
the deletion of 'sh' from ISO 639-1 is no longer listed on the ISO 
639-2/RA Change Notice page, even though we know it was once there 
because we gave 'sh' a Deprecated date of 2000-02-18 in the Registry 
(the same day Bosnian and many others were added).

The resurrection of Serbo-Croatian as an ISO 639-3 macrolanguage poses 
an interesting challenge, because deprecation is supposed to be forever 
in the Registry.  (Section 3.4, item 1: "Values in the fields 'Type', 
'Subtag', 'Tag', 'Added', 'Deprecated' and 'Preferred-Value' MUST NOT be 
changed and are guaranteed to be stable over time.")  My assumption is 
that 'sh' has to remain deprecated, and 'hbs' has to remain on the 
outside looking in.

Back to your question:  If you know the content is Serbian, use 'sr'; if 
Croatian, use 'hr'; if Bosnian, use 'bs'.  If all you know is that it's 
some type of Serbo-Croatian, and you don't feel comfortable picking one 
of the other three arbitrarily, your best bet might be to use 'sh' even 
though Section 3.1.5 says you SHOULD NOT do so, and even though it feels 
wrong.

---

From: Addison Phillips <[email-masked]>
Date: Wed, 08 Aug 2007 01:03:49 -0700

> that 'sh' has to remain deprecated, and 'hbs' has to remain on the 

Well... yeah. That's what stability really means.

> though Section 3.1.5 says you SHOULD NOT do so, and even though it feels 

SHOULD NOT is not the same as MUST NOT. You should not use 'sh' unless 
you have a (pretty good) reason to do so. The fact that you have (audio) 
data that cannot otherwise be distinguished or is already labeled as 
'sh' seems like a "good reason".

Addison

-- 
Addison Phillips
Globalization Architect -- Yahoo! Inc.
Chair -- W3C Internationalization Core WG

Internationalization is an architecture.
It is not a feature.

---

From: [email-masked]
Date: Wed, 8 Aug 2007 14:59:08 -0700

OK, "sh" it is.

Just confirming that the deprecated tag is preferred.  (Seems a little 
counterintuitive?)

Regards,

Karen 

Addison Phillips <[email-masked]> 
08/08/2007 01:03 AM

To
Doug Ewell <[email-masked]>
cc
[email-masked], LTRU Working Group <[email-masked]>
Subject
Re: [Ltru] Re: Serbo-Croatian Deprecations and Variants

Addison

---

From: "Doug Ewell" <[email-masked]>
Date: Wed, 8 Aug 2007 23:45:09 -0700

> Just confirming that the deprecated tag is preferred.  (Seems a little 

It won't be the last time.  The ISO 639-3 folks have made it known that 
A might be withdrawn and replaced by B and C, or vice versa, on an 
annual-review basis as supported by linguistic research.  We might have 
a lot more cases like Serbo-Croatian in the future.

---

From: Peter Constable <[email-masked]>
Date: Thu, 9 Aug 2007 11:51:12 -0700

Clarification on "withdrawn": nothing that gets coded will later be un-coded: if language Foo is assign an ID "foo", then Foo will always be coded as "foo". But "foo" can be retired/deprecated.

Peter

---

From: Addison Phillips <[email-masked]>
Date: Thu, 09 Aug 2007 13:13:39 -0700

Does "retired" mean that the code would be formally withdrawn? Assuming 
it means this, does that mean that the code would never be reused for 
some other language?

Addison

---

From: Peter Constable <[email-masked]>
Date: Thu, 9 Aug 2007 23:19:14 -0700

I thought my statement made that clear: once encoded, always encoded. "Retired" means it's no longer recommended -- basically the same as deprecated. But I think I've made clear for a long time: ISO 639-3 will be stable in the sense that an identifier will always retain its semantic and will never get re-defined.

Peter

-----Original Message-----
From: Addison Phillips [mailto:[email-masked]]
Sent: Thursday, August 09, 2007 1:14 PM
To: Peter Constable
Cc: LTRU Working Group
Subject: Re: [Ltru] Re: Serbo-Croatian Deprecations and Variants

Addison

---

From: Martin Duerst <[email-masked]>
Date: Thu, 09 Aug 2007 09:05:48 +0900

>You can't use 'hbs' in any case, because it is defined in ISO 639-3 as equivalent to 'sh' (see http://www.sil.org/iso639-3/documentation.asp?id=hbs).  Whenever an alpha-2 and an alpha-3 code element exist for the same language, only the alpha-2 can be used in a language tag.  

Yes, very much so.

>That has been the rule since RFC 3066 first allowed alpha-3 subtags, and in the 4646 era we implement it by not putting the alpha-3 in the Registry.

Just cross-checking to be sure (and offline, so not able to look at
the document directly): Is this rule worded so that it clearly includes
all kinds of different sources?

Regards,   Martin.

>What makes Serbo-Croatian unusual in this regard is that there was a code element for it in ISO 639-1 (sh) and there is now one in 639-3 (hbs), but apparently never in ISO 639-2.  Furthermore, and strangely, the deletion of 'sh' from ISO 639-1 is no longer listed on the ISO 639-2/RA Change Notice page, even though we know it was once there because we gave 'sh' a Deprecated date of 2000-02-18 in the Registry (the same day Bosnian and many others were added).
>
>The resurrection of Serbo-Croatian as an ISO 639-3 macrolanguage poses an interesting challenge, because deprecation is supposed to be forever in the Registry.  (Section 3.4, item 1: "Values in the fields 'Type', 'Subtag', 'Tag', 'Added', 'Deprecated' and 'Preferred-Value' MUST NOT be changed and are guaranteed to be stable over time.")  My assumption is that 'sh' has to remain deprecated, and 'hbs' has to remain on the outside looking in.

Is there any way in the registry to find that instead of 'hbs', one
has to use 'sh'? Or is 'hbs' just simply totally absent from the
registry, even from comments?

>Back to your question:  If you know the content is Serbian, use 'sr'; if Croatian, use 'hr'; if Bosnian, use 'bs'.  If all you know is that it's some type of Serbo-Croatian, and you don't feel comfortable picking one of the other three arbitrarily, your best bet might be to use 'sh' even though Section 3.1.5 says you SHOULD NOT do so, and even though it feels wrong.

I think the resurrection of 'hbs' in iso-639-3 shows that the depreciaton
of 'sh' was somehow less serious than maybe other depreciations. On a
simple level, it didn't make sense to keep in 'sh' if subdivisions
were added, especially from a political viewpoint.

Regards,    Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: "Doug Ewell" <[email-masked]>
Date: Fri, 10 Aug 2007 07:50:49 -0700

Martin Duerst <duerst at it dot aoyama dot ac dot jp> wrote:

>> That has been the rule since RFC 3066 first allowed alpha-3 subtags, 
>> and in the 4646 era we implement it by not putting the alpha-3 in the 
>> Registry.
>
> the document directly): Is this rule worded so that it clearly 
> includes all kinds of different sources?

Section 2.2.1:  "Note: For languages that have both an ISO 639-1 
two-character code and an ISO 639-2 three-character code, only the ISO 
639-1 two-character code is defined in the IANA registry."

To be very strict, this should probably be "... only one subtag, 
corresponding to the ISO 639-1 two-character code, is defined..."  We 
don't define ISO codes, we define subtags based on them.

> registry, even from comments?

'hbs' is totally absent from the Registry, just like 'eng' and 'fra' and 
'deu'.

> I think the resurrection of 'hbs' in iso-639-3 shows that the 
> depreciaton of 'sh' was somehow less serious than maybe other 
> depreciations. On a simple level, it didn't make sense to keep in 'sh' 
> if subdivisions were added, especially from a political viewpoint.

I don't know if it was a less-serious deprecation.  ISO 639-3 brought it 
back as a macrolanguage, a concept which didn't formally exist in -1 
and -2.

The whole business of splitting the concept of "Serbo-Croatian" on 
political grounds troubles me.  I know there are linguistic differences 
as well, but mutual non-intelligibility?  And in any case, 'no' was not 
withdrawn when 'nb' and nn' were added.

---

From: Peter Constable <[email-masked]>
Date: Fri, 10 Aug 2007 10:37:21 -0700

> don't define ISO codes, we define subtags based on them.

I'd be inclined to say we don't define subtags; we register them.

Peter