Subject: [Ltru] LTRU progress and ISO 639-6 (was WiktionaryZ language codes)

From: "Debbie Garside" <[email-masked]>
Date: Mon, 13 Nov 2006 22:20:35 -0000

Stephane wrote:

> > Can you send me details of the project?
> 
> Attached.

Ah!  So you really mean LTRU!

What exactly is going on with LTRU.  There seems to be no progress being
made at all.  What has happened to issue tracker.  How do we find out what
the issues are and in what state of discussion they are in?

Randy, Martin, I would really appreciate it if we could have a list of the
issues and possible solutions.  Or have I missed something? Are RFC4645/6bis
on track without me realising?

What are the timescales that we are now working to?  When can we expect the
next draft?  Questions, questions! And why, when it is so relevant to all
the variants currently under discussion
(written/spoken/signed/orthographies/writing systems etc) is ISO 639-6 not
being discussed?

As I have mentioned, both on list and privately, ISO 639-6 is at a stage
where this WG can have significant input.  If members of this WG feel that
there are elements of the 639-6 system that will be difficult to incorporate
within RFC4646bis+ then I need to know NOW!  If I do not hear to the
contrary, I will assume that this WG has no problem with the current
development of ISO 639-6 and will continue along the same vein. 

Having asked so many times for your input, I will be pretty mad if you come
back in a years time and tell me it is not what you want.

Debbie Garside
Editor

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of 
> Stephane Bortzmeyer
> Sent: 13 November 2006 21:25
> To: Debbie Garside
> Cc: [email-masked]
> Subject: Re: WiktionaryZ language codes
> 
> On Mon, Nov 13, 2006 at 06:27:54PM -0000,  Debbie Garside 
> <[email-masked]> wrote  a message of 36 lines which said:
> 
> > Can you send me details of the project?
> 
> Attached.
> 
> [It seems currently stalled.]
>  
> 

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: Martin Duerst <[email-masked]>
Date: Tue, 14 Nov 2006 10:54:33 +0900

At 07:20 06/11/14, Debbie Garside wrote:
>made at all.

For several weeks before an IETF meeting, draft submission is closed.

>What has happened to issue tracker.  How do we find out what

I haven't officially tracked any issues, because they are still
too much in flux.

>on track without me realising?

I think there are several areas of issues. Doug has sent a list
that he need resolved for his next draft a few days ago.

>(written/spoken/signed/orthographies/writing systems etc) is ISO 639-6 not

I think one reason is that most people do not know very much about
639-6. If you don't mind, could you send a short summary of how
639-6 addresses things such as written/spoken/signed/orthographies/writing
systems etc? With concrete examples, indicating whether these are
currently in the draft or are hypothetical. I think that would help
the WG to have a good disucussion.

Regards,    Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: "Debbie Garside" <[email-masked]>
Date: Tue, 14 Nov 2006 14:05:09 -0000

>For several weeks before an IETF meeting, draft submission is closed.

But that doesn't stop us discussing issues and updating the draft ready for
submission.

>I haven't officially tracked any issues, because they are still too much in
flux.

Maybe they are still too much in flux because nobody can readily identify
the issues and the possible solutions that have been presented!

>[re ISO 639-6 ]I think one reason is that most people do not know very much
about 639-6. If you don't mind, could you send a short summary of how
639-6 addresses things such as written/spoken/signed/orthographies/writing
systems etc? With concrete examples, indicating whether these are currently
in the draft or are hypothetical. I think that would help the WG to have a
good disucussion.

Thanks for that. I will prepare a short summary with examples and will
forward it to the WG for discussion asap.

Regards

Debbie

---

From: "Debbie Garside" <[email-masked]>
Date: Wed, 15 Nov 2006 23:54:39 -0000

In response to Martin's request for an overview of ISO 639-6.

Apologies for the length of this post. I have tried to condense it but feel
that I must also give enough information to facilitate informed discussion.

For those that are not familiar with ISO 639-6, a brief description. It is a
hierarchical system of alpha4 identifiers for language variants based on
genetic and/or geographic affinities.  The system provides for the highest
level of linguistic classification, e.g. phyla, down to the lowest level of
granularity (not definitive but could be as low as linguistic "register" and
orthographies). Although, the system itself does not classify.  There are no
boundaries for languages/dialects/complexes etc. within ISO 639-6 and this
is intentional.  The system also offers codes for different language/variant
modes - written/spoken/signed/sung - and is extensible.  I am currently
looking at the inclusion of linguistic "registers". See
http://en.wikipedia.org/wiki/Register_(linguistics)

In devising a system that will work in synergy with other ISO 639 standards,
no alpha4 identifier is assigned to a linguistic entity that already has an
alpha3 in either ISO 639-3 or -5.  Thus, the hierarchical system,
necessarily, includes ISO 639-3 and -5 identifiers and their associated
reference names; ISO 639-3 is already mapped to ISO 639-1 and ISO 639-2T/B.
This means that the whole of the ISO 639 family is related through ISO
639-6.

The proposed data tables for ISO 639-6 will include the following fields:

Alpha4 Identifier
Alpha4 Parent Identifier
Reference Name

Here are some examples of the draft/unverified data (this means that more
information - especially temporal - is currently being compiled and
verification procedures are in process):

NB: Please note ISO 639-5 codes have not yet been included so the code list
displayed is for demonstration purposes only.

Neapolitan (showing upper levels)

inht		Indo-Hittite		
indo	inht	Indo-European	
itlc	indo	Italic
lnof	itlc	Latino-Faliscan
rmce	lnof	Romance 
cont	rmce	Romance Continental
wctl	cont	Romance Western Continental
itor	wctl	Italo-Romance 
itnc	itor	Italian Cluster
nap	itnc	Neapolitan
napw	nap	Neapolitan Written
napl	napw	Neapolitan Written Latin Script
naps	nap	Neapolitan Spoken
mags	naps	Marchigiano-S
reno	naps	Reatino- Aquilano
abze	naps	Abruzzese
mols	naps	Molisano
cpno	naps	Campano
npou	naps	Napoletano-U

Khanty (showing lower levels of written only)

kca	obgc	Khanty 
kcaw	kca	Khanty Written
kcal	kcaw	Khanty Written Latin Script
kcac	kcaw	Khanty Written Cyrillic Script
kcao	kcaw	Khanty Written Cyrillic Script Obdorsk Model 
kccc	kcaw	Khanty Written Cyrillic Script Central Ob Model
kcav	kcaw	Khanty Written Cyrillic Script Vakh-Vasyugan Model
kcak	kcaw	Khanty Written Cyrillic Script Kazym Model
kcam	kcaw	Khanty Written Cyrillic Script Vakh-Surgut-Shuryshkar Model

There has already been some off-list discussion on whether ISO 639-6 should
be incorporated within RFC4646bis/ter and if so, how.  Comments so far have
highlighted that there is an overlap with entities currently available in
RFC4646bis. For instance, the entity with the proposed alpha4 "kcal"
representing "Khanty Written Latin Script" could be deemed the same as
"kca-Latn-RU". These sort of problems would need to be addressed if ISO
639-6 is to be incorporated within a future RFC. Needless to say, I think
there is a need for ISO 639-6 within this RFC.  There is a need for
variants.  There is a need for a hierarchical system that will facilitate
the creation of pick lists; 7,200 entities are more than a little unwieldy
for the end user when setting language preferences!

Currently, RFC4646 has reserved the alpha4 primary language sub tag position
for ISO 639-6.  This does not mean that ISO 639-6 is definitely going to be
incorporated within a future RFC but rather that the space was reserved to
give an option for inclusion.

Given the problem with the overlap between RFC4646 conformant tags and the
meaning of some of the alpha4 identifiers of ISO 639-6, one of the possible
solutions is to include ISO 639-6 alpha4s as variant sub tags.  For example
(and I don't know what singleton would/could be used so I am using the
number "6" to indicate that the variant is an ISO 639-6 identifier):

"kcao	Khanty Written Cyrillic Script Obdorsk Model" could be tagged as:

----

kca-Latn-RU-6-kcao

----

This would give the level of precision and granularity required by some,
whilst retaining the "top level" information required for backward
compatibility purposes and some applications.

The ISO 639-6 hierarchical system would, as an estimate, increase the size
of the Registry by approximately 25,000 sub tags.  This could also be seen
as a barrier to inclusion.  However, there is no reason why these subtags
should be included within the registry.  The ISO 639-6 code tables will be
made available on-line hosted by the RA on a number of servers. The
important aspect, IMHO, is to design a mechanism for the inclusion of ISO
639-6 alpha4 identifiers within RFC4646 (bis or ter). 

ISO DIS 639-6 is scheduled for publication in January 2008.

Debbie Garside
Editor ISO DIS 639-6

Managing Director
GeoLang Ltd
www.geolang.com 

> Martin Duerst
> Sent: 14 November 2006 01:55
> To: [email-masked]; [email-masked]; [email-masked]
> Subject: Re: [Ltru] LTRU progress and ISO 639-6 (was 
> WiktionaryZ language codes)
> 
> >What exactly is going on with LTRU.  There seems to be no progress 
> >being made at all.
> 
> the issues 
> >are and in what state of discussion they are in?
> 
> I haven't officially tracked any issues, because they are 
> still too much in flux.
> 
> >Randy, Martin, I would really appreciate it if we could have 
> a list of 
> >the issues and possible solutions.  Or have I missed something? Are 
> >RFC4645/6bis on track without me realising?
> 
> I think there are several areas of issues. Doug has sent a 
> list that he need resolved for his next draft a few days ago.
> 
> >What are the timescales that we are now working to?  When 
> can we expect 
> >the next draft?  Questions, questions! And why, when it is 
> so relevant 
> >to all the variants currently under discussion 
> >(written/spoken/signed/orthographies/writing systems etc) is 
> ISO 639-6 
> >not being discussed?
> 
> I think one reason is that most people do not know very much 
> 639-6 addresses things such as 
> written/spoken/signed/orthographies/writing
> systems etc? With concrete examples, indicating whether these 
> are currently in the draft or are hypothetical. I think that 
> would help the WG to have a good disucussion.
> 
> 
> Regards,    Martin.
> 
> 
> 
> Ietf-languages mailing list
> [email-masked]
> http://www.alvestrand.no/mailman/listinfo/ietf-languages
> 
> 
> 
>

---

From: John Cowan <[email-masked]>
Date: Thu, 16 Nov 2006 01:22:08 -0500

Debbie Garside scripsit:

> Given the problem with the overlap between RFC4646 conformant tags
> and the meaning of some of the alpha4 identifiers of ISO 639-6, one
> of the possible solutions is to include ISO 639-6 alpha4s as variant
> sub tags.  For example (and I don't know what singleton would/could
> be used so I am using the number "6" to indicate that the variant is
> an ISO 639-6 identifier):
> 
> kca-Cyrl-RU-6-kcao

I think this is basically the right idea, but I'd add the following
qualifiers (this is very preliminary):

0) Before publication, 639-6 should be harmonized with ISO 15924 and ISO
3166-1 where possible; that is, new optional fields should be added to map
kcal to Latn and kcac/kcao/kccc/kcav/kcak/kcam to Cyrl, and likewise to
map any country-specific code elements to the 3166-1 code for the country.
Until this is done, I don't think 639-6 is ready for use in BCP 47.

1) In the case of a code element identical to a 639-3 code element,
the 639-6 addendum should be disallowed: kca, not kca-6-kca, obviously.

2) Supralanguage code elements (those which have no 639-3-equivalent
ancestor) should be IMHO excluded from BCP 47; they are unsuitable
for language tagging.  I'd like to see the 639-2 language collections
deprecated as well (though not removed, of course).  There is simply
no point in tagging a document as being in some Indo-European language --
the tag is too vague to be useful.  This is not to say, of course,
that these code elements are not useful for other purposes.

3) Infralanguage code elements (roughly, those that are not equivalent to
any code element, but have at least one of them as an ancestor) should
be represented using a variant rather than an extension.  We could for
example use 6kcao, which is a variant, rather than jumping through hoops
to declare a 6 extension.  The script and country values
should also be used, and the variant should be omitted if it
contains no further data:  kca-Latn, not kca-Latn-6kcal.

-- 
The first thing you learn in a lawin' family    John Cowan
is that there ain't no definite answers         [email-masked]
to anything.  --Calpurnia in To Kill A Mockingbird

---

From: John Cowan <[email-masked]>
Date: Thu, 16 Nov 2006 01:23:20 -0500

> any code element, but have at least one of them as an ancestor) should

That is, not equivalent to any 639-3 code element.

-- 
Your worships will perhaps be thinking          John Cowan
that it is an easy thing to blow up a dog?      http://www.ccil.org/~cowan
[Or] to write a book?
    --Don Quixote, Introduction                 [email-masked]

---

From: "Debbie Garside" <[email-masked]>
Date: Thu, 16 Nov 2006 09:15:36 -0000

John wrote:

> I think this is basically the right idea, but I'd add the 
> following qualifiers (this is very preliminary):
> 
> 3166-1 where possible; that is, new optional fields should be 
> added to map kcal to Latn and kcac/kcao/kccc/kcav/kcak/kcam 
> to Cyrl, and likewise to map any country-specific code 
> elements to the 3166-1 code for the country.

Can you elaborate as to your reasons for requiring this please?  It is not a
problem as this information is currently being compiled anyway.

> 1) In the case of a code element identical to a 639-3 code 
> element, the 639-6 addendum should be disallowed: kca, not 
> kca-6-kca, obviously.

This is absolutely correct. kca represents an ISO 639-3 code element
 
> ancestor) should be IMHO excluded from BCP 47; they are 
> unsuitable for language tagging.  I'd like to see the 639-2 
> language collections deprecated as well (though not removed, 
> of course).  There is simply no point in tagging a document 
> as being in some Indo-European language -- the tag is too 
> vague to be useful.  This is not to say, of course, that 
> these code elements are not useful for other purposes.

The hierarchical system facilitatates the exclusion of code elements "above"
ISO 639-3.

> 3) Infralanguage code elements (roughly, those that are not 
> equivalent to any code element, but have at least one of them 
> as an ancestor) should be represented using a variant rather 
> than an extension.  

I am not sure I understand this.  Please elaborate.

>We could for example use 6kcao, which is 
> a variant, rather than jumping through hoops to declare a 6 
> extension.  

OK, sounds reasonable.

>The script and country values should also be 
> used, and the variant should be omitted if it contains no 
> further data:  kca-Latn, not kca-Latn-6kcal.

Agreed, other than when suppress script comes into play.

Debbie

---

From: "Debbie Garside" <[email-masked]>
Date: Thu, 16 Nov 2006 09:17:56 -0000

John wrote:

> equivalent 
> > to any code element, but have at least one of them as an ancestor) 
> > should
> 

Now I'm with you.

Debbie

> From: John Cowan [mailto:[email-masked]] 
> Sent: 16 November 2006 06:23
> To: Debbie Garside
> Cc: [email-masked]
> Subject: Re: [Ltru] LTRU progress and ISO 639-6
> 
> Scripsi:
> 
> equivalent 
> > to any code element, but have at least one of them as an ancestor) 
> > should
>

---

From: John Cowan <[email-masked]>
Date: Thu, 16 Nov 2006 10:26:05 -0500

Debbie Garside scripsit:

> > 0) Before publication, 639-6 should be harmonized with ISO 15924 and
> > ISO 3166-1 where possible; that is, new optional fields should be
> > added to map kcal to Latn and kcac/kcao/kccc/kcav/kcak/kcam to Cyrl,
> > and likewise to map any country-specific code elements to the 3166-1
> > code for the country.  Until this is done, I don't think 639-6 is
> > ready for use in BCP 47.
> 
> Can you elaborate as to your reasons for requiring this please?  

It's required to make mechanical translatability possible.  To translate
'kcal' into BCP 47 style, I need to know these things:

1) By chasing the parent links, I know that the appropriate language
subtag is 'kca'.

2) By looking at the ISO 15942 mapping, I know that 'Latn' is the
appropriate script subtag.  (I then need to check the LSR to see if it
should be suppressed.)

3) By seeing that there is no ISO 3166-1 mapping, I know that adding a
region subtag is a bad idea.

4) There also needs to be a flag field in the 639-6 database which says
that this mapping is complete, that I do *not* need to add a variant
subtag of '6kcal'.  This flag would be false for 'kcao', telling me that
I *do* need a variant subtag '6kcao' in that case.  I didn't realize
the need for this field until now.

-- 
Here lies the Christian,                        John Cowan
        judge, and poet Peter,                  http://www.ccil.org/~cowan
Who broke the laws of God                       [email-masked]
        and man and metre.

---

From: "Debbie Garside" <[email-masked]>
Date: Thu, 16 Nov 2006 16:39:10 -0000

John Cowan wrote:

> It's required to make mechanical translatability possible.  
> To translate 'kcal' into BCP 47 style, I need to know these things:
> 
> 1) By chasing the parent links, I know that the appropriate 
> language subtag is 'kca'.

Yes, that's the idea.

> 
> 2) By looking at the ISO 15942 mapping, I know that 'Latn' is 
> the appropriate script subtag.  (I then need to check the LSR 
> to see if it should be suppressed.)

I can see the enormous value of this to RFC4646bis.  However, if we are
always translating the alpha4 back to the Alpha3 and/or script/country codes
would it not be better to make this a requirement of ISO 639-3?  I have to
query whether this is the job of ISO 639 in general.  One problem with this
is when you have more than one script code.  Representing this in a flat
data format is messy.  I think XML would do it nicely.  
 
> 3) By seeing that there is no ISO 3166-1 mapping, I know that 
> adding a region subtag is a bad idea.

Can you give me some examples of when ISO 3166-1 code is not required? i.e.
when it is a bad idea (I need to be sure I have the correct picture).  Also,
the same comment applies.  I can see the value but why is this mapping not
required for ISO 639-3?  Please don't misunderstand, I am not saying ISO
639-6 will not facilitate this.  
 
> 4) There also needs to be a flag field in the 639-6 database 
> which says that this mapping is complete, that I do *not* 
> need to add a variant subtag of '6kcal'.  This flag would be 
> false for 'kcao', telling me that I *do* need a variant 
> subtag '6kcao' in that case.  I didn't realize the need for 
> this field until now.

Hmmm... Yes, I can see the value of this as well.  I think I can also see
how this could be done.

Basically you are not really suggesting general mapping to ISO 3166-1/ISO
15924, what you are suggesting is a bespoke mapping to facilitate
RFC4646bis/ter tag generation.  I am happy to look at this but I want to
start with the absolute basics that are required to facilitate ISO 639-6
tags within a future RFC.  To me, those basics are items 1) and 4).  What do
you think? 

Thanks

Debbie

---

From: John Cowan <[email-masked]>
Date: Thu, 16 Nov 2006 13:36:54 -0500

Debbie Garside scripsit:

> always translating the alpha4 back to the Alpha3 and/or script/country
> codes would it not be better to make this a requirement of ISO 639-3?

No, because 639-3 identifiers don't have *specific* mappings to 15942
or 3166-1, but 639-6 identifiers do, at least some of them.  'kcal'
refers to the variety of Khanty written in the Latin script, so it's
appropriate to map it to 'Latn' in the 15942 field.

It's not a requirement to map 'kca' itself, either in 639-3 or 639-6,
to "Latn; Cyrl".  If you want to know what scripts 'kca' can be
written in, you examine its immediate children.

> One problem with this is when you have more than one script code.
> Representing this in a flat data format is messy.

As I understand it, each 639-6 code for a written variety refers to
at most one script: there is one for Khanty written in Latin, one for
Khanty written in Cyrillic, one for Khanty written in each of the
various Cyrillic orthography variants.

It would not make sense for 639-3 to carry this information.  It
does make sense for 639-6 to do so -- in fact, it's right there
in the names of the code elements.

> Can you give me some examples of when ISO 3166-1 code is not
> required? i.e.  when it is a bad idea (I need to be sure I have the
> correct picture).

Well, when the language does not come in region-specific varieties:
if your Khanty subtree is complete, there is no need for a mapping
to 3166-1, since Khanty does not vary according to the nations
in which it is spoken.

> I can see the value but why is this mapping not required for ISO 639-3?

Simply because 639-3 codes do not reflect regional variations.

> Basically you are not really suggesting general mapping to ISO
> 3166-1/ISO 15924, what you are suggesting is a bespoke mapping to
> facilitate RFC4646bis/ter tag generation.

I don't think it is bespoke.  It just encodes the information that is
already in the code element names in a machine-processable form.
I presume there is a code for "American English", so that would have
a mapping of "US" in the 3166-1 field, as would its descendants.

> I want to start with the absolute basics that are required to facilitate
> ISO 639-6 tags within a future RFC.  To me, those basics are items 1)
> and 4).  What do you think?

I think that all the fields mentioned are required to get a good
crosswalk: code element, name, parent, script, country, complete-flag.
The result would be that BCP 47 variant subtags would be required
for those, and only those, 639-6 code elements where the complete-flag
is false (and a variant subtag is needed to specify a subnational
region, an orthography variant, or whatever).

-- 
John Cowan        http://www.ccil.org/~cowan          [email-masked]
Please leave your values                Check your assumptions.  In fact,
at the front desk.                      check your assumptions at the door.
--sign in Paris hotel                   --Cordelia Vorkosigan

---

From: "Debbie Garside" <[email-masked]>
Date: Thu, 16 Nov 2006 19:19:13 -0000

John wrote:

> Debbie Garside scripsit:
> 
> > I can see the enormous value of this to RFC4646bis.  However, if we 
> > are always translating the alpha4 back to the Alpha3 and/or 
> > script/country codes would it not be better to make this a 
> requirement of ISO 639-3?
> 
> No, because 639-3 identifiers don't have *specific* mappings 
> to 15942 or 3166-1, but 639-6 identifiers do, at least some 
> of them.  'kcal'
> refers to the variety of Khanty written in the Latin script, 
> so it's appropriate to map it to 'Latn' in the 15942 field.

Yes, I see your point.

> It's not a requirement to map 'kca' itself, either in 639-3 
> or 639-6, to "Latn; Cyrl".  If you want to know what scripts 
> 'kca' can be written in, you examine its immediate children.

Yes.
 
> As I understand it, each 639-6 code for a written variety 
> refers to at most one script: there is one for Khanty written 
> in Latin, one for Khanty written in Cyrillic, one for Khanty 
> written in each of the various Cyrillic orthography variants.

Yes, I misconstrued and was looking at providing the mapping attached to the
alpha3, thus several scripts, giving a one to many relationship - my
mistake.
> 
> It would not make sense for 639-3 to carry this information.  
> It does make sense for 639-6 to do so -- in fact, it's right 
> there in the names of the code elements.

Yes it is.  But there are some (mainly older) scripts that do not have ISO
15924 codes.   How would we deal with these?
 
> > Can you give me some examples of when ISO 3166-1 code is 
> not required? 
> > i.e.  when it is a bad idea (I need to be sure I have the correct 
> > picture).
> 
> if your Khanty subtree is complete, there is no need for a 
> mapping to 3166-1, since Khanty does not vary according to 
> the nations in which it is spoken.

Here lies my biggest problem.  This would effectively have to be done by
hand looking at each code element with an ISO 639-3 identifier and coding
the children as necessary.

> I don't think it is bespoke.  It just encodes the information 
> that is already in the code element names in a 
> machine-processable form.
> I presume there is a code for "American English", so that 
> would have a mapping of "US" in the 3166-1 field, as would 
> its descendants.

Ah right, now I get it, so everywhere I have an alpha3 that has more than
one ISO 3166-1 code attached I need to allocate the correct country code to
the children.
 
> > I want to start with the absolute basics that are required to 
> > facilitate ISO 639-6 tags within a future RFC.  To me, those basics 
> > are items 1) and 4).  What do you think?
> 
> The result would be that BCP 47 variant subtags would be 
> required for those, and only those, 639-6 code elements where 
> the complete-flag is false (and a variant subtag is needed to 
> specify a subnational region, an orthography variant, or whatever).

OK.  Seems reasonable to me.

Debbie

---

From: John Cowan <[email-masked]>
Date: Thu, 16 Nov 2006 14:27:47 -0500

> 15924 codes.   How would we deal with these?

Fill out the form at http://www.unicode.org/iso15924/iso15924form.html .

> OK.  Seems reasonable to me.

Great.  How about everyone else?  To summarize:

1) No language tags for language collections except for the grandfathered
   639-2 ones.

2) All existing language tags unchanged.

3) 639-3 tags which represent a finer granularity than we have today
   are registered as variant subtags of the form '6xxxx'
   with an appropriate prefix.

Comments?

-- 
John Cowan        http://ccil.org/~cowan   [email-masked]
Lope de Vega: "It wonders me I can speak at all.  Some caitiff rogue did
rudely yerk me on the knob, wherefrom my wits still wander."
An Englishman: "Ay, a filchman to the nab betimes 'll leave a man
crank for a spell." --Harry Turtledove, Ruled Britannia

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 16 Nov 2006 20:45:14 +0100

Debbie Garside wrote:

>> The script and country values should also be used,
>> and the variant should be omitted if it contains
>> no further data:  kca-Latn, not kca-Latn-6kcal.

> Agreed, other than when suppress script comes into play.

Yes, if nap has an Suppress-Script Latn (as it should),
then nap-6-napl isn't the same as nap-Latn, because
nap-Latn is reduced to nap for most practical purposes.

In other words nap-6-napl is okay in 4646ter territory.

That's a bit tricky for a future equivalence table.  If
nap has no Suppress-Script Latn, because nobody bothered
to register it, you have "use nap-Latn, not nap-6-napl,
let alone nap-Latn-6-napl".

If that's later fixed by adding a proper Suppress-Script
nap-6-napl suddenly makes sense.  And any old nap-Latn
tags are dubious.

One point with the Suppress-Scripts is that we must get
it right a.s.a.p.  4646bis should be put on hold until
the 4646 registry without 639-3 is complete.

Frank

---

From: "Debbie Garside" <[email-masked]>
Date: Thu, 16 Nov 2006 19:50:56 -0000

>    with an appropriate prefix.

That should read:

 3) 639-6 tags which represent a finer granularity than we have today
    are registered as variant subtags of the form '6xxxx'
    with an appropriate prefix.

Debbie

> Sent: 16 November 2006 19:28
> To: Debbie Garside
> Cc: [email-masked]
> Fill out the form at 
> http://www.unicode.org/iso15924/iso15924form.html .
> 
> Lope de Vega: "It wonders me I can speak at all.  Some 
> caitiff rogue did rudely yerk me on the knob, wherefrom my 
> wits still wander."
> An Englishman: "Ay, a filchman to the nab betimes 'll leave a 
> man crank for a spell." --Harry Turtledove, Ruled Britannia
> 
> 
> 
> 
>

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 16 Nov 2006 17:08:58 -0800

Hi -

As a technical contributor...

> From: "John Cowan" <[email-masked]>
> To: "Debbie Garside" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Thursday, November 16, 2006 11:27 AM
...
> 1) No language tags for language collections except for the grandfathered
...

I disagree.  I think it is simpler to *not* prohibit language collections.
Yes, some of them may have limited utility, but that is not a good
reason for prohibiting them, in my opinion.  If it results in a simpler
specification, while permitting meaningful tags that would otherwise
be prohibited, I think the choice is simple.

(Considering the cataloguer's plight, I think that tags for language
collections could be very useful for material whose language can be
roughly categorized, pending a closer look by a specialist.  "zh" is
already used in this way.)

Randy

---

From: John Cowan <[email-masked]>
Date: Fri, 17 Nov 2006 00:01:50 -0500

> I disagree.  I think it is simpler to *not* prohibit language collections.

It's *simpler* not to prohibit anything, but the trend of RFC 4646
is to set more limits so that people can rely on things.

> be prohibited, I think the choice is simple.

Of course, but meaningful for what?  What good does it do to be
able to tag a document "ger" (Germanic languages)?  It doesn't
tell you anything useful about it.

> already used in this way.)

"zh" is a macrolanguage, not a collection.  The only way to know if
something is North Germanic is to recognize it as Swedish, Danish,
Norwegian, Icelandic, Faroese, ...  and consult the current definition
of the term.  (Okay, *that* term is stable, but plenty of other
language-collection terms aren't; languages move in and out all the time.)

-- 
Winter:  MIT,                                   John Cowan
Keio, INRIA,                                    [email-masked]
Issue lots of Drafts.                           http://www.ccil.org/~cowan
So much more to understand!
Might simplicity return?                        (A "tanka", or extended haiku)

---

From: John Cowan <[email-masked]>
Date: Fri, 17 Nov 2006 01:48:04 -0500

> nap-Latn is reduced to nap for most practical purposes.

I don't understand this.  The meaning of saying that napl has
a Suppress-Script of Latn is that nap and nap-Latn are
synonymous.

> In other words nap-6-napl is okay in 4646ter territory.

What for?

-- 
On the Semantic Web, it's too hard to prove     John Cowan    [email-masked]
you're not a dog.  --Bill de hOra               http://www.ccil.org/~cowan

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 16 Nov 2006 22:41:34 -0800

Hi -

> To: "Randy Presuhn" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Thursday, November 16, 2006 9:01 PM
...
...

Actually, this is a good example.  If I were helping to catalog
material in a library, I know the major East and West Germanic
languages well enough to tag material in those languages.
I also know enough to recognize something as being in a North
Germanic language, but for a more precise tagging I'd just to
hand it off to a "specialist". If the specialist wasn't handy,
being able to tag it as "North Germanic", even as a temporary
measure, would be quite useful, even if the category is fuzzy
or unstable.

There are times when fuzziness can be a feature, rather than a bug.

Folks that don't need these don't need to use them.  I'd just hate
to prohibit something that could potentially be useful, especially
when I don't yet see any clear benefit arising from the prohibition.

Randy

---

From: "Debbie Garside" <[email-masked]>
Date: Fri, 17 Nov 2006 08:54:45 -0000

Randy wrote:

> Actually, this is a good example.  If I were helping to 
> catalog material in a library, I know the major East and West 
> Germanic languages well enough to tag material in those languages.
> hand it off to a "specialist". If the specialist wasn't 
> handy, being able to tag it as "North Germanic", even as a 
> temporary measure, would be quite useful, even if the 
> category is fuzzy or unstable.
> 
> Folks that don't need these don't need to use them.  I'd just 
> hate to prohibit something that could potentially be useful, 
> especially when I don't yet see any clear benefit arising 
> from the prohibition.

I agree with your sentiments.  However, as far as I know the library
community do not use RFC4646 (or do they?).

John is not proposing that these tags be taken out of ISO 639-6 altogether
but rather that they be flagged as not to be used in RFC4646. If there is a
possibility of a legitimate use case then they should, perhaps, stay in.

There is a (slightly messy) way around this and that is to create a rule
that says "-6kcal" can be used as a variant but the upper levels can be used
in the primary language sub tag position; as the alpha4 space is already
reserved.  In other words create a rule that states, if the 639-6 tag is
flagged for use and has a 639-3 tag as a parent then use as variant
otherwise use as primary language subtag. The rule could also state that
alpha4s used in the Primary Language subtag position should include no other
subtags (other than, possibly, extensions).

Debbie

---

From: "Debbie Garside" <[email-masked]>
Date: Fri, 17 Nov 2006 09:15:09 -0000

Frank wrote:

> Yes, if nap has an Suppress-Script Latn (as it should), then 
> nap-6-napl isn't the same as nap-Latn, because nap-Latn is 
> reduced to nap for most practical purposes.

> In other words nap-6-napl is okay in 4646ter territory.

I don't think it is. If "nap" is equal to "nap-Latn" (as it is with suppress
script) and "napl" is also equal to  "nap-Latn" there lieth an overlap. But
I do sort of see where you are coming from because essentially "nap"
represents just the language and not the language plus script.   Is there a
liklihood that "nap-6napl" would be useful? I suppose if we suddenly had a
need for "nap-Grek" it would be.  Hmmm... I can see a glimmer of a need.
The rules for suppress script, once invoked are they set in stone?  Could
you suppress "nap" now and unsuppress in the future?  

Debbie

> From: Frank Ellermann [mailto:[email-masked]] 
> Sent: 16 November 2006 19:45
> To: [email-masked]
> Subject: [Ltru] Re: LTRU progress and ISO 639-6
> 
> Debbie Garside wrote:
> 
> >> The script and country values should also be used, and the variant 
> >> should be omitted if it contains no further data:  kca-Latn, not 
> >> kca-Latn-6kcal.
> 
> Yes, if nap has an Suppress-Script Latn (as it should), then 
> nap-6-napl isn't the same as nap-Latn, because nap-Latn is 
> reduced to nap for most practical purposes.
> 
> That's a bit tricky for a future equivalence table.  If nap 
> has no Suppress-Script Latn, because nobody bothered to 
> register it, you have "use nap-Latn, not nap-6-napl, let 
> alone nap-Latn-6-napl".
> 
> nap-6-napl suddenly makes sense.  And any old nap-Latn tags 
> are dubious.
> 
> One point with the Suppress-Scripts is that we must get it 
> right a.s.a.p.  4646bis should be put on hold until the 4646 
> registry without 639-3 is complete.
> 
> Frank
> 
> 
>

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 17 Nov 2006 13:17:06 +0100

Randy Presuhn wrote:

> Our charter says nothing about waiting for the state of the
> registry to reach any particular state before we complete
> our work in the ltru WG, and, given the very nature of a
> registry, there is no technical reason to impose such a
> requirement.

As far as I'm concerned 4646bis goes nowhere without completing
the Suppress-Scripts or revising the procedure to add them.

The 4646 idea was to let the review list complete this in any
way they like, and so far they completed no Suppress-Scripts
in about a year.

We have that all documented in 4646-tickets, in the LTRU list
archive, and about four attempts to add Suppress-Scripts on
the review list (two individual, and John's two proposals for
bulk updates).

A plausible plan was to get it right for languages adopted
from 639-1 / 639-2, and ignore the future 639-3 languages as
less interesting wrt this compatibility kludge.  So far the
4645bis registry won't tell us the source of a language subtag,
this plan works reliably only _before_ 4645bis is last called.

We could also decide it here directly in 4645bis, as proposed
by the current subtag reviewer.  I don't care how it's solved
before a 4645bis "last call", but we've now seen what happens
if we delegate this taks:  Twice as nothing.

The charter says "evaluate the 4646 procedure, adjust it as
needed" or similar.  I'm no big fan of the "review committee"
idea proposed here some weeks ago, but if that's what it takes
to get many missing Suppress-Scripts right, why not... <sigh>

Frank

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 17 Nov 2006 13:22:10 +0100

John Cowan wrote:
 
>> if nap has an Suppress-Script Latn (as it should),
 
> I don't understand this.  The meaning of saying that napl
> has a Suppress-Script of Latn is that nap and nap-Latn
> are synonymous.

nap has (or rather should or could have) a Suppress-Script
Latn.  For the purposes of 4646 and backwards compatibility
a tag nap is then better (SHOULD) than nap-Latn.

For the purposes of 639-6 napl is a proper subset of nap,
napl implies "written using Latn".  For a 639-6 nap that's
not the case, it can be spoken, or written using another
script, etc., including but not limited to all 639-6 code
elements with nap as their parent.  In other words nap and
napl are not synonymous wrt 639-6.

But nap and nap-Latn are (or could be) to some degree wrt
4646 ff.  Taggers going to the trouble of using 639-6 as
extension or variant could try to preserve that relation
using nap-6-napl or nap-6napl, intuitively.

I'm not sure about your variant idea, maybe it's important
for the outcome.  For an extension it's clearer, a tag like
nap-6-napl is okay, taggers using extensions aren't really
interested in backwards compatible Suppress-Script kludges.

...BUT...

At the moment nap doesn't have a Suppress-Script Latn.  With
that a tag nap-Latn is perfectly okay, it doesn't violate 
the SHOULD about avoiding Suppress-Script subtags, at worst
it violates a general SHOULD about tagging wisely.  Based on
that nap-6-napl or nap-6napl are "verboten", because there's
a short and simple nap-Latn for this purpose.

The outcome depends on accurate Suppress-Scripts, as far as
that's possible.  Switching them later without compelling
reasons is messy.

We know this, it was clearly understood for the introduction
of 4646, and "let the review list complete the stuff found
in an earlier CLDR database" was only to make sure that no
nonsense (Tatar and similar cases) goes into the registry.

But the precise number of Suppress-Scripts completed by the
review list in about a year is ZERO, not counting one case
of Nkoo, where it was done simultaneously with the addition
of ngo.

Frank

---

From: John Cowan <[email-masked]>
Date: Fri, 17 Nov 2006 11:30:52 -0500

> > category is fuzzy or unstable.

Okay, you've convinced me.

> There is a (slightly messy) way around this and that is to create a
> rule that says "-6kcal" can be used as a variant but the upper levels
> can be used in the primary language sub tag position; as the alpha4
> space is already reserved.  In other words create a rule that states,
> if the 639-6 tag is flagged for use and has a 639-3 tag as a parent then
> use as variant otherwise use as primary language subtag. The rule could
> also state that alpha4s used in the Primary Language subtag position
> should include no other subtags (other than, possibly, extensions).

That sounds like the Right Thing to me.  It does add complications to
the method of treating 639-6 tags, but I think there is no getting
away from it:  the tag for "Indo-European languages" simply can't be
made subordinate to anything language subtag.

-- 
John Cowan    [email-masked]    http://ccil.org/~cowan
Rather than making ill-conceived suggestions for improvement based on
uninformed guesses about established conventions in a field of study with
which familiarity is limited, it is sometimes better to stick to merely
observing the usage and listening to the explanations offered, inserting
only questions as needed to fill in gaps in understanding. --Peter Constable

---

From: Martin Duerst <[email-masked]>
Date: Sat, 18 Nov 2006 11:45:46 +0900

>Agreed, other than when suppress script comes into play.

Just an idea that came up: It looks like some of the 639-6 data
could help figuring out some more suppress-scripts (or otherwise
decide on how to solve that issues).

What I'm assuming is that the 639-6 data might be a good source
for what language is written with what scripts. Just an idea.

Regards,    Martin.

---

From: Martin Duerst <[email-masked]>
Date: Sat, 18 Nov 2006 11:45:46 +0900

Regards,    Martin.

---

From: Frank Ellermann <[email-masked]>
Date: Sun, 19 Nov 2006 14:56:52 +0100

Debbie Garside wrote:

> Is there a liklihood that "nap-6napl" would be useful? I suppose if we
> suddenly had a need for "nap-Grek" it would be.  Hmmm... I can see a
> glimmer of a need.

If "napl" would be never interesting you probably wouldn't list it (?)
You're not forced to consider backwards compatibility with 3066 tags and
applications.  From your POV the complete "Suppress-Script" business in
4646 is plain nonsense, you want as much info as you can possibly cram
into alpha4 code elements.  I think, please correct me... :-)

> The rules for suppress script, once invoked are they set in stone?

No.  A Suppress-Script won't jump from Aaaa to Bbbb - unless Aaaa was an
unlikely error, confusing Latg and Goth, or something in that direction.
But adding or removing a Suppress-Sript is possible.

> Could you suppress "nap" now and unsuppress in the future?

Unsuppress is unlikely in that case, why would they start to use a new
script in addition to Latn, justifying to explicitly distinguish Latn
and this new script in language tags ?

The problem we have now is the missing Suppress-Script Latn, which could
be introduced later.  At the moment a tag nap-Latn is perfectly okay,
and therefore nap-6-napl or nap-Latn-6-napl would be wrong / redundant.

As soon as nap gets a Suppress-Script Latn the nap-Latn-6-napl is still
wrong / redundant.  But suddenly nap-6-napl would be allowed instead of
the then suppressed nap-Latn.  Kind of messy.  IMO we must finish this
kludge for the 4646 languages, hoping that it will be almost never used
for the 639-3 languages, and almost never changed where it's used.  The
"hope" could go as far as a NOT RECOMMENDED.

It was a huge mistake to delegate this task to the review list, and of
course it's impossible to deal with it as soon as thousands of 639-3
languages are in.  It's mainly relevant for existing 3066 tags, and for
existing 3066 applications.  For the new 639-3 languages we can demand
that taggers get this right, adding script subtags where they feel that
it's necessary.

For existing 3066 tags it's different, the taggers couldn't add scripts
even where it's necessary before it was allowed by 4646.  The about ten
languages added after 4646 was introduced are a border case, for "ngo"
a Suppress-Script was added, for "frr" and "frs" it wasn't.  That's no
consistent solution, IMO we've to fix it for all language subtags added
before the 4645bis bulk update.

Frank

---

From: "Debbie Garside" <[email-masked]>
Date: Sun, 19 Nov 2006 17:27:07 -0000

Martin wrote:

> Just an idea that came up: It looks like some of the 639-6 
> data could help figuring out some more suppress-scripts (or 
> otherwise decide on how to solve that issues).
> 
> What I'm assuming is that the 639-6 data might be a good 
> source for what language is written with what scripts. Just an idea.

This is true.  However, I would only be happy to recommend suppress-script
options once the data has been verified.

Best 

Debbie

Debbie Garside
www.geolang.com

---

From: "Debbie Garside" <[email-masked]>
Date: Sun, 19 Nov 2006 17:55:08 -0000

Frank wrote:

> If "napl" would be never interesting you probably wouldn't 
> list it (?) You're not forced to consider backwards 
> compatibility with 3066 tags and applications.  From your POV 
> the complete "Suppress-Script" business in
> 4646 is plain nonsense, you want as much info as you can 
> possibly cram into alpha4 code elements.  I think, please 
> correct me... :-)

Yes, from the POV of ISO 639-6 suppress-script is not relevant; napl is
relevant to other users of ISO 639-6 as the system, necessarily, needs to
offer "stand-alone" tags.  However, from the POV of extension coding and my
participation in this WG it is and if ISO 639-6 data can help RFC4646bis and
the compilation of data for the Registry, all well and good.

> No.  A Suppress-Script won't jump from Aaaa to Bbbb - unless 
> Aaaa was an unlikely error, confusing Latg and Goth, or 
> something in that direction.
> Unsuppress is unlikely in that case, why would they start to 
> use a new script in addition to Latn, justifying to 
> explicitly distinguish Latn and this new script in language tags ?

As a hypothetical case: Say, for instance, that an archaeological dig
unearthed materials written in nap-Grek; the academic community would then
be interested in searching/archiving/matching nap-Grek. 
 
> The problem we have now is the missing Suppress-Script Latn, 
> which could be introduced later.  At the moment a tag 
> nap-Latn is perfectly okay, and therefore nap-6-napl or 
> nap-Latn-6-napl would be wrong / redundant.

Why would it be wrong? nap-Latn-6-napl means the same as nap-Latn.  The only
thing wrong would be that it is not tagging content wisely. I seem to
remember a paragraph in RFC4646 that states to (SHOULD) use only the tags
that are necessary to accurately identify the object.  
 
> As soon as nap gets a Suppress-Script Latn the 
> nap-Latn-6-napl is still wrong / redundant.  But suddenly 
> nap-6-napl would be allowed instead of the then suppressed 
> nap-Latn.

I am not sure I really understand this.  Surely if nap was allocated the
suppress-script Latn, then nap becomes equal to nap-Latn.  Therefore, if
nap-Latn-6-napl is disallowed prior to the suppress-script then nap-6-napl
is also disallowed subsequently as in effect its meaning is the same as
nap-Latn-6-napl.  Or maybe I am just completely suppress-script illiterate
:-)

>  Kind of messy.  IMO we must finish this kludge for 
> the 4646 languages, hoping that it will be almost never used 
> for the 639-3 languages, and almost never changed where it's 
> used.  The "hope" could go as far as a NOT RECOMMENDED.

By "NOT RECOMMENDED" I assume you mean not recommended to suppress scripts?
> 
> It was a huge mistake to delegate this task to the review 
> list, and of course it's impossible to deal with it as soon 
> as thousands of 639-3 languages are in. 

Can you remind me of the reasoning behind suppress-script being included?
(perhaps off-list as everyone has heard it before).  All I can remember is
that there are applications and protocols that cannot deal with script tags.

Best

Debbie

Debbie Garside
www.qeolang.com

---

From: Frank Ellermann <[email-masked]>
Date: Sun, 19 Nov 2006 20:09:26 +0100

Debbie Garside wrote:

> if ISO 639-6 data can help RFC4646bis and the compilation of data for
> the Registry, all well and good.

...and maybe avoid some discussions about the best variant subtag, what
would 639-6 offer for say the arevela / arevmda subtags ?

> unearthed materials written in nap-Grek; the academic community would
> then be interested in searching/archiving/matching nap-Grek.

That's no problem, if Latn is suppressed anything else is still okay.

They could also unearth tons of de-Latf books, it wouldn't affect the
Suppress-Script Latn for de.

>> At the moment a tag nap-Latn is perfectly okay, and therefore
>> nap-6-napl or nap-Latn-6-napl would be wrong / redundant.
 
> Why would it be wrong? nap-Latn-6-napl means the same as nap-Latn.

Because John's algorithm to eliminate dupes / redundancies would say
that nap-Latn is good enough, no 6napl or 6-napl needed or allowed.

Maybe I'm wrong, but apparently that algorithm determines all 639-6
code elements already covered by other means (script, region, variant),
and only the rest of 639-6 (not already covered) is used to form tags.

So if 639-6 offers "enlu" with the same meaning as en-Latn-US, then
en-Latn-US wins, and "enlu" is never used in 4646ter tags, and also
no candidate for a 4646bis variant "6enlu".

But if 639-6 offers "enny" for en as in New York it's finer than any
RFC 4646 tag (at the moment), and en-6enny or en-6-enny can be used -
maybe directly with 4646ter.  Or registered as variant 6enny on demand,
and then used.  Or as the extension 6-enny as soon as the 6-extension
registry is specified and populated.

> I seem to remember a paragraph in RFC4646 that states to (SHOULD)
> use only the tags that are necessary to accurately identify the
> object.

John's algorithm as an application of this SHOULD ?  I thought it's
a MUST.  I wouldn't like a situation where nap-Latn and nap-6-napl
are in theory both possible.

> Surely if nap was allocated the suppress-script Latn, then nap 
> becomes equal to nap-Latn.  Therefore, if nap-Latn-6-napl is 
> disallowed prior to the suppress-script then nap-6-napl is also 
> disallowed subsequently as in effect its meaning is the same as
> nap-Latn-6-napl.

Why should nap-6-napl be disallowed ?  It's Suppress-Script: Latn,
not Suppress-Extension: 6-napl.  It's less fuzzy than nap after the
Suppress-Script suppressed nap-Latn.

> Or maybe I am just completely suppress-script illiterate :-)

Could be also me, but nap-6-napl is written, for nap we're not sure.
The Suppress-Script intentionally removes some info for the higher
goal of backwards compatibility and better matching.  If somebody
uses nap-6-napl they intentionally don't care about that. 

Maybe we could decree that a Suppress-Script also affects 639-6 code
elements identifying the supressed script if used in language tags,
then it's as you say, all would be disallowed.  So far I assumed 
that the Suppress-Script only affects script subtags, not 639-6 (?)

>> messy.  IMO we must finish this kludge for the 4646 languages, 
>> hoping that it will be almost never used for the 639-3 languages,
>> and almost never changed where it's used.  The "hope" could go
>> as far as a NOT RECOMMENDED.
 
> By "NOT RECOMMENDED" I assume you mean not recommended to suppress

Yes, for anything added by 4645bis and afterwards.  Doug said that
there's no chance to figure this out for thousands of languages, or
to deal with it piecemeal on the review list.  For the about 500
languages registered now, almost all old 3066-languages, including
"nap", it's still possible.

> All I can remember is that there are applications and protocols
> that cannot deal with script tags.

That's all, they can handle en or en-US, but not en-Latn-US.  And of
course most existing documents using these tags won't have scripts,
it's a new 4646-invention.

Frank

---

From: "Randy Presuhn" <[email-masked]>
Date: Sun, 19 Nov 2006 17:25:50 -0800

Hi -

> From: "Debbie Garside" <[email-masked]>
> To: <[email-masked]>; <[email-masked]>
> Sent: Sunday, November 19, 2006 9:55 AM
> Subject: RE: [Ltru] Re: LTRU progress and ISO 639-6
...
...

As I see it...

Supress-script exists to encourage consistent tagging.
It is only needed for those languages which meet BOTH of
these criteria:
    - there is already a significant amount of material tagged
      for the language, but without any script subtag
    - virtually all material in that language is written in the
      same script

If the language is written in multiple scripts, Suppress-script
clearly doesn't apply.  If there is not a significant amount
of material already tagged (yes, this is a judgment call, and
I hope [email-masked] would be cautious) there
is also no reason to use Suppress-script for that language,
since there is not an existing "base" with which compatibility
is needed.

Randy

---

From: John Cowan <[email-masked]>
Date: Sun, 19 Nov 2006 20:58:41 -0500

>     - virtually all material in that language is written in the

I would even add a third criterion: that the language is likely
to be used with a region-tag.  The problem being solved is that "en-US"
will not match "en-Latn-US" properly; there really isn't a
problem with "ka" vs. "ka-Geor", say, though we do have it suppressed.

> since there is not an existing "base" with which compatibility

+1

-- 
John Cowan                                [email-masked]
At times of peril or dubitation,          http://www.ccil.org/~cowan
Perform swift circular ambulation,
With loud and high-pitched ululation.

---

From: John Cowan <[email-masked]>
Date: Sun, 19 Nov 2006 21:29:52 -0500

> that nap-Latn is good enough, no 6napl or 6-napl needed or allowed.

Just so.

> and only the rest of 639-6 (not already covered) is used to form tags.

Exactly, with the addendum that a 639-6 tag that covers multiple languages
would be used directly as an unqualified 4-alpha tag.

Yes.

> registry is specified and populated.

Yes, though I think the first of these is the best, as it involves
the smallest amount of process for users.

> uses nap-6-napl they intentionally don't care about that. 

Saying that 'nap' has a Suppress-Script of 'Latn' means that
we've decided as a matter of public policy that the distinctions
between Neapolitan unspecified, Neapolitan spoken, and Neapolitan
written in Latin are insignificant.  Reintroducing a way of making
those distinctions through the back door is a Bad Thing.

-- 
John Cowan   [email-masked]    http://ccil.org/~cowan
I come from under the hill, and under the hills and over the hills my paths
led. And through the air. I am he that walks unseen.  I am the clue-finder,
the web-cutter, the stinging fly. I was chosen for the lucky number.  --Bilbo

---

From: John Cowan <[email-masked]>
Date: Sun, 19 Nov 2006 21:35:57 -0500

> and this new script in language tags ?

Politics.  If Greece reconquered Neapolis (or bought it back from
Italy, more likely), nap-Grek might well become an important language.

> the then suppressed nap-Latn.  

No, I disagree.  Quite independent of Suppress-Script, 'napl' does not
contain any extra information above "nap" or "nap-Latn" as the case
may be.  So it remains forbidden to use either "nap-6napl" or
"nap-Latn-6napl".  See my previous post for the rationale.

-- 
Evolutionary psychology is the theory           John Cowan
that men are nothing but horn-dogs,             http://www.ccil.org/~cowan
and that women only want them for their money.  [email-masked]
        --Susan McCarthy (adapted)

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 20 Nov 2006 19:40:32 +0100

John Cowan wrote:

>> apparently that algorithm determines all 639-6 code elements already
>> covered by other means (script, region, variant),> and only the rest
>> of 639-6 (not already covered) is used to form tags.

> Exactly, with the addendum that a 639-6 tag that covers multiple
> languages would be used directly as an unqualified 4-alpha tag.

Interesting, stuff for a 4646ter when 639-6 is published.

> written in Latin are insignificant.  Reintroducing a way of
> making those distinctions through the back door is a Bad Thing.

Why's that bad ?  In another article you or Randy pointed out that
the worst thing about e.g. fy-Latn-DE is the _position_ of Latn
between fy and DE confusing old applications looking for fy-DE.

Variants or extensions are noted after the region, unlike script
subtags.  Some nap-FR-6napl could still match nap-FR (constructed
example, it's not about Corsica), unlike nap-Latn-FR.

Frank

---

From: Addison Phillips <[email-masked]>
Date: Tue, 21 Nov 2006 11:22:13 -0800

The more I think about this thread, the more I'm of the opinion that ISO 
639-6 either defines an extension or defines a parallel standard.

As near as I can tell, most of the codes in ISO 639-6 have a 1:1 
relationship with some language tag. That is, they either map to 
currently defined language tags using one or more subtags (napl => 
nap-Latn) or they define a variant which is eligible for registration.

I oppose including redundant subtags in the registry. It will confuse 
people to no good purpose. Language tags, for various historical and 
practical reasons, consist of subtags. ISO 639-6 codes might serve as a 
useful reference when registering variants (perhaps a restriction should 
exist requiring variant registrations collect a rejection slip form ISO 
639-6 RA before independently being registered, a la language subtags??) 
and might even serve as the source for specific variants not currently 
registered in some 4646ter timeframe. That is, any ISO 639-6 codes that 
don't have an associated language tag would be registered using some 
format such as the below to identify the variation.

Just my current thoughts on the matter.

Addison

-- 
Addison Phillips
Globalization Architect -- Yahoo! Inc.

Internationalization is an architecture.
It is not a feature.

---

From: John Cowan <[email-masked]>
Date: Tue, 21 Nov 2006 15:22:13 -0500

Addison Phillips scripsit:

> The more I think about this thread, the more I'm of the opinion that
> ISO 639-6 either defines an extension or defines a parallel standard.

I disagree with this conclusion, while agreeing with most of your
specific points.

> nap-Latn) or they define a variant which is eligible for registration.

I agree with this, although I would put it more precisely thus:  most
of the codes in 639-6 have a 1:1 relationship with some *potential*
language tag.

Randy has expressed a desire to allow the 639-6 language-collection
tags to be made available also, and the reserved syntax for 4-alphas in
the language subtag position makes this possible.  In 639-6, a language
collection is any code element that does not have a 639-3-defined language
ancestral to it.  Some of these are already included by virtue of their
presence in 639-2.

> people to no good purpose.

Agreed.

> Language tags, for various historical and practical reasons, consist
> of subtags.

Indeed.

> ISO 639-6 codes might serve as a useful reference when registering
> variants (perhaps a restriction should exist requiring variant
> registrations collect a rejection slip form ISO 639-6 RA before
> independently being registered, a la language subtags??)

I agree that this is a good idea for the 4646ter process.

> registered in some 4646ter timeframe. That is, any ISO 639-6 codes
> that don't have an associated language tag would be registered using
> some format such as the below to identify the variation.

That is precisely what I propose be done, en masse, in 4645ter:

        639-6 language-collection code elements to be registered as
        language subtags;

        639-6 code elements that are exactly equivalent to some valid
        4646bis language tag to be excluded from 4646ter;

        all other 639-6 code elements to be accepted as variants in
        4646ter, prefixed with '6' to make them long enough and unique,
        and with a Prefix: value corresponding to the appropriate language,
	script (if any), and region (if any) subtags associated with that
        code element.

This leaves open the fate of existing variants that match 639-6 code elements:
should they be deprecated in favor of their 639-6 equivalents, or left alone
and the 639-6 equivalent excluded?

-- 
Knowledge studies others / Wisdom is self-known;      John Cowan
Muscle masters brothers / Self-mastery is bone;       [email-masked]
Content need never borrow / Ambition wanders blind;   http://ccil.org/~cowan
Vitality cleaves to the marrow / Leaving death behind.    --Tao 33 (Bynner)

---

From: Peter Constable <[email-masked]>
Date: Tue, 21 Nov 2006 17:01:14 -0800

In my mind, this raises the question of what use having "napl" in 639-6 is anyway? It has the same semantics as "nap-Latn" and I don't see a point in having a metadata element to denote a semantic for which we already have an existing metadata element unless someone has a syntactic requirement -- their metadata elements must all be alpha4 -- but I see no indication of anyone having such a requirement.

Peter

---

From: John Cowan <[email-masked]>
Date: Tue, 21 Nov 2006 20:33:19 -0500

Peter Constable scripsit:

> In my mind, this raises the question of what use having "napl" in 639-6
> is anyway? It has the same semantics as "nap-Latn" and I don't see
> a point in having a metadata element to denote a semantic for which
> we already have an existing metadata element unless someone has a
> syntactic requirement -- their metadata elements must all be alpha4 --
> but I see no indication of anyone having such a requirement.

This is off-topic, but the theory of 639-6 is that there is a single
flat code space (mostly 4-alpha, but 3-alpha where a code element
has the same semantics as one from the 639-2/3/5 code space) for everything
from phyla to varieties, with a straightforward tree structure superimposed.
The identifiers are invariant, but the tree structure is subject to change.

The closest analogue is the Linnaean tree, if we treat binomial species
and trinomial subspecies names as single identifiers; however, Linnaean
identifiers are not even close to fixed-length.

I agree that there is no point in allowing 'napl' into BCP 47 codespace,
since it does indeed mean the same as 'nap-Latn'.

-- 
John Cowan  [email-masked]  http://ccil.org/~cowan
The competent programmer is fully aware of the strictly limited size of his own
skull; therefore he approaches the programming task in full humility, and among
other things he avoids clever tricks like the plague.  --Edsger Dijkstra

---

From: "Debbie Garside" <[email-masked]>
Date: Wed, 22 Nov 2006 08:45:16 -0000

Peter wrote:

> In my mind, this raises the question of what use having 
> "napl" in 639-6 is anyway? It has the same semantics as 
> "nap-Latn" and I don't see a point in having a metadata 
> element to denote a semantic for which we already have an 
> existing metadata element unless someone has a syntactic 
> requirement -- their metadata elements must all be alpha4 -- 

Not everyone wants to use extension coding.  There is a need for an alpha4
"stand alone" tag.  By mapping the stand alone alpha4s to the extension
mechanism, they can translated either way.

Debbie

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 22 Nov 2006 18:13:25 +0100

> and the 639-6 equivalent excluded?

Deprecate the ad-hoc-variants.  Same idea as with i-whatever in 4646.

Frank