Subject: [Ltru] Proposed edits (davis)

From: "Mark Davis" <[email-masked]>
Date: Wed, 13 Feb 2008 11:01:12 -0800

I went through the document a final time. It is in very good shape,
thanks to the many contributions on this list.

I do have the following suggested edits, numbered for reference. My
notes are in [...]

Mark

-----

MD1
   values are not recognized.  Thus, a parser need not have an up-to-
   date copy (or any copy at all) of the subtag registry to perform most
   searching and matching operations.
=>
   values are not recognized.  Thus, a parser need not have an up-to-
   date copy (or any copy at all) of the subtag registry to perform common
   searching and matching operations.

[most => common]

---------
MD2

   language      = (2*3ALPHA) ; shortest ISO 639 code
                 / 4ALPHA                 ; reserved for future use
                 / 5*8ALPHA               ; registered language subtag
=>
   language      = (2*3ALPHA) ; shortest ISO 639 code
                 / 4ALPHA                 ; reserved for possible future use
                 / 5*8ALPHA               ; registered language subtag

[insert 'possible', matching description below.]

-----
MD3
   appropriate) in forming a language tag with the variant.  That is,
   each of the subtags in the prefix SHOULD appear before the variant.
   For example, the subtag 'nedis' has a Prefix of "sl", making it
=>
   appropriate) in forming a language tag with the variant.  That is,
   each of the subtags in the prefix SHOULD appear, in order, before
the variant.
   For example, the subtag 'nedis' has a Prefix of "sl", making it

[add 'in order']

------
MD4
   For example: Users who wished to utilize codes from the Ethnologue
   publication of SIL International for language identification might
   agree to exchange tags such as "az-Arab-x-AZE-derbend".  This example
   contains two private use subtags.  The first is 'AZE' and the second
   is 'derbend'.
=>
   For example: The Unicode Consortium defines a set of private use
extensions in LDML (Locale Data Markup Language), such as
en-US-x-ldml-POSIX-k-calendar-islamic-k-colStren-secondar, which
indicates that en-US is being used, but with the (non-standard) POSIX
variant is being used, and that formatting of dates should use the
islamic calendar, with case being ignored in sorted lists. For more
information, see [LDML].

[Reason: replace a hypothetical example with a real one.]

-----
MD5
   to validate the tag.  A more-recent copy of the registry might
=>
   to validate the tag.  A more recent copy of the registry might
[typo]

-----
MD6
   The first record in the registry is a "File-Date" record.  This
   record contains the single field whose field-name is "File-Date" (see
   Figure 4).  The field-body of this record contains the last
   modification date of this copy of the registry, making it possible to
   compare different versions of the registry.  The registry on the IANA
   website is the most current.  Versions with an older date than that
   one are not up-to-date.

   File-Date: 2004-06-28
   %%
=>
   The first record in the registry is a "File-Date" record.  This
   record contains a required first field whose field-name is "File-Date" (see
   Figure 4).  The field-body of File-Date contains the last
   modification date of this copy of the registry, making it possible to
   compare different versions of the registry.  The registry on the IANA
   website is the most current.  Versions with an older date than that
   one are not up-to-date.

  Subsequent fields in the File-Date record are optional Comment fields.
  Comments contains additional information about the registry, as
  deemed appropriate for understanding the current state of the registry.

  For example:

   File-Date: 2009-09-10
   Comment: Updated on 2009-09-10 to add new codes from ISO 639-3.
   Comment: Updated on 2009-06-14 to add new codes from ISO 3166.
   Comment: Updated on 2008-07-12 to in accordance with RFC XXX (4646bis).
   %%

-----
MD7
         variant 'nedis' is 'sl', meaning that the tags "sl-nedis" and
         "sl-IT-nedis" might be appropriate while the tag "is-nedis" is
         not.
=>
         variant 'nedis' is 'sl', meaning that the tags "sl-nedis" and
         "sl-IT-nedis" are appropriate while the tag "is-nedis" is
         not.
[consistence mood]

-----
MD8
   replace the content of the source standard itself.  The descriptions
   are not intended to be the localized English names for the subtags.
   Localization or translation of language tag and subtag descriptions
   is out of scope of this document.
=>
   replace the content of the source standard itself.  The descriptions
   are not intended to be the localized English names for the subtags.
   Localization or translation of language tag and subtag descriptions
   is out of scope of this document.

   Descriptions should contain all and only that information necessary
   to distinguish one subtag from others that it might be confused with.
   They are not intended to provide general background information,
   nor to provide all possible alternate names or designations.

[add more guidance on scope of descriptions]

-----
MD9
   The field 'Comments' conveys additional information about the record
   and MAY appear more than once per record.  The field-body MAY include
=>
   The field 'Comments' conveys additional information about the record
   as needed for clarity: it is not intended to provide general background
   information.

   It MAY appear more than once per record. The field-body MAY include

[add more guidance on scope of comments]

-----
MD10
   To ensure consistent backward compatibility, this document contains
   several provisions to account for potential instability in the
   standards used to define the subtags that make up language tags.
   These provisions mean that no language tag created under the rules in
   this document will become invalid.
=>
   To ensure consistent backward compatibility, this document contains
   several provisions to account for potential instability in the
   standards used to define the subtags that make up language tags.
   These provisions mean that no language tag created under the rules in
   this document will become invalid, nor will that language tag have a
   narrower scope in the future (it may have a broader scope).

[clarify broadening, relating to use elsewhere]

-----
MD11
   2.  Subtags of type 'Region' that have a Preferred-Value mapping in
       the IANA registry (see Section 3.1) SHOULD be replaced with their
       mapped value.  Note: In rare cases, the mapped value will also
       have a Preferred-Value.

   3.  Redundant or grandfathered tags that have a Preferred-Value
       mapping in the IANA registry (see Section 3.1) MUST be replaced
       with their mapped value.  These items either are deprecated
       mappings created before the adoption of this document (such as
       the mapping of "no-nyn" to "nn" or "i-klingon" to "tlh") or are
       the result of later registrations or additions to this document
       (for example, "zh-hakka" was deprecated in favor of the ISO 639-3
       code 'hak' when this document was adopted).

   4.  Other subtags that have a Preferred-Value mapping in the IANA
       registry (see Section 3.1) MUST be replaced with their mapped
       value.  These items consist entirely of clerical corrections to
       ISO 639-1 in which the deprecated subtags have been maintained
       for compatibility purposes.
=>
   2.  Subtags of type 'Region' that have a Preferred-Value mapping in
       the IANA registry (see Section 3.1) are recursively replaced with their
       mapped value.  Note: In rare cases, the mapped value will also
       have a Preferred-Value.

   3.  Redundant or grandfathered tags that have a Preferred-Value
       mapping in the IANA registry (see Section 3.1) are recursively replaced
       with their mapped value.  These items either are deprecated
       mappings created before the adoption of this document (such as
       the mapping of "no-nyn" to "nn" or "i-klingon" to "tlh") or are
       the result of later registrations or additions to this document
       (for example, "zh-hakka" was deprecated in favor of the ISO 639-3
       code 'hak' when this document was adopted).

   4.  Other subtags that have a Preferred-Value mapping in the IANA
       registry (see Section 3.1) are recursively replaced with their mapped
       value.  These items consist entirely of clerical corrections to
       ISO 639-1 in which the deprecated subtags have been maintained
       for compatibility purposes.
[We are defining 'canonical order'. MUST and SHOULD are inappropriate
in a definition -- and were inconsistently used, since #1 and #5 don't
use them. Also, we have to have recursive replacement of tags in order
to get an idempotent result!]

-----
MD12
   The 'Preferred-Value' field in records of type "grandfathered" and
   "redundant" contains whole language tags that are strongly
   RECOMMENDED for use in place of the record's value.  In many cases,
   the mappings were created by deprecation of the tags during the
   period before this document was adopted.  For example, the tag "no-
   nyn" was deprecated in favor of the ISO 639-1-defined language code
   'nn'.
[ADD]
   If the value of a Preferred-Value is not canonical (such as when it is
   a tag or contains a subtag that itself has a Preferred-Value), then
a Comment
   SHOULD be added to the record to that effect. For example:

   Preferred-Value: zh-hakka
   Comment: The canonical form of "zh-hakka" is "hak".

-----
[MD13]
   The relationship between the tag and the information it identifies is
   defined by the context in which the tag appears.  Accordingly, this
   section gives only possible examples of its usage.
[add at bottom of list]
   o  Language tags form the basis for most implementations of locale
       identifiers. For example, see the Unicode Locales project [CLDR].

-- 
Mark
_______________________________________________
Ltru mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/ltru

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 13 Feb 2008 13:07:59 -0800

Hi -

> From: "Mark Davis" <[email-masked]>
> To: "LTRU Working Group" <[email-masked]>
> Sent: Wednesday, February 13, 2008 11:01 AM
> Subject: [Ltru] Proposed edits (davis)
...
> [Reason: replace a hypothetical example with a real one.]
...

As a technical contributor, I object to the proposed change.  The
proposed replacement is not an example of language tagging,
but rather of a locale.

Randy

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 13 Feb 2008 13:10:38 -0800

Hi -

As co-chair, I'm splitting this into a separate thread, since it is,
in my judgement, a proposal for a technical change rather than
being merely editorial.

Randy

...

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 13 Feb 2008 13:13:25 -0800

Hi -

As a technical contributor...

...
> [add more guidance on scope of descriptions]

"should" -> "SHOULD" (since this really is a normative usage)
otherwise the proposal looks ok to me.

Randy

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 13 Feb 2008 13:19:09 -0800

Hi -

As co-chair, I'm splitting this item into a separate thread since it
calls for what in my judgement could be understood to be a technical
change.

Randy

...

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 13 Feb 2008 13:20:48 -0800

Hi -

As co-chair, I'm splitting this comment into a separate thread
since in my judgement it could be understood to be a technical
change.

Randy

...

---

From: Peter Constable <[email-masked]>
Date: Wed, 13 Feb 2008 13:21:29 -0800

I agree with Randy: we shouldn't use an example based on locales.

I do think it would be good to change that wording in 2.2.7, however, since at this point most/all entries in Ethnologue would be in 639-3. And, given the comprehensive nature of 639-3, a private-use example would have to involve something it doesn't cover.

(I'm at home, sick, and don't have the energy to think of better replacement text. I'll leave that for someone else.)

Peter

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Randy Presuhn
> Sent: Wednesday, February 13, 2008 1:08 PM
> To: LTRU Working Group
> Subject: [Ltru] Proposed edits (davis) MD4
>
> Hi -
>

---

From: "Mark Davis" <[email-masked]>
Date: Wed, 13 Feb 2008 13:26:03 -0800

+1

---

From: "Mark Davis" <[email-masked]>
Date: Wed, 13 Feb 2008 13:28:39 -0800

I do not believe it is a technical change. When you state a definition, you
don't use the term MUST. When you *use* that definition you might have MUST.

Moreover, as I said, the other 2 processing steps don't use MUST.

Mark

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 13 Feb 2008 13:39:36 -0800

Hi -

As co-chair - 

For example:

These two descriptions of step 2 have potentially different results -
in the first version, the replacement is optional, in the second
version it is unconditional.  This is a technical change.

As a technical contributor:

I agree with making this change.  It's probably what we meant
all along.  :-)

Randy

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Thu, 14 Feb 2008 16:06:16 +0100

> >    Comment: Updated on 2009-09-10 to add new codes from ISO 639-3.

To me, it is not an editorial fix but a new technical feature. It has
merits but, IMHO, WGLC is not the moment to add it.

[And, if we add new stuff in the format of the registry, I would
prefer ordinary comments, as all formats have:

# Updated on 2009-09-10 to add new codes from ISO 639-3

]

---

From: Addison Phillips <[email-masked]>
Date: Thu, 14 Feb 2008 07:27:46 -0800

> # Updated on 2009-09-10 to add new codes from ISO 639-3

All formats but the registry! This is one of those revived battles from 
the 4646 era. The WG didn't feel then that file-level comments were 
necessary and didn't allow for them (I don't agree with that decision).

The question now is whether adding them will break things. I personally 
prefer to use the record separator to introduce comments, on the theory 
that empty records will be ignored or at least harmless. But there is no 
telling what foolishness has been done in someone else's implementation.

OTOH, I tend to think that the number of record-jar or registry 
processors extant is going to be small and reasonably tolerant of this 
change.

I *would* thus propose once again that we include file-level comments, 
but this is WGLC and there was consensus before that this was 
unnecessary. So that suggests we be cautious. We've lived without 
comments until now. We may as well keep it that way.

Addison

-- 
Addison Phillips
Globalization Architect -- Yahoo! Inc.
Chair -- W3C Internationalization Core WG

Internationalization is an architecture.
It is not a feature.
_______________________________________________
Ltru mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/ltru

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 14 Feb 2008 07:33:03 -0800

Ok, I agree that this is not a crucial feature.

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 14 Feb 2008 07:37:07 -0800

I don't view this as a technical change; it doesn't change the registry
format; it doesn't change what qualifies as language tags; it just makes it
easier for implementers to see what the intended result is.

If we changed the SHOULD to should, would that make it easier?

Mark

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 14 Feb 2008 07:37:45 -0800

It is a valid example. The private use subtags (x-...) can be used to add
arbitrary information to a language tag. We do not limit what that
information is, and we couldn't in any event.

My only thought here was to use an actual, documented example of the use of
private use subtags, rather than an artificial example.

Mark

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 14 Feb 2008 07:39:43 -0800

Other than the items that Randy separated out with separate subjects, does
anyone object to the other items?

Should I split them also into separate subjects?

Mark

---

From: "Kent Karlsson" <[email-masked]>
Date: Thu, 14 Feb 2008 18:44:13 +0100

Since the label chosen was "Preferred-Value", rather than something more neutral,
like "Mapto-Value", I think that the value listed should not be a deprecated value
with another preferred value. I.e. I think the transitiveness should be shorted out
when maintaining the registry, rather than by users of the registry.
 
The current text also says:
      *  For fields of type 'language', 'Preferred-Value' contains the
         primary language subtag that is preferred when forming the
         language tag.

and similarly for the other types. This does in no way imply that there may
be further steps of preferredness, and trying to make it so may result in
inconsistent wording in various places in the text, and will surely (even if
we get all the wording consistent) introduce confusion: "Is not the preferred
value preferred?".
 
Surely it is better to map directly to the true preferred value (if there is any),
and allow, or rather require, that this is maintained both in this bulk update,
and during regular maintenance. I see that as much better than allowing
for some comment on that issue in the registry record.
 
        /kent k

  _____  

From: [email-masked] [mailto:[email-masked]] On Behalf Of Mark Davis
Sent: Thursday, February 14, 2008 4:37 PM
To: Randy Presuhn
Cc: LTRU Working Group
Subject: Re: [Ltru] Proposed edit MD12 (davis)

I don't view this as a technical change; it doesn't change the registry format; it doesn't change what qualifies as language tags; it just makes it easier for implementers to see what the intended result is.

Mark

On Wed, Feb 13, 2008 at 1:20 PM, Randy Presuhn <[email-masked]> wrote:

Hi -

Randy

...

-- 
Mark

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 14 Feb 2008 11:12:55 -0800

There is a prohibition on changing Preferred-Value:
3.4.  Stability of IANA Registry EntriesValues in the fields 'Type',
'Subtag', 'Tag', 'Added', 'Deprecated' and 'Preferred-Value' MUST NOT be
changed and are guaranteed to be stable over time.

I think the goal of that was to have the canonical form not vary over time,
but of course that can't be achieved. If "US" changes to "UX", then the
formerly canonical form "en-US" would change under canonicalization once
someone updated to the registry. So logically speaking (see below though), I
agree that it would be better to change the Preferred-Value to be the
recursive result. That would prevent mistakes in using the registry.

Moreover, I don't even think that we should have a prohibition on removing
the Preferred-Value or Deprecated fields. Suppose that on 2011-11-11,
Myanmar changes its name back to Burma, and ISO changes the country code
back to BU. In that case, the correct course of action would be for us to do
the following, which is not currently permitted by the spec.

FROM:

%%
Type: region
Subtag: BU
Description: Burma
Added: 2005-10-16
Preferred-Value: MM
Deprecated: 1989-12-05
%%
...

%%
Type: region
Subtag: MM
Description: Myanmar
Added: 2005-10-16
%%

TO:

%%
Type: region
Subtag: BU
Description: Burma
Added: 2005-10-16
%%
...

%%
Type: region
Subtag: MM
Description: Myanmar
Added: 2005-10-16
Preferred-Value: BU
Deprecated: 2011-11-11
%%

However, now doesn't seem to be the time to make such a change, so we may
have to resign ourselves to yet another version.

Mark

On Thu, Feb 14, 2008 at 9:44 AM, Kent Karlsson <[email-masked]>
wrote:

> like "Mapto-Value", I think that the value listed should *not* be a
> deprecated value
>  ------------------------------
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of
> *Mark Davis
> *Sent:* Thursday, February 14, 2008 4:37 PM
> *To:* Randy Presuhn
> *Cc:* LTRU Working Group
> *Subject:* Re: [Ltru] Proposed edit MD12 (davis)
>

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 14 Feb 2008 12:03:41 -0800

Hi -

As co-chair, it looks to me like we have a rough consensus
to *not* add the proposed file-level comments.

Randy

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 14 Feb 2008 12:11:23 -0800

Hi -

> To: "Randy Presuhn" <[email-masked]>
> Cc: "LTRU Working Group" <[email-masked]>
> Sent: Thursday, February 14, 2008 7:37 AM
...

As co-chair:

No.  It still changes the instructions for the maintenance of the registry,
and is consequently, from the perspective of this document as a BCP-to-be,
is a technical change.

As a technical contributor:
I think that the change is innocuous and cannot cause any new problems.

As a co-chair:

There's nothing wrong with accepting an innocuous technical change during WG
last call, particularly if no one can figure out any way in which it might
cause a problem.

Randy

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 14 Feb 2008 12:14:36 -0800

Hi -

> Subject: Re: [Ltru] Proposed edits (davis) MD4
>

> private use subtags, rather than an artificial example.

As a technical contributor, I agree with the intent.  However, the proposed
example is *not* a language tag, even though it happens to have a compatible
syntax.  There has been too much grief in the past due to confusion between
language tags and locales.  This is not a good time to blur the distinction.

Randy

---

From: "Randy Presuhn" <[email-masked]>
Date: Thu, 14 Feb 2008 12:22:07 -0800

Hi -

> Sent: Thursday, February 14, 2008 7:39 AM
> Subject: Re: [Ltru] Proposed edits (davis)
>

As a technical contributor, I had no problem with those other items.

In my judgement as a co-chair, I regarded them as being purely editorial.

> Should I split them also into separate subjects?

Only if someone has a problem with one of them.  Let's treat them
as being accepted.  If someone finds out that they cannot live with
one of those changes, let them open a new thread and we'll deal with
the issue then.

Randy

---

From: Addison Phillips <[email-masked]>
Date: Thu, 21 Feb 2008 19:59:12 -0800

A technical contributor:

I dimly recall that we allowed for a SHOULD for some actual purpose, 
although the following example of "en-BU" -> "en-MM" indicates that we 
intended MUST.

As editor:

I made this change. I also changed the example involving alphabetization 
of extensions, since it had the extension subtags in uppercase (which is 
not recommended).

Addison

---

From: Addison Phillips <[email-masked]>
Date: Thu, 21 Feb 2008 21:04:46 -0800

Editor hat on unless otherwise noted. Actions/comments follow, with 
Randy's extracted items omitted:

>    searching and matching operations.

DONE.

> [insert 'possible', matching description below.]

Love too. Can't. It makes it too wide.

As technical contributor: I think it's not strictly important.

>    For example, the subtag 'nedis' has a Prefix of "sl", making it

DONE. Although I think this is kind of redundant.

> [Reason: replace a hypothetical example with a real one.]

As technical contributor: Moreover, the SIL example is now probably 
harmful, seeing as the items are now all first-class primary language 
subtags!

As editor:

DONE. Edited as follows:

<t>For example: The Unicode Consortium defines a set of private use
extensions in LDML (<xref target="UTS35"></xref>, Locale Data Markup 
Language, a standard for defining locale data). One LDML-defined use of 
private use subtags might be the tag 
"en-US-x-ldml-POSIX-k-calendar-islamic-k-colStren-secondar", which, in 
addition to indicating that the language "en-US" is being used, 
indicates locale-related variations, such as that the (non-standard) POSIX
variant is being used; that formatting of dates might use the
Islamic calendar; and that case is being ignored in sorted lists. </t>

Note also the addition of an *informative* reference to UTS#35 (LDML)

>    to validate the tag.  A more recent copy of the registry might

DONE.

> MD6

===omitted as randy extracted it===

>          "sl-IT-nedis" are appropriate while the tag "is-nedis" is

DONE.

> MD8

===omitted as randy extracted it===

> [add more guidance on scope of comments]

As technical contributor: I'm not sure we want to limit it this way. 
ietf-languages might register all sorts of weird general background 
information :-).

> [clarify broadening, relating to use elsewhere]

DONE. Note corrected "will that" to "will a".
> MD12

===omitted as randy extracted them===
>        identifiers. For example, see the Unicode Locales project [CLDR].

DONE. I spelled out CLDR.

<t>Language tags form the basis for most implementations of locale 
identifiers. For example, see Unicode's CLDR (Common Locale Data 
Repository) project.</t>

---

From: Addison Phillips <[email-masked]>
Date: Fri, 22 Feb 2008 08:46:46 -0800

Mark and I were working on a few edits. Herewith a summary of a couple 
things that we should put on-list.

>     DONE. Edited as follows:
....
> 
> I'd suggest the following change to your text, to make it pithier also.
> 
> <t>For example: The Unicode Consortium defines a set of private 
> use extensions in LDML (<xref target="UTS35"></xref>, Locale Data 
> Markup Language, the Unicode standard for defining locale data) such as 
> in the tag "es-419-x-ldml-collation-traditio", which indicates Latin 
> American Spanish with traditional order for sorted lists.</t>

Sounds good. I'll make that edit. I was a bit worried by having such a 
long example tag.

> I think the issue was to have a clear statement that we don't want tons 
> of text in the registry -- for that, people should look at other 
> sources. Can you think of an alternative? 

In RFC 4646 (during one of the drafts) we removed the warning:

--
Long screeds about a particular subtag are frowned upon.
--

It was felt that the word "screed" was both too obscure and too 
judgmental. Personally, I thought it did the job. Perhaps:

--
Large amounts of information about the use, history, or general 
background of a subtag are frowned upon: these generally belong and are 
encouraged in registration requests, but do not belong in the registry 
record proper.
--

Best Regards,

Addison

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 22 Feb 2008 08:51:24 -0800

+1