Subject: Re: [Ltru] a modest proposal...

From: Martin Duerst <[email-masked]>
Date: Mon, 02 Jun 2008 17:50:24 +0900

[technical hat on]

At 04:38 08/05/30, Phillips, Addison wrote:

>I would like to submit a "modest proposal" for addressing the impasse:
>
>1. Restore a *single* extlang to the ABNF.

This very, very much looks like a compromize. Of course, if we have
at least one language with extlang, we need at least one subtag in
the ABNF, so this is more a consequence of the proposals below than
a proposal in itself.

I personally understand the position of the XML Schema WG, and
I also think that we should not change the wellformedness definition
in RFC 4646, independent of how many (or how few, even zero) extlangs
we use. We have in this WG (though unofficially) produced test data
that includes three extlangs to test implementations, and changing
this doesn't seem to be a good idea to me (I can very quickly change
my implementation, but I have no idea how many or how few downloaded/
deployed there already are).

>2. Keep the Macrolanguage field in the registry for all encompassed 
>languages, which information may be use by implementations however makes 
>the most sense.

Fine with me.

>3. Cherry pick *only* the 'zh' (and possibly the 'ar') encompassed 
>languages for registration as extlangs. This is done in the name of 
>compatibility alone.

I'm personally okay with any kind of selection that would be able to make
the "rough consensus" that we currently have a bit less rough. As I know
there are some people who, for good reasons, don't like cherry-picking,
such a selection, so having a good justification would help.

I have stated my preference for also including 'ar' in a separate thread.
Personally, I'd prefer including all those languages that have a
'dominant' (or whatever the politically/linguistically correct term)
encompassed language.

>4. Permit implementations to treat the 'language' 
>production as atomic (that is, the sequence "zh-yue" MAY be treated as if 
>it were a single subtag but MAY be treated as separate subtags, notably by 
>existing implementations). Note that the 'language' production is the one 
>that includes both the primary and extended language subtags.

That definitely seems to be a very good idea. As far as I understand,
we already allow virtually any kind of modification for matching
provided "you know what you do", but the above definitely adds
a lot of clarity.

Regards,     Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]     

_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: Martin Duerst <[email-masked]>
Date: Mon, 02 Jun 2008 17:31:53 +0900

[co-chair hat on]

I don't think we have enough feedback on this yet to be able
to declare this to be close to consensus. In particular, I
have seen several people that I'd count in the extlang camp
to agree on this, but I haven't seen anybody that I'd count
as being firmly against extlangs commenting. Ultimately,
for a real compromize, we would need support from a significant
number of people from both camps.

On the other hand, I don't remember seeing anybody actively
opposing to this proposal, which is definitely a good sign.
If you have support/comments, or a somewhat different proposal
for a compromize, don't hesitate to send it to the list.

I probably have expressed this before, but I'm also quite a bit
worried that we don't have anybody on this list from one of the
directly affected communities (Chinese, Arabic, Sign Languages,...).

Regards,    Martin.

At 04:38 08/05/30, Phillips, Addison wrote:
>Since:
>
>1. We appear to be stuck.
>2. We (most of us, anyway) would like to see this work completed.
>3. We haven't a compelling-enough technical argument that resolves the issue.
>
>compatibility alone. 4. Permit implementations to treat the 'language' 
>Thoughts?
>Addison
>Addison Phillips
>Globalization Architect -- Lab126
>
>Internationalization is not a feature.
>It is an architecture.
>
>

---

From: "Phillips, Addison" <[email-masked]>
Date: Mon, 2 Jun 2008 07:18:42 -0700

Martin wrote:

> we use.

We don't have to change it, but it seems slightly silly to keep the extra extlangs around if we foreclose their use (writing code/tests for never-to-be-implemented features is considered bad practice).

> We have in this WG (though unofficially) produced test data

It would be okay with me to keep well-formedness the same. Having the three-extlang well-formedness check doesn't hurt very many people. However: a single extlang is much easier to code other operations for, such as validation or tag generation. The first repeating option comes at variant, at the end of the tag. Having written more than one implementation, I found multiple extlangs one of the more difficult things to implement well each time---and all for naught?

I guess the more compelling issue for me is that well-formedness checking ALREADY changes in 4646bis because we fixed the grandfathered production (which was much more permissive). In practice the bigger problem are 3066 well-formedness implementations, which we know are widespread.

> such a selection, so having a good justification would help.

See Peter's email on this topic and see my justification just above: all this talk about macrolanguages has obscured the reason we're doing it, which is compatibility with existing tagging practice. Thus the Chinese, Sign Language, and possibly Arabic cherries make sense. Stuff like Malay and Konkani don't make the grade.

Addison

---

From: "Mark Davis" <[email-masked]>
Date: Mon, 2 Jun 2008 11:42:52 -0700

We can live with having extlang if limited to only zh and possibly ar or
sgn, and as long as we include Shawn's proviso, that the non-extlang
versions (eg "yue-Hant") be also valid. That is, one of the two equivalent
forms can be deprecated with a preferred value of the other.
Mark

On Mon, Jun 2, 2008 at 7:18 AM, Phillips, Addison <[email-masked]>
wrote:

> We don't have to change it, but it seems slightly silly to keep the extra
> extlangs around if we foreclose their use (writing code/tests for
> never-to-be-implemented features is considered bad practice).
>
> It would be okay with me to keep well-formedness the same. Having the
> three-extlang well-formedness check doesn't hurt very many people. However:
> a single extlang is much easier to code other operations for, such as
> validation or tag generation. The first repeating option comes at variant,
> at the end of the tag. Having written more than one implementation, I found
> multiple extlangs one of the more difficult things to implement well each
> time---and all for naught?
>
> I guess the more compelling issue for me is that well-formedness checking
> ALREADY changes in 4646bis because we fixed the grandfathered production
> (which was much more permissive). In practice the bigger problem are 3066
> well-formedness implementations, which we know are widespread.
>
>
> >
> See Peter's email on this topic and see my justification just above: all
> this talk about macrolanguages has obscured the reason we're doing it, which
> is compatibility with existing tagging practice. Thus the Chinese, Sign
> Language, and possibly Arabic cherries make sense. Stuff like Malay and
> Konkani don't make the grade.
>
>
> Addison

-- 
Mark

---

From: "Phillips, Addison" <[email-masked]>
Date: Mon, 2 Jun 2008 11:51:05 -0700

You mean:

Type: language
Subtag: yue
Description: Cantonese Chinese
...

Type: extlang
Subtag: yue
Prefix: zh
Description: Cantonese Chinese
Preferred-Value: yue
Deprecated: 2008-06-02

This gives me a great deal of heartburn, but I guess I could live with it. I really don't like or see much point in creating a class of dead-at-birth subtags.

Interestingly, though, in practice it means that if the tag matches the extlang production (but not the grandfathered production), you trim the first subtag and then process from there.

Addison

Addison Phillips

From: [email-masked] [mailto:[email-masked]] On Behalf Of Mark Davis
Sent: Monday, June 02, 2008 11:43 AM
To: Phillips, Addison
Cc: Martin Duerst; LTRU Working Group
Subject: Re: [Ltru] a modest proposal...

We can live with having extlang if limited to only zh and possibly ar or sgn, and as long as we include Shawn's proviso, that the non-extlang versions (eg "yue-Hant") be also valid. That is, one of the two equivalent forms can be deprecated with a preferred value of the other.

Mark
On Mon, Jun 2, 2008 at 7:18 AM, Phillips, Addison <[email-masked]> wrote:

Addison

---

From: Shawn Steele <[email-masked]>
Date: Mon, 2 Jun 2008 12:05:57 -0700

My original thought was to not have the deprecated field.  I haven't thought that all the way through....

Type: language
Subtag: yue

Type: extlang
Subtag: yue
Prefix: zh

- Shawn

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Phillips, Addison
Sent: Monday, June 02, 2008 11:51 AM
To: Mark Davis
Cc: LTRU Working Group
Subject: Re: [Ltru] a modest proposal...

You mean:

Type: language
Subtag: yue

Type: extlang
Subtag: yue
Prefix: zh

Addison

Addison Phillips

Mark

Addison

---

From: "Phillips, Addison" <[email-masked]>
Date: Mon, 2 Jun 2008 12:19:40 -0700

Under no circumstances would I support having both forms be canonical.
>
> My original thought was to not have the deprecated field.  I
> haven't thought that all the way through....

Preferred-Value requires Deprecated. Deprecated doesn't make it less valid.

Addison

---

From: "Mark Davis" <[email-masked]>
Date: Mon, 2 Jun 2008 12:41:39 -0700

I agree with Addison.

---

From: John Cowan <[email-masked]>
Date: Mon, 2 Jun 2008 16:28:18 -0400

> forms can be deprecated with a preferred value of the other.

Same objection as for 639-5.  Also, which is to be pre-deprecated,
and why?

-- 
John Cowan    http://ccil.org/~cowan  [email-masked]
The Penguin shall hunt and devour all that is crufty, gnarly and
bogacious; all code which wriggles like spaghetti, or is infested with
blighting creatures, or is bound by grave and perilous Licences shall it
capture.  And in capturing shall it replicate, and in replicating shall
it document, and in documentation shall it bring freedom, serenity and
most cool froodiness to the earth and all who code therein.  --Gospel of Tux
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Mark Davis" <[email-masked]>
Date: Mon, 2 Jun 2008 14:02:39 -0700

Add+deprecate allows people to have the forms they really care about, and
yet preserves a formal relationship between them, so that people can convert
between them.
We're trying to find a compromise here, so that we can work our way out of
this impasse!
As to which is deprecated, the simpler way would be to deprecate the extlang
form, as Addison said. If it were necessary to achieve a compromise, we
could live with deprecating the conventional form.

Mark

---

From: John Cowan <[email-masked]>
Date: Mon, 2 Jun 2008 19:50:45 -0400

> yet preserves a formal relationship between them, so that people can convert

That's a good point.

> We're trying to find a compromise here, so that we can work our way out of

Agreed.

> could live with deprecating the conventional form.

I can live with pre-deprecation, provided it is the conventional form
that is pre-deprecated.

-- 
A rose by any other name                            John Cowan
may smell as sweet,                                 http://www.ccil.org/~cowan
but if you called it an onion                       [email-masked]
you'd get cooks very confused.          --RMS
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: Martin Duerst <[email-masked]>
Date: Tue, 03 Jun 2008 10:20:40 +0900

>[co-chair hat on]

Still on.

>for a compromize, don't hesitate to send it to the list.

Actually, this was inaccurate, Karen expressed her concerns.
Karen, could you be more explicit about where this proposal
makes your guidance task worse? Is the main problem with
the fact that there would be two allowed ways to tag e.g.
Cantonese, one of them deprecated? Or is the main problem
that there would be only extlangs for Chinese and Arabic,
but not for some other languages (e.g. Malay)? Or anything
else?

Regards,    Martin.

---

From: "Phillips, Addison" <[email-masked]>
Date: Mon, 2 Jun 2008 22:13:06 -0700

Actually, the proposal was NOT to make two ways to tag Cantonese. That was Mark's addendum today.

Addison

Addison Phillips

> From: Martin Duerst [mailto:[email-masked]]
> Sent: Monday, June 02, 2008 6:21 PM
> To: Phillips, Addison; LTRU Working Group