Subject: [Ltru] extlang or not extlang

From: "Phillips, Addison" <[email-masked]>
Date: Fri, 9 May 2008 08:44:55 -0700

Co-chairs,

It seems we have arrived back at an impasse. I still think not having extlang is the correct course.

I would not be opposed to restoring extlang, if that is the consensus of the group and provided it can be done expeditiously.

Can you please institute some process for resolving this issue--either closing off the reappearance of extlang or deciding to restore it and focusing on completing the draft in the next few weeks under that banner.

Addison

Addison Phillips

Internationalization is not a feature.
It is an architecture.

_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 9 May 2008 15:55:03 -0700

I'm completely opposed.

We spent a long time reviewing all of the issues before coming to a way to
move forward on the extlang issue. There has been no new information that
was not available at the time we made the decision. It was a very long and
involved process. While there are advantages and disadvantages to both
sides, I see no reason to believe that we would come to a different
conclusion, and it would undoubtedly take a similar amount of time before we
come to resolution. I have refrained from responding to those suggesting
that extlang was the solution just so as to not complicate the issue,
because I really don't think we want to go through that process again.

The way that the current text reads, it is absolutely trivial for those who
want to support particular macrolanguage fallbacks to do so, and it does not
end up with the fallback in the many, many cases where it leads to worse
results.

Reopening the issue will delay us for no good reason. We have spent long
enough on 4646bis, it is time to get this revision out.

Mark

On Fri, May 9, 2008 at 8:44 AM, Phillips, Addison <[email-masked]>
wrote:

> I would not be opposed to restoring extlang, if that is the consensus of
> the group and provided it can be done expeditiously.
>
> Can you please institute some process for resolving this issue--either
> closing off the reappearance of extlang or deciding to restore it and
> focusing on completing the draft in the next few weeks under that banner.
>
> Addison
>
> Addison Phillips
>

-- 
Mark

---

From: "Phillips, Addison" <[email-masked]>
Date: Fri, 9 May 2008 16:00:00 -0700

+1

I have to admit that I find it extremely curious that the arguments seem to like the idea of using a mapping table when it comes to ISO 639-6 but oppose using it in BCP 47 for the same.

I don’t *WANT* to restore extlang. What I’m in favor of is *DECIDING* on a course of action.

Addison

Addison Phillips
Globalization Architect -- Lab126

From: [email-masked] [mailto:[email-masked]] On Behalf Of Mark Davis
Sent: Friday, May 09, 2008 3:55 PM
To: Phillips, Addison
Cc: Randy Presuhn; Martin Duerst; LTRU Working Group
Subject: Re: [Ltru] extlang or not extlang

I'm completely opposed.

We spent a long time reviewing all of the issues before coming to a way to move forward on the extlang issue. There has been no new information that was not available at the time we made the decision. It was a very long and involved process. While there are advantages and disadvantages to both sides, I see no reason to believe that we would come to a different conclusion, and it would undoubtedly take a similar amount of time before we come to resolution. I have refrained from responding to those suggesting that extlang was the solution just so as to not complicate the issue, because I really don't think we want to go through that process again.

The way that the current text reads, it is absolutely trivial for those who want to support particular macrolanguage fallbacks to do so, and it does not end up with the fallback in the many, many cases where it leads to worse results.

Reopening the issue will delay us for no good reason. We have spent long enough on 4646bis, it is time to get this revision out.

Mark
On Fri, May 9, 2008 at 8:44 AM, Phillips, Addison <[email-masked]<mailto:[email-masked]>> wrote:
Co-chairs,

Addison

Addison Phillips

_______________________________________________
Ltru mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/ltru

---

From: [email-masked]
Date: Fri, 9 May 2008 17:05:54 -0700

The extlang decision was mostly made on  ill-announced conference calls, 
not the list.  While you may feel you discussed this to death, some of us 
feel bullied.  There must be some new information as we've changed the 
draft as a result of recent discussion and our recommendation for the 
world's most popular language. Your recent statement that it would be 
better if "cmn" was deprecated was totally shocking to me. The suggestion 
that "in practice" zh should be treated as Mandarin is an affront to 
non-Mandarin Chinese languages and will create problems.  I believe there 
are at least four or five longtime followers of this work who are not 
totally comfortable with this decision.

I'm starting to think it would be better for my needs if RFC 4646bis was 
never released. While this may not be true for everyone, RFC 4646bis now 
does not add anything that I personally need and it changes the semantics 
of tags I already use. I do not want to see the "zh" tag narrowed "in 
practice" as that would mean I can't trust any "zh" tag to mean what I 
think it means. I'm not even 100% sure where we stand on Chinese languages 
now and this affects how I feel about extlang. I am afraid I will regret 
putting the words "RFC 4646bis or its successor" into numerous A/V 
standards now that I see ISO 639-6 nearing completion and the unexpected 
turn in the direction of this work.

This update could be a death knell for xml:lang as I don't think 
developers will be able to follow these guidelines consistently.

Regards,

Karen Broome
Sony Pictures Entertainment

[email-masked] wrote on 05/09/2008 03:55:03 PM:

> I'm completely opposed. 
> 
> We spent a long time reviewing all of the issues before coming to a 
> way to move forward on the extlang issue. There has been no new 
> information that was not available at the time we made the decision.
> It was a very long and involved process. While there are advantages 
> and disadvantages to both sides, I see no reason to believe that we 
> would come to a different conclusion, and it would undoubtedly take 
> a similar amount of time before we come to resolution. I have 
> refrained from responding to those suggesting that extlang was the 
> solution just so as to not complicate the issue, because I really 
> don't think we want to go through that process again. 
> 
> The way that the current text reads, it is absolutely trivial for 
> those who want to support particular macrolanguage fallbacks to do 
> so, and it does not end up with the fallback in the many, many cases
> where it leads to worse results.
> 
> Reopening the issue will delay us for no good reason. We have spent 
> long enough on 4646bis, it is time to get this revision out.
> 
> Mark

wrote:
> I would not be opposed to restoring extlang, if that is the 
> consensus of the group and provided it can be done expeditiously.
> 
> Can you please institute some process for resolving this issue--
> either closing off the reappearance of extlang or deciding to 
> restore it and focusing on completing the draft in the next few 
> weeks under that banner.
> 
> Addison
> 
> Addison Phillips
> 
> Mark _______________________________________________
> Ltru mailing list
> [email-masked]

---

From: Shawn Steele <[email-masked]>
Date: Fri, 9 May 2008 22:42:54 -0700

+1 on reaching a resolution.

Unlike Mark, I think that some new ideas have surfaced since this was discussed, and I understand Karen's frustration with original discussion.  Like Mark, I'd like to get this thing done because I have deadlines and products that will depend on whatever's decided here.  I want to make sure its something everyone in the industry can feel OK about.

As most of you know, I personally supported extlang.  Very recently, Peter & I (& others here at Microsoft) convinced each other that extlang wasn't really necessary and I felt that the concepts of "lists" would perhaps address my concerns regarding matching.

Since then there has been discussion about the lack of "lists" in some standards/contexts and how that might make it difficult for applications to correlate zh & yue in a document's tags if necessary.  Now I find myself wondering if maybe extlang makes sense again for some cases, but, even worse, it seems to me that zh, cmn AND zh-cmn should be legal to properly tag some documents (Chinese of some form, known Mandarin, and Mandarin that's also acceptable for general Chinese use would be my interpretation).  So now I'm pretty neutral.  I'm reasonably convinced that extlang isn't necessary for my immediate concerns, however if I try to consider Karen's and other scenarios besides my own immediate problems, perhaps extlang would be better.

One thing I'm very certain of is that zh != cmn and the standards shouldn't make an unnecessary correlation that doesn't exist in practice.

I'm also disconcerted that Karen feels this effort may not address her needs because I don't want a disconnect between A/V standards that might be useful for on-demand content and other software on the system.  Everyone's needs certainly aren't the same, but information is getting more and more connected and these things show up in odd places.

- Shawn

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of [email-masked] [[email-masked]]
Sent: Friday, May 09, 2008 5:05 PM
To: Mark Davis
Cc: LTRU Working Group
Subject: Re: [Ltru] extlang or not extlang

Regards,

Karen Broome

---

From: Peter Constable <[email-masked]>
Date: Sat, 10 May 2008 07:08:46 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Shawn Steele

> +1 on reaching a resolution.

Ditto.

> Unlike Mark, I think that some new ideas have surfaced since this was
> discussed

I also think there have been some new ideas, though I'm not saying there's been anything compelling us to change past decisions. (What is for me significant new thinking is (a) that it may be useful in some scenarios to declare a language attribute for an info obj with two levels of specificity for cases like zh, and (b) that zh-xxx provides a way to do that using 4646 syntax, meaning that it would be supportable in any BCP47 app.)

> and I understand Karen's frustration with original
> discussion.  Like Mark, I'd like to get this thing done because I have
> deadlines and products that will depend on whatever's decided here.

+1

> One thing I'm very certain of is that zh != cmn and the standards
> shouldn't make an unnecessary correlation that doesn't exist in
> practice.

Very strong +1. I feel we should not be stipulating or even suggesting that zh is the same as cmn, or that cmn should be deprecated. Saying that, in general, zh content is most often Mandarin is fine.

> I
> want to make sure its something everyone in the industry can feel OK
> about...
> I'm also disconcerted that Karen feels this effort may not address her
> needs because I don't want a disconnect between A/V standards that
> might be useful for on-demand content and other software on the system.
> Everyone's needs certainly aren't the same, but information is getting
> more and more connected and these things show up in odd places.

Very strong +1

Peter

---

From: "Debbie Garside" <[email-masked]>
Date: Sat, 10 May 2008 09:33:26 +0100

+1

I have to say that there are some strong characters in this forum and if
people are feeling intimidated and bullied this will put other people off
participating.

Also, I think that for such an important issue to be decided during
conference calls means that there is no discussion history to fall back on
and this doesn't help those of us who missed the conference calls.

Those of us who missed the conference calls are feeling intimidated into not
discussing extlang as others within this forum are saying it is dealt with.
Even those who were pro extlang are being 'bullied' into not mentioning it.
This is unacceptable.

Like Karen, I feel that the discussion re 'zh' and Mandarin has got out of
hand - hence my comment on Doug's draft.

Best

Debbie

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of [email-masked]
> Sent: 10 May 2008 01:06
> To: Mark Davis
> Cc: LTRU Working Group
> The extlang decision was mostly made on  ill-announced
> conference calls, not the list.  While you may feel you
> discussed this to death, some of us feel bullied.  There must
> be some new information as we've changed the draft as a
> result of recent discussion and our recommendation for the
> world's most popular language. Your recent statement that it
> would be better if "cmn" was deprecated was totally shocking
> to me. The suggestion that "in practice" zh should be treated
> as Mandarin is an affront to non-Mandarin Chinese languages
> and will create problems.  I believe there are at least four
> or five longtime followers of this work who are not totally
> comfortable with this decision.
>
> I'm starting to think it would be better for my needs if RFC
> 4646bis was never released. While this may not be true for
> everyone, RFC 4646bis now does not add anything that I
> personally need and it changes the semantics of tags I
> already use. I do not want to see the "zh" tag narrowed "in
> practice" as that would mean I can't trust any "zh" tag to
> mean what I think it means. I'm not even 100% sure where we
> stand on Chinese languages now and this affects how I feel
> about extlang. I am afraid I will regret putting the words
> "RFC 4646bis or its successor" into numerous A/V standards
> now that I see ISO 639-6 nearing completion and the
> unexpected turn in the direction of this work.
>
> This update could be a death knell for xml:lang as I don't
> think developers will be able to follow these guidelines consistently.
>
> Regards,
>
> Karen Broome
> > would come to a different conclusion, and it would
> undoubtedly take a
> > similar amount of time before we come to resolution. I have
> refrained
> > from responding to those suggesting that extlang was the
> solution just
> > so as to not complicate the issue, because I really don't think we
> > want to go through that process again.
> >
> > those who want to support particular macrolanguage
> fallbacks to do so,
> > and it does not end up with the fallback in the many, many
> cases where
> > it leads to worse results.
> >
> > It seems we have arrived back at an impasse. I still think
> not having
> > extlang is the correct course.
> >
> consensus
> > of the group and provided it can be done expeditiously.
> >
> > either closing off the reappearance of extlang or deciding
> to restore
> > it and focusing on completing the draft in the next few weeks under
> > that banner.
> >
> > Addison
> >
> > Addison Phillips
> >

---

From: John Cowan <[email-masked]>
Date: Sat, 10 May 2008 14:01:17 -0400

> Even those who were pro extlang are being 'bullied' into not mentioning it.

Well, here's how it looked to me.  The list was stuck, and the conference
calls (which were publicly announced on the list) were an attempt to
move past that stuckness.  Most of those who spoke on those calls were
anti-extlang, with me as the most visible pro-extlang person.

What happened?  Pro-extlang person #1 (I'm not naming names, because
this is not a personal attack) thought the effort was futile, the
antis would win no matter what, so I got no public support there.
Pro-extlang person #2 was suddenly saying there was merit in both sides,
and so I got no public support there either.  Pro-extlang persons #3 and
#4 had other priorities at the time: no fault with that, people *do*
have other priorities.  Pro-extlang person #5 wasn't even on the list
at the time.  That left me, pro-extlang person #6, standing all alone.

And (as I posted here) I came to realize that if I didn't give in, we'd
never have an RFC 4646bis.  And that would be a tragedy.  Because what
really matters is not how we tag Cantonese, but whether we are even able
to tag Aromanian.  What really matters is not the details of the matching
algorithm, but whether the minority languages, the non-official languages,
the non-commercial languages, can be referred to on computers on equal
terms with the rest.

Did I have to give up something I really wanted?  You bet.  But I
publicly surrendered my position, because what matters most to me is
that we get off the pot and get this draft published as an official RFC,
so that things like the XML Recommendation can be updated (actually,
will update themselves) to accept all the tags for the long tail of
languages we are still speaking here on planet Earth.

-- 
John  Cowan  http://ccil.org/~cowan   [email-masked]
'My young friend, if you do not now, immediately and instantly, pull
as hard as ever you can, it is my opinion that your acquaintance in the
large-pattern leather ulster' (and by this he meant the Crocodile) 'will
jerk you into yonder limpid stream before you can say Jack Robinson.'
        --the Bi-Coloured-Python-Rock-Snake
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Debbie Garside" <[email-masked]>
Date: Sat, 10 May 2008 19:39:59 +0100

Hi John

I can see how this situation has arisen and if it was not causing too many
problems I would probably let it slide as like you I would like to see this
RFC published.

However, the more I hear about macrolanguages and extlang the more I feel it
is important to re-visit this.  I also don't like the change in what was
purported to be a "stable" syntax.

>From what I have seen so far it pretty much seems like a dogs breakfast and,
if published as is, I don't think anyone would know how to tag
macrolanguages or their encompassed entities - we would get a mish mash of
tagging and that is not what standardization is about.  I think after all
these months it is worth delaying for a few more weeks to get it as right
and useful as we possibly can for as many end users as we can.

With a minimum of 6 people saying they would at least like to revisit the
extlang issue, how many within the teleconferences were against the
inclusion of extlang?

Do you consider there are enough people making "noise" about extlang to
revisit this?

BTW there seems to be an 8 hour delivery delay for some of my messages and
others are not being delivered at all - checked the archives and no sign of
them.  Not sure what is happening but it is quite probable that I am missing
half the conversations on the LTRU.  If I don't respond to a message I
probably haven't seen it.

Best

Debbie

> From: John Cowan [mailto:[email-masked]]
> Sent: 10 May 2008 19:01
> To: Debbie Garside
> Cc: [email-masked]; 'Mark Davis'; 'LTRU Working Group'
> > I have to say that there are some strong characters in this
> forum and
> > if people are feeling intimidated and bullied this will put other
> > people off participating.
> >
> > conference calls means that there is no discussion history to fall
> > back on and this doesn't help those of us who missed the
> conference calls.
> >
> > Those of us who missed the conference calls are feeling intimidated
> > into not discussing extlang as others within this forum are
> saying it is dealt with.
> Well, here's how it looked to me.  The list was stuck, and
> the conference calls (which were publicly announced on the
> list) were an attempt to move past that stuckness.  Most of
> those who spoke on those calls were anti-extlang, with me as
> the most visible pro-extlang person.
>
> What happened?  Pro-extlang person #1 (I'm not naming names,
> because this is not a personal attack) thought the effort was
> futile, the antis would win no matter what, so I got no
> public support there.
> Pro-extlang person #2 was suddenly saying there was merit in
> both sides, and so I got no public support there either.
> Pro-extlang persons #3 and
> #4 had other priorities at the time: no fault with that,
> people *do* have other priorities.  Pro-extlang person #5
> wasn't even on the list at the time.  That left me,
> pro-extlang person #6, standing all alone.
>
> And (as I posted here) I came to realize that if I didn't
> give in, we'd never have an RFC 4646bis.  And that would be a
> tragedy.  Because what really matters is not how we tag
> Cantonese, but whether we are even able to tag Aromanian.
> What really matters is not the details of the matching
> algorithm, but whether the minority languages, the
> non-official languages, the non-commercial languages, can be
> referred to on computers on equal terms with the rest.
>
>
> Did I have to give up something I really wanted?  You bet.
> But I publicly surrendered my position, because what matters
> most to me is that we get off the pot and get this draft
> published as an official RFC, so that things like the XML
> Recommendation can be updated (actually, will update
> themselves) to accept all the tags for the long tail of
> 'My young friend, if you do not now, immediately and
> instantly, pull as hard as ever you can, it is my opinion
> that your acquaintance in the large-pattern leather ulster'
> (and by this he meant the Crocodile) 'will jerk you into
> yonder limpid stream before you can say Jack Robinson.'

---

From: Felix Sasaki <[email-masked]>
Date: Sun, 11 May 2008 10:43:51 +0900

> I'm also disconcerted that Karen feels this effort may not address her needs because I don't want a disconnect between A/V standards that might be useful for on-demand content and other software on…

I too hope very much that this discussion will lead to a result which 
will be satisfactory for Karen's use case - also to be able to ensure 
the applicability of xml:lang in her community.

Felix

---

From: John Cowan <[email-masked]>
Date: Sun, 11 May 2008 20:46:41 -0400

> purported to be a "stable" syntax.

I had hoped -- many had hoped -- it would be stable, but there were never any
guarantees until RFC 4646bis was final.  (Note that the syntax *is* stable
in another sense: it is the validity of particular tags that has changed.)

> and useful as we possibly can for as many end users as we can.

Part of the trouble has nothing to do with syntax: it's the issue of "zh" vs.
"cmn", which is the same as "zh" vs. "zh-cmn".  That's being struggled with
now, and I think, usefully so.

So if we reopen extlangs, we are reopening a syntax because we think it's
useful to do so.

> extlang issue

Not yet six.

> how many within the teleconferences were against the
> inclusion of extlang?

It's not about how many, it's about how firmly.

> Do you consider there are enough people making "noise" about extlang to

That's Randy's call, ultimately.  Is there rough consensus?

-- 
"But I am the real Strider, fortunately,"       John Cowan
he said, looking down at them with his face     [email-masked]
softened by a sudden smile.  "I am Aragorn son  http://www.ccil.org/~cowan
of Arathorn, and if by life or death I can
save you, I will."  --LotR Book I Chapter 10
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: Martin Duerst <[email-masked]>
Date: Sun, 11 May 2008 17:31:36 +0900

Just to make sure people don't worry too early:
I have seen this message, and I'm coordinating
with Randy to make sure we take the best way
forward (procedurally) for the WG.

Regards,    Martin.

>Internationalization is not a feature.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: Martin Duerst <[email-masked]>
Date: Mon, 12 May 2008 18:39:09 +0900

>It's not about how many, it's about how firmly.

The IETF is not about how firmly, but about sound technical argumentation.

Regards,    Martin.

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 13 May 2008 15:22:43 +0200

On Sat, May 10, 2008 at 07:39:59PM +0100,
 Debbie Garside <[email-masked]> wrote 
 a message of 118 lines which said:

> I don't think anyone would know how to tag macrolanguages or their
> encompassed entities - we would get a mish mash of tagging

Even with RFC 4646, people make mistakes. Many (most?) tags visible on
the public Web (where no prior checking of language tags occur) are
broken. May be language tagging is already too complicated.

But it is naïve to expect that we can find a solution which will avoid
user mistakes. Now that we've dropped extlangs, they appear as the
solution to everything, but that's simply the "grass is greener"
effect. Extlangs would be as complicated for users.

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 13 May 2008 14:52:32 +0200

On Sat, May 10, 2008 at 09:33:26AM +0100,
 Debbie Garside <[email-masked]> wrote 
 a message of 154 lines which said:

> I have to say that there are some strong characters in this forum
> and if people are feeling intimidated and bullied this will put
> other people off participating.

"Bullied" may be a word too strong. And, by definition, people who
complain, like Karen and you, are not silenced by the bullies :-)

The problem, for a group which is supposed to work on language tags
for the whole world, is that participation seems to be almost only
from people in Europe/USA. But that's a complicated topic, for which I
offer no solution today (RFC 3774, 2.6.6).

> back on and this doesn't help those of us who missed the conference
> calls.

I was not able to participate to any teleconference (for various
reasons, one of them being that I'm quite bad with spoken english, see
again RFC 3774, 2.6.6). But the whole discussion was also on the
mailing list and I feel I have a complete picture from the list.

True, the mailing list is difficult to follow and there is no
synthesis or summary but this is a general IETF problem (apparently
not discussed in RFC 3774), not specific to LTRU. Asking on the
mailing list for a summary generally provides good results, as you
have seen last week.

---

From: Leif Halvard Silli <[email-masked]>
Date: Thu, 15 May 2008 03:26:01 +0200

> offer no solution today (RFC 3774, 2.6.6).

But there are many issues that can be solved by people from our spheres 
- we cover many languages and many language situations that are common 
throughout the world, I believe. We also have a big influx of speakers 
from "the rest of the World". Europe/USA does not mean "white and English".

The real issue problem is not the one you describe, therefore, but 
rather that it seems as if dominant language users want to solve the 
problems of other dominant language users, as well.

Are there anyone else except me with an "encomassed language, which is 
not the dominant language either" as mothertongue on this list?

Since we have at least one person from Finland on this list: I recently 
heard a speech about the situation for the biggest minority language in 
Finland, Swedish. The speaker said that the Finnnish government 
described themselves as world masters in equality. Except that was far 
from how the Swedish speakers expeirenced it.

The problem is: The more you know the situation, the more you feel that 
you can ignore the arguements, because then you see that it is politics. 
And when it comes to politics, you yourself is a partaker.

In this list, I sense a fear from actually deciding *anything* that 
could make majority language users suffers the slightest. This can be 
examplified by Addision's draft text where he says that Macrolanguge tag 
*should* be used for the dominant language.

There seems to be a fundamental lack of will to be spesiific, except 
when it comes to the dominant language. Dominance wins over principles.

With a such aproach, thre is no need to get input from other majority 
language users from other spheres, because it will just end up with the 
same proposals.
-- 
leif halvard silli
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru