Subject: [Ltru] Language Tag Special Cases

From: "Mark Davis" <[email-masked]>
Date: Wed, 18 Apr 2007 13:48:14 -0700

I revised the examples I circulated earlier in light of the discussion, and
posted at

http://docs.google.com/Doc?id=dfqr8rd5_11g425c9

(everyone should be able to get to that document -- let me know if you have
trouble.)

-- 
Mark
_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: [email-masked]
Date: Wed, 18 Apr 2007 15:16:35 -0700

The last use case seems problematic. If "zxx" is non-linguistic, it's not 
a language so there is only one language present.  Also, the text in that 
mystery image may very well be in German -- you can't tell by looking at 
the binary.  The binary and <image> tags are not intended to be read by a 
human and would not appear in a browser, but text that occurs in <code> 
tags in XHTML is. That's a distinction I think a parser would need to 
make.

Does anyone have a real use case for zxx applied to code where a <code> 
tag wouldn't do? I don't think the text cited would realistically occur 
anywhere except in HTML and the presentation layer of HTML is what needs 
tagging, not the source code. 

It seems to me the most common use case for "zxx" is something like the 
Library of Congress where musical works may or may not be associated with 
a language tag. If instrumental, then "zxx." Or the silent film scenario 
that resulted in the tag. I think these are better use cases and less 
likely to confuse users familiar with the code tag and HTML/XHTML 
standards.

Regards,

Karen Broome

"Mark Davis" <[email-masked]> wrote on 04/18/2007 01:48:14 PM:

> I revised the examples I circulated earlier in light of the 
> discussion, and posted at 
> 
> (everyone should be able to get to that document -- let me know if 
> you have trouble.)
> 
> -- 
> Mark _______________________________________________
> Ltru mailing list
> [email-masked]

---

From: "Mark Davis" <[email-masked]>
Date: Wed, 18 Apr 2007 16:23:31 -0700

comments below.

On 4/18/07, [email-masked] <[email-masked]> wrote:
> a language so there is only one language present.

That's a possible interpretation. If we have a protocol that allows multiple
codes, I would tag it as  <en, zxx>, indicating that it has both kinds of
content. If we don't, then I don't know what we should do.

BTW, I numbered the examples to make them easier to reference.

Also, the text in that mystery image may very well be in German -- you can't
> tell by looking at the binary.  The binary and <image> tags are not intended
> to be read by a human and would not appear in a browser, but text that
> occurs in <code> tags in XHTML is. That's a distinction I think a parser
> would need to make.

The intent for the binary in the example is to not have content.

Does anyone have a real use case for zxx applied to code where a <code> tag
> wouldn't do?

Yes. I said before, *not every environment is HTML/XML". Language tags are
not just limited to HTML or XML.

I don't think the text cited would realistically occur anywhere except in
> HTML and the presentation layer of HTML is what needs tagging, not the
> source code.
>
> Library of Congress where musical works may or may not be associated with a
> language tag. If instrumental, then "zxx." Or the silent film scenario that
> resulted in the tag. I think these are better use cases and less likely to
> confuse users familiar with the code tag and HTML/XHTML standards.

That's a reasonable example, I'll add it.

Regards,

---

From: [email-masked]
Date: Wed, 18 Apr 2007 17:20:30 -0700

Mark,

Yes, I understand that there are worlds other than XHTML and HTML, but I 
can't come up with a use case (for identifying programming code with zxx) 
outside of those instances.   And I can see a danger in suggesting these 
codes for use in XHTML and HTML code markup -- not an insignificant 
consideration. 

The "code snippet" scenario seems most frequent in technical tutorials and 
books where code is intended to be read. In an instance of running code, 
you might identify the language of a "language snippet" but not the code 
around it. Isn't most code self-identifying? 

Honestly, I just don't see the real-life scenario here. Please enlighten 
me. I'm not saying there isn't a use case. I'm just saying I can't think 
of one and don't think your example is realistic or useful. If you updated 
the example to include other scenarios, that helps.  I'd just like to get 
away from thinking of this tag as being synonymous with code or binary 
junk. There are other more likely uses for the esteemed zxx tag.

Regards,

Karen Broome

---

From: Peter Constable <[email-masked]>
Date: Wed, 18 Apr 2007 20:54:55 -0700

In the context of BCP 47, it's about as appropriate to tag "What?" as mis as it is to tag it fr. In *some* application of ISO 639, mis might be appropriate, but I would not consider BCP 47 to be such a case.

Peter

From: Mark Davis [mailto:[email-masked]]
Sent: Wednesday, April 18, 2007 1:48 PM
To: LTRU Working Group
Subject: [Ltru] Language Tag Special Cases

I revised the examples I circulated earlier in light of the discussion, and posted at

(everyone should be able to get to that document -- let me know if you have trouble.)

---

From: "Mark Davis" <[email-masked]>
Date: Wed, 18 Apr 2007 21:59:10 -0700

I disagree. As far as the ISO 639-2 standard is concerned, the only thing
defining "mis" is that it is the phrase "Miscellaneous languages", and the
phrasing clarifying that "languages" means it is a collection. Nothing else.
Nada. Zilch.

It is, of course, woefully underspecified, but there is no particular reason
in the ISO 639-2 standard to think that it applies to English any more or
less than that it applies to Ainu or that it applies to Kamchadal, or any of
the others you listed. It may in fact be true that "mis" was intended to
mean something else before it got into the ISO standard, but that doesn't
matter: it doesn't change the fact that as defined in the ISO 639-2 standard
which is the basis for the BCP 47 codes, it applies equally well to any of
the three above.

One could argue that "mis" is so badly specified that we should just
discourage its use. I mention that possibility in my document, and at this
point, that may just be the best approach.

Now separately, I think it would be useful to have a special code, special
in the same sense as zxx and und, that just meant the opposite of zxx: that
is "some (unspecified) linguistic content". And it had seemed to me that
"mis" fit the bill, or with slight broadening could fit that bill, and is
otherwise pretty useless anyway. But it appears so contentious to push for
this interpretation of "mis" that despite the utility of taking this
approach, it is probably not worth it.

I think what is more important is to follow up on your suggestion, and
remove the exclusionary nature of the "(Other)" collections, preferably in
ISO 639-2 (but failing that, in 4646bis) so that we have a much more useful
and consistent approach to collections overall.

Mark

On 4/18/07, Peter Constable <[email-masked]> wrote:
>
>  In the context of BCP 47, it's about as appropriate to tag "What?" as mis
> as it is to tag it fr. In **some** application of ISO 639, mis might be
> appropriate, but I would not consider BCP 47 to be such a case.
>
>
>
> Peter
>
>
>
> *From:* Mark Davis [mailto:[email-masked]]
> *Sent:* Wednesday, April 18, 2007 1:48 PM
> *To:* LTRU Working Group
> *Subject:* [Ltru] Language Tag Special Cases
>
>
>

---

From: John Cowan <[email-masked]>
Date: Thu, 19 Apr 2007 01:53:31 -0400

Mark Davis scripsit:

> I disagree. As far as the ISO 639-2 standard is concerned, the
> only thing defining "mis" is that it is the phrase "Miscellaneous
> languages", and the phrasing clarifying that "languages" means it is
> a collection. Nothing else.  Nada. Zilch.

And as far as the ISO 639-2 standard is concerned, the only thing defining
"bal" is the phrase "Baltic (Other)", which means anything tagged by it
is in a language belonging to the "Baltic" collection.  So I decide that
"Baltic" means the Baltic Sea, and therefore that Swedish, Finnish,
Danish, German, Russian, and Polish are "Baltic (Other)" and it's
perfectly legitimate to tag them "bal".

One gerrymandered interpretation of a short ambiguous phrase is as good
as another.

-- 
John Cowan                                   [email-masked]
        "You need a change: try Canada"  "You need a change: try China"
                --fortune cookies opened by a couple that I know

---

From: Addison Phillips <[email-masked]>
Date: Thu, 19 Apr 2007 08:04:28 +0100

> One gerrymandered interpretation of a short ambiguous phrase is as good

+1

"mis" means "here's some otherwise uncategorized language". I'm sure 
that the librarians (see: ISO 639-2/B) quite frequently have a use for 
this code in their card catalogs (for manuscripts in various languages 
not otherwise given a code). The usefulness of this code in other 
information systems (such as Mark or I work on) is *quite* limited (i.e. 
non-existent), but clearly there is a use for tagging/finding "stuff not 
otherwise cataloged".

In many ways, the relative ambiguity/underspecification of the codes is 
useful. Rather than having dense arguments about where (say) German 
stops and some dialect of German begins, we are free to tag most 
recognizably German documents with "de" and have done. We provide 
several means of applying more specificity (especially with the advent 
of ISO 639-3). But we still can usually get by with the vagueness of 
"de" quite frequently (substitute your favorite language here).

Addison

-- 
Addison Phillips
Globalization Architect -- Yahoo! Inc.

Internationalization is an architecture.
It is not a feature.

---

From: "Kent Karlsson" <[email-masked]>
Date: Thu, 19 Apr 2007 10:28:59 +0200

Mark Davis wrote: 

> defining "mis" is that it is the phrase "Miscellaneous languages", and
> the phrasing clarifying that "languages" means it is a collection. Nothing
> else. Nada. Zilch. 

I mostly agree with Mark here. (The standard does say a little bit more, and there
are indeed a lot left open for the collections, esp. what the hierarchy really is.)

> I think what is more important is to follow up on your suggestion,
> and remove the exclusionary nature of the "(Other)" collections, preferably
> in ISO 639-2 (but failing that, in 4646bis) so that we have a much more
> useful and consistent approach to collections overall. 
	
The standard itself does not support any "exclusionary nature" interpretation,
though that seems to be suggested by the use of the word "other".

		/kent k

---

From: "Kent Karlsson" <[email-masked]>
Date: Thu, 19 Apr 2007 13:40:03 +0200

> though that seems to be suggested by the use of the word "other".

Indeed, the sentence (from the normative text of 639-2) "A collective language
code is not intended to be used when an individual language code or another
more specific collective language code is available." does not even make sense
unless there is an inclusion (specificity) relationship between the coverages
of the collection codes. If "other" really was exclusionary, that would
effectively remove all and any inclusionary relationships.

So, as I suggested earlier, I think that the names should be clarified by
removing the word "other" (replacing it by "language" if not already there,
as well as clarifying "languages to "language" (adding it where missing,
but it is really a collection). Both of these changes would be purely
clarifications, no semantic change, as all of the collection codes are
already inclusive w.r.t. coverage. E.g. 'tai' covers all Tai languages (one
at the time), 'gem' covers all Germanic languages (one at the time), and 'mis'
covers all languages (one at the time). A usage recommendation is not the
same as coverage.

		/kent k

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 20 Apr 2007 13:41:09 +0200

> everyone should be able to get to that document
> -- let me know if you have trouble.

It works with "mozilla 3" Netscape 3 and Lynx 2.8.5.

ACK for 1, 2, 3, 5, 6, 8, 10, (11), 12, 13, 14
 
NAK for 4, "What?" isn't "mis".  

INCONCLUSIVE for 7, "und" and "und-Latn" are fine,
but "mis-Latn" is utter dubious at best.

NAK for 9, "und" is possible, but "zxx" ia also
possible, while "mis" is clearly mis-leading.

ACK for 11, in the sense of stability it would be
odd to force worldwide re-tagging of "ger" as "gsw"
after its introduction for Schwyzerdütsch.

INCONCLUSIVE for 15, is a tag "en-Zxxx" allowed ?
Clearly "en" is no "unwritten language", only the
content is unwritten in this video/audio example.

Maybe add "i-default" to the discussion of case 9,
or add a new case where "i-default" is appropriate.

Frank

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 20 Apr 2007 13:56:01 +0200

Addison Phillips wrote:

> we are free to tag most recognizably German documents with "de" and
> have done. We provide several means of applying more specificity 
> (especially with the advent of ISO 639-3). But we still can usually
> get by with the vagueness of "de" quite frequently (substitute your
> favorite language here).

-1

For the purpose of matching it's quite important not to confuse "de"
with "ger" while talking about something that should be actually
"gsw", "nds", or "nl".  Likewise it's harmful to confuse "mis" with 
"*" or "und".

Frank

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 20 Apr 2007 14:10:06 +0200

[email-masked] wrote:

> I understand that there are worlds other than XHTML and HTML, but
> I can't come up with a use case (for identifying programming code
> with zxx) outside of those instances.

You could add code as inline text/plain part of a MIME message, and
an IMO "mis"-guided implementation could decide that it "must" use
some odd heuristics to determine a Content-Language header field
(instead of simply not using this header field).

There are other uses of language tags with MIME, see also RFC 2231.
I don't think that "MIME or XML or HTML" is a complete enumeration
of all use cases for language tags.

Frank

---

From: Addison Phillips <[email-masked]>
Date: Fri, 20 Apr 2007 17:17:38 +0100

So if I understand the proposal, it is to deal with the different 
"special case" codes assigned by ISO 639. To paraphrase what 4646 
contains (plus my interpretation of the additions):

"mul" is used to identify content in multiple languages. It SHOULD NOT 
be used when a list of languages (such as Content-Language) can be used 
instead and it SHOULD NOT be used for individual content items (such as 
a single <p> element in an XHTML document).

"und" is used to identify linguistic content whose language cannot be 
determined. It SHOULD NOT be used unless a language tag is required and 
the language cannot be determined.

"zxx" is used to identify non-lingusitic content. It SHOULD NOT be used 
unless a language tag is required by some application or protocol for 
some non-human language content item.

"mis" is used to identify linguistic content whose language is known but 
cannot otherwise be identified. It is commonly used when the range of 
language tags is constrained (for example, to ISO 639-2, as in MARC, for 
example) or for languages not otherwise categorized. It SHOULD NOT be 
used unless a language tag is required and no other means of identifying 
the language is available.

In all cases, where permitted, omitting the language tag is preferable 
to using these codes. Language tags SHOULD NOT incorporate these subtags 
as a prefix under any circumstances.

This covers all of the cases except that where the language is known and 
can be identified using a regular subtag.

Any other cases?

Addison

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 20 Apr 2007 09:23:55 -0700

comments below. BTW, not everyone will be familiar with ACK and NAK.

> NAK for 4, "What?" isn't "mis".

As I said in email, I see nothing in the standard that indicates that "mis"
is not valid for English, but valid for, say, Ainu or anything else. (I
added a note to the examples) If you can demonstrate that, by quoting from
the ISO 639 standard or data, I'd appreciate it.

> after its introduction for Schwyzerdütsch.

Agreed.

> content is unwritten in this video/audio example.

It certainly makes sense to say that a video of Bugs Bunny has no written
language content, and that's how we interpret Zxxx.
(Zxxx has no use otherwise. It would not make sense to have a script tag
that could only be applied to content in languages which are currently
unwritten, since you know that from the language anyway, and such an
interpretation would make BCP 47 by definition unstable -- it is always
possible to develop written form of that language which would invalidate the
tag.)
Added note to document.

> or add a new case where "i-default" is appropriate.

I added "i-default" at the top, since as far as I can tell, it is
essentially equivalent to "und" or maybe "mis". I think we should discourage
it, myself, since it is grandfathered and cannot be combined.

Frank

---

From: "Kent Karlsson" <[email-masked]>
Date: Fri, 20 Apr 2007 18:42:37 +0200

> content is unwritten in this video/audio example.

My take on Mark's examples list:

1. Ok.

2. Depends, if that is actually some image with text in some
language or a sound snippet in some language, then clearly a
language tag can be applied. However, in practice I don't
expect many images to be language tagged, especially not if
the text is incidental.

3. From a "bottom-up language derivation" point of view, ok
(maybe a human tagger knows the language, perhaps from context).

4. 'en' is very much recommended over 'mis', though 'en'
is covered by 'mis'. One should only use 'mis' if a more specific
tag is not available (say for example that one only uses tags for
individual languages in the 'tai' family, 'en' would not be
available, so use 'mis' for content that is actually 'en').

5. I would side with Peter on "predominant language" of the text.
Minor digressions (short quotes) should not lead to 'mul' tagging;
ideally the quote fragment should be tagged as 'de' (if possible),
while the full sentence would have a (usually inherited) 'en' tag.
(But in practive I would not expect people writing quotes in another
language than the main text of the document to actually language
tag each little quote.)

6. Ok.

7. The key here is that the "process" (or human) *recognises* 
the text as being in a language it (she) "knows". If there is
no more specific tag for that language, then use 'mis' (which
covers any language), not 'und'.

8. 'und' or 'mis' or 'mul' (depending). Best bet is probably 'und'.

9. "" (if allowed as an extension), 'zxx', or 'und';
NOT 'art', NOT 'mis', (NOT i-default in a BCP 47 context).

10. 'de', but 'gem' can be used if one for some reason does
not have access to using 'de' (in a BCP 47 context we do have
access to using 'de'...). Note: there is no 'ger' tag, so
I used 'gem' here, which I think was the intent.

11. Ok, but 'gsw' is recommended over 'gem' (when one has
access to using the 'gsw' code). Note: there is no 'ger' tag,
so I used 'gem' here, which I think was the intent.

12. Ok, but 'sa' is recommended over 'pra' (when one has
access to using the 'sa' code).

13. See my comments on cases 2 and 5 above. Just because one
has images in/with the text does not make the 'mul' tag
appropriate. 'zxx' is a kind of neutral here.

14. Ok.

15. 'en', not 'en-Zxxx' (in a BCP 47 context). I don't think
'Zxxx' was intended for such a case. Zxxx does NOT mean "no
script content". OTOH, I'm not sure when it *is* appropriate
to use.

	/kent k

---

From: [email-masked]
Date: Fri, 20 Apr 2007 10:13:47 -0700

> expect many images to be language tagged, especially not if

Advertisements, animations, header images, and marketing graphics 
frequently have text that is not incidental. A very common use case.

Karen Broome

---

From: "Kent Karlsson" <[email-masked]>
Date: Fri, 20 Apr 2007 19:24:25 +0200

> "special case" codes assigned by ISO 639. To paraphrase what 4646 
...
> "mis" [...]

'mis' is NOT a "special" code in the 639 sense. It is a collection
code (among many other collection codes). The only thing "special"
about it is that it is the "top level" collection (in the inclusion
hierarchy), but that does not make it "special" in the ISO 639 sense.

		/kent k

---

From: [email-masked]
Date: Fri, 20 Apr 2007 10:24:21 -0700

I think this is a broadening of the definition and perhaps the scope of 
ISO 639. To say "or programming source code" makes a positive statement of 
identification that is outside the bounds of the standard. 

The Zxxx script tag and Zxx tag would have been better defined as "Not 
Applicable" which is the semantic suggested below. The concepts of 
"non-linguistic language" and "unwritten script" are oxymoronic.

Regards,

Karen Broome

Frank Ellermann <[email-masked]> 
04/20/2007 05:10 AM

To
[email-masked]
cc

Subject
[Ltru] Re: Language Tag Special Cases

Frank

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 20 Apr 2007 19:37:47 +0200

Addison Phillips wrote:

> "special case" codes assigned by ISO 639.

Is "the proposal" Mark's table with examples ?  I hope we identify
some cases where subtags aren't intuitively clear (based on their
registry records), and fix this by adding comments, and/or text to
4646bis, and/or publish a "final table" on Stephane's site as part
of the BCP 47 FAQ, and/or ask Richard to add this table or a link
to his article.  Last but not least I guess that Mark has his own
idea what to with this or a "final" table for CLDR.

> To paraphrase what 4646 contains (plus my interpretation of the
> additions):

> a single <p> element in an XHTML document).

The 4646 4.1 bullet 5 explanation for "mul" is okay.  I don't get
what you're saying here, why would a <p lang="mul" xml:lang="mul">
be falsch eller verboten if nada est autrefois acquit</p> ?

> determined. It SHOULD NOT be used unless a language tag is required
> and the language cannot be determined.

Yes, that's better than 4646 4.1 bullet 4, it allows und-Latn.

s/cannot/could not/  At the time when it was attempted it turned
out to be too difficult.  It's not necessarily forever impossible
to determine the language (or its absence in the form of "zxx").

> some non-human language content item.

+1, maybe add programming languages as example.  But it's probably
better to tackle this case in a comment directly in the registry.

> "mis" is used to identify linguistic content whose language is
> known but cannot otherwise be identified. It is commonly used when
> the range of language tags is constrained (for example, to ISO 639-2,
> as in MARC, for example) or for languages not otherwise categorized.
> It SHOULD NOT be used unless a language tag is required and no other
> means of identifying the language is available.

Yes, s/, as in MARC, for example// unless you intend to create fresh
informative references with "acronym expansion on first use".   We're
not yet directly supporting restricted subsets of the subtag registry,
and "mis" isn't important enough to start this major task.  How about:

| "mis" is used to identify linguistic content whose language is
| known but cannot otherwise be identified. It is commonly used for
| languages not otherwise categorized.  It SHOULD NOT be used unless
| a language tag is required and no other means of identifying the
| language is available.

After looking at <http://xml.coverpages.org/marc.html> I'd really
prefer to stay OUT of this minefield.  It has "beware of ratholes"
all over the simple page.

> In all cases, where permitted, omitting the language tag is
> preferable to using these codes. Language tags SHOULD NOT
> incorporate these subtags as a prefix under any circumstances.

NAK, Mark's und-Latn was at least possible.  And some of his other
examples could be tagged as "zxx-Latn" if the tagger really wants
this.

> Any other cases?

i-default with a reference to RFC 2277.

Frank

---

From: [email-masked]
Date: Fri, 20 Apr 2007 10:49:00 -0700

Hmm. Seems like a rather unlikely use case.  In the example below, how 
would tagging the MIME text as "zxx" help this misguided implementation? 
Could it be potentially misleading to tag this content as non-linguistic? 
I think the case below is "und".

(Again, I never implied that lang tags are only used in HTML and XHTML. I 
don't even use lang tags in those contexts in my current work! I am 
specifically talking about use cases for the zxx tag used in reference to 
a code snippet. That's it.)

Thanks for the example, Frank.

Karen Broome

To
[email-masked]
cc

Subject

Frank

---

From: Addison Phillips <[email-masked]>
Date: Fri, 20 Apr 2007 19:31:11 +0100

You're right. My bad. Looking at the actual definition (which I finally 
got around to---this list generated a ton of very long emails in the 10 
days I was offline) helps a lot. 'mis' includes exactly 133 specific 
"miscellaneous" languages and doesn't include the definition I supplied 
in the previous message.

Addison

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 20 Apr 2007 21:17:38 +0200

Mark Davis wrote:

> BTW, not everyone will be familiar with ACK and NAK.

After a good result with "define:MARC" I now also checked out
"define NAK" (with or without colon has different effects),
it works as designed.

> As I said in email, I see nothing in the standard that indicates
> that "mis" is not valid for English, but valid for, say, Ainu or
> anything else.

Yes, you said this, but I found John's explanations, supported by
Peter to some degree, far more convincing.  Roughly a set of other
languages not belonging to any other registered set of languages.

For our purposes they got the name "mis" wrong.  The semantics is
okay, once you know what it's supposed to be.  It has the decent
charme of UM in country codes, a better name could have been "mol",
minor outlying languages.

> If you can demonstrate that, by quoting from the ISO 639 standard
> or data, I'd appreciate it.

<rant> The LOC pages don't work with my browser, I'm not inclined
to check why.  There's a reason why copying such lists to a public
ASCII text file in a reasaonable format hosted by IANA is a very
good thing.  I'm also not going to pay CHF 64 for an obscure 639-5,
a PDF with 356 KB, that's likely three pages DIN A4, and nobody
knows what version of "proprietary data format" it needs. </rant>

I trust that John and Peter know what they're talking about when
it passes my subjective plausibility checks.  The "mis" issue was
plausible, it makes sense to have a set of items not belonging to
any other set in an enumeration of sets.  Your concept of a set of
all items also makes sense, but I'd expect a name like "all" or
"any" for this purpose, not "mis".

Some of my most important local directories have names like "misc"
or "etc", in both cases this isn't the root.  If you feel that you
must have an "any" tag go for it.  But IMO you're free to (ab)use
"mis" as it pleases you, locally, just don't use your local "mis"
on public pages where it could confuse tag consumers with another
idea about the non-existing "any" tag.

> It certainly makes sense to say that a video of Bugs Bunny has no
> written language content, and that's how we interpret Zxxx.
> (Zxxx has no use otherwise

Maybe the Zxxx description "code for unwritten language" should
also get a comment and paragraph in 4646bis then:  Your use case
is apparently "none", not only "n/a".

We have "zxx" for "not a (human) language", and then "Zxxx" for
"not a script" fits.  But "not a script" isn't exactly the same
as "unwritten", it's a border case.  Maybe Addison can copy the
proposed "zxx" paragraph into a "Zxxx" paragraph, using the same
normative SHOULD NOT clauses.

> I added "i-default" at the top, since as far as I can tell, it
> is essentially equivalent to "und" or maybe "mis".

John's RFC 2277 quote was a bit more specific than "und", it's
related to "en" with a touch of "und".

> I think we should discourage it

Yes, the source (RFC 2277) already does this.  We could add a
comment "see BCP 18 (RFC 2277)" to the BCP 47 registry.

Frank

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 20 Apr 2007 12:47:52 -0700

On 4/20/07, Frank Ellermann
<[email-masked]<//[email-masked]/>>
> languages not belonging to any other registered set of languages.

That is one interpretation, but it is not supported by the text. It would be
a bad interpretation, since it breaks stability. As I say, I think the best
thing we could do at this point would be to strongly discourage the use of
"mis" in 4646bis because the semantics are unclear.

> okay, once you know what it's supposed to be.

But we don't know that that is the only interpretation, according to the
text it could be a variety of things.

 It has the decent
> knows what version of "proprietary data format" it needs. </rant>

Pick a different browser. The text doesn't , as far as I can see, support
the contention, and I haven't seen anything from John or Peter to contradict
that. Much as I respect their judgments on a whole variety of subjects, on
this one I think they are interpreting the text as they think it should have
been written, not as it is.

Again, if we can agree to discourage "mis", then we don't have to keep
having this discussion.

> idea about the non-existing "any" tag.

"Misc" as exclusionary makes sense any time you are in an environment that
forces a disjunctive partition. But ISO 639 does not require a partition.

It's not abusing a code if there is no text in ISO 639-2 that would lead one
to think that one is abusing it.

> normative SHOULD NOT clauses.

I disagree. Zxxx fits a perfectly legitimate need; it clearly applies to
languages. There are two possible interpretations that could be given to an
application of it to content.

   1. content that is not written
   2. content that cannot be written

The second one I find incoherent.

> related to "en" with a touch of "und".

Yes, there are some nuanced differences between them, but when I look at
examples, it looks really hard to say what the difference would be between
them in terms of tagging content. Perhaps I'm just missing something.

> comment "see BCP 18 (RFC 2277)" to the BCP 47 registry.

Or even better yet, a comment in the registry and the text explicitly
discouraging it.

Frank
> [email-masked] <//[email-masked]/>

---

From: "Kent Karlsson" <[email-masked]>
Date: Fri, 20 Apr 2007 21:58:32 +0200

Addison Phillips wrote:
> "miscellaneous" languages [...]

Not quite. 'mis' is a collection code, and as such is more vaguely
specified than the individual language codes. The only reasonable
interpretation of 'mis', I maintain, is that it covers all an any
languages, but that is should not be used ("not intended for") *when 
there is a more specific language code (collection or individual)
"available"*. In a 639-3 setting there would thus be very little
use for 'mis', even though it covers all (any, really) language(s).

However, in a hypothetical setting of just having access to say
some of the codes covered by the 'fiu' collection, as well as
'mis' of course, one would use 'mis' for, among many other languages,
English.

I have no idea where you got the list of 133 specific languages from.
As far as I can tell, it appears to have no normative connection
with the 639 standards themselves.

	/kent k

---

From: Peter Constable <[email-masked]>
Date: Fri, 20 Apr 2007 13:25:54 -0700

+1

-----Original Message-----
From: Addison Phillips [mailto:[email-masked]]
Sent: Friday, April 20, 2007 9:18 AM
To: Frank Ellermann
Cc: [email-masked]
Subject: Re: [Ltru] Re: Language Tag Special Cases

Any other cases?

Addison

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 20 Apr 2007 22:37:19 +0200

[email-masked] wrote:

> Seems like a rather unlikely use case. In the example below, 
> how would tagging the MIME text as "zxx" help this misguided
> implementation?

Speculation, the implementation could be designed to use some
heuristics to guess the language of plain text files.  

Mark reported that they use a statistical approach for the
identification of languages, because language tags in the wild
are more often wrong than anything else.

> Could it be potentially misleading to tag this content as 
> non-linguistic? I think the case below is "und".

Depends on implementation details.  It could recognize some
MIME types, and look "into" the file for known types, and at
some point it might decide "programming language X" (somehow
http://code.google.com has to work, after all, even if they
don't yet support REXX as my favourite "missing language" :-)

At this point it's possible that a smart procedure says "it's
REXX", and a dumb caller then decides, "okay, if it's not 'en'
I use 'zxx'".  We could say that this is a bug, and Addison's
proposal "SHOULD NOT use 'zxx'" confirms this theory, but it 
can take some time until somebody fixes this plausible bug in
a hypothetical application adding Content-Language tags to all
inline MIME text parts, even where that's a bad idea.

All we can do is say "SHOULD NOT" in 4646bis, and hope that
implementors get the idea.  And implementations based on 4646
or 3066 have an excuse to violate this future "SHOULD NOT", it
isn't obvious.

> I am specifically talking about use cases for the zxx tag
> used in reference to a code snippet. That's it.

RFC 3282 updates RFC 1766.  Somehow RFC 4646 managed to have no
reference to RFC 3282 and RFC 2277.  IMHO we should fix that in
4646bis, these RFCs are really relevant.

A very important example in RFC 3282 is this:

|     Content-type: multipart/alternative
|     Content-Language: da, de, el, en, fr, it

Content-Language can be used with multipart/alternative, here 
for alternative parts in the languages da, de, el, en, fr, it.

Maybe it can be also used with other multipart/* or message/*
subtypes, and then the question is, do all individual MIME parts
"inherit" the Content-Language, unless they specify their own
Content-Language ?

That would be exactly the same situation as a <html lang="en">
or a similar tag containing an element that's clearly not "en".

One thing is sure, the syntax is "one or more tags" in RFC 3282,
there's no xml:lang="" trick for Content-Language header fields.

Frank

---

From: Addison Phillips <[email-masked]>
Date: Fri, 20 Apr 2007 22:57:29 +0100

> heuristics to guess the language of plain text files.  

(laughing) This is, in practice, the case for most Web content today: 
even when language tags are right, heuristics are still generally used 
to determine the language.

> are more often wrong than anything else.

The more direct problem is that language tags tend to be rare. When >80% 
of your data has to be scanned heuristically for language and a 
significant proportion scanned heuristically for character encoding 
and/or format, it is usually more optimal to pass 100% of your data to 
the scanner than to supply costly additional looks to see if there is 
some embedded metadata.

And then Mark's not wrong about the prevalence of "wrong" tags. There is 
another problem as well: lots of data is tagged with an inconvenient tag 
(either too long or too short) requiring additional scanning to remediate.

> inline MIME text parts, even where that's a bad idea.

I purposefully proposed SHOULD NOT: we cannot ban the usage and there 
are valid use cases for it. But we should be as direct and stringent as 
practical to prevent the majority of taggers for using it for actual 
natural language tagging.

Actually, if you want to tag a file full of code with a language tag, 
the tag should probably match the language used for the comments (you 
comment your code, right?) or for the variable names (you have variables 
with names other than 'x' or 's3', right?). If a lot of these turn out 
to be "en-x-hacker", so be it. Embedded binary junk or embedded 
programming junk are just noise.

> or 3066 have an excuse to violate this future "SHOULD NOT", it

The recommendations on tag choice really should guide users of any of 
these older implementations. Remember: we're talking about tag *choice* 
here. Tags that uses these subtags, even junk like 
"zxx-Hant-AQ-x-foolish", are well-formed and valid.

> 4646bis, these RFCs are really relevant.

Actually, I think I proposed it but it was turned down as inconvenient. 
I agree we should obsolete 3282 (by updating it) or at least reference it.

> for alternative parts in the languages da, de, el, en, fr, it.

ACK :-)

> there's no xml:lang="" trick for Content-Language header fields.

Yes there is: it's called "no Content-Language header present".

Addison

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 20 Apr 2007 17:36:23 -0700

> be used when a list of languages (such as Content-Language) can be used

agreed

> a single <p> element in an XHTML document).

BCP 47 is silent on this point, so I don't think this is a good paraphrase.

> determined. It SHOULD NOT be used unless a language tag is required

agreed.

and
> the language cannot be determined.

BCP 47 is silent on this point, so I don't think this is a good paraphrase.
Also, the most we could require in a future version is "is not determined",
not "cannot be determined".

> some non-human language content item.

BCP 47 contains no instance of 'zxx'. Thus the above is not a paraphrase of
BCP 47. Moreover, I disagree that it should not be used.

> the language is available.

The above is not a paraphrase of BCP 47, or of ISO 639-2. It may be your own
opinion of what we should say in 4646bis (and I agree with part of it), but
is not justifiable on the basis of the current standard.

> as a prefix under any circumstances.

Also not a paraphrase of BCP 47, but your own addition. Moreover, I disagree
strongly with this. "und-Latn" is a perfectly reasonable code, that conveys
useful information. So is "zxx". If I have

<content xml:lang="en">He showed me a picture of </content><content
xml:lang="zxx">143kl;ufa) iop(&uweiorqhjkl2341l
kj#@!$Jkdfj;afe</content><content xml:lang="en"> and said </content><content
xml:lang="mis">Igonda flatunicai vbinkli?<content>

then according to your proposal, I SHOULD nuke various  information and be
left with:

<content xml:lang="en">He showed me a picture of
</content><content>143kl;ufa) iop(&uweiorqhjkl2341l
kj#@!$Jkdfj;afe</content><content xml:lang="en"> and said
</content><content>Igonda flatunicai vbinkli?<content>

I don't see where this is a good thing. I do think, which is maybe the
direction you are headed, that we should have language that says that you
SHOULD tag with more specific primary language tags than collections or
specials, where possible.

> can be identified using a regular subtag.

I added some notes to http://docs.google.com/Doc?id=dfqr8rd5_11g425c9 on
these issues. I don't think you cover all the cases (and I disagree with
your approach in some cases).

Addison

---

From: John Cowan <[email-masked]>
Date: Sat, 21 Apr 2007 02:02:16 -0400

Mark Davis scripsit:

> written language content, and that's how we interpret Zxxx.  (Zxxx has
> no use otherwise. It would not make sense to have a script tag that
> could only be applied to content in languages which are currently
> unwritten, since you know that from the language anyway, and such
> an interpretation would make BCP 47 by definition unstable -- it is
> always possible to develop written form of that language which would
> invalidate the tag.)

+1, and I have posted an ISO 15924 change request to this effect.

> essentially equivalent to "und" or maybe "mis". I think we should
> discourage it, myself, since it is grandfathered and cannot be combined.

It makes no sense to tag a received document "i-default", as i-default
represents an abstraction.  It makes sense to request an i-default
version of a document, which may contain English only or may be
multilanguage.  It must include English, but need not be the same
as the English version (if there is one) of the document.

-- 
John Cowan  [email-masked]   http://ccil.org/~cowan
Consider the matter of Analytic Philosophy.  Dennett and Bennett are well-known.
Dennett rarely or never cites Bennett, so Bennett rarely or never cites Dennett.
There is also one Dummett.  By their works shall ye know them.  However, just as
no trinities have fourth persons (Zeppo Marx notwithstanding), Bummett is hardly
known by his works.  Indeed, Bummett does not exist.  It is part of the function
of this and other e-mail messages, therefore, to do what they can to create him.

---

From: John Cowan <[email-masked]>
Date: Sat, 21 Apr 2007 02:07:57 -0400

> I have no idea where you got the list of 133 specific languages from.

http://www.ethnologue.com/14/show_iso639.asp?code=mis , probably.

> with the 639 standards themselves.

True.  As I have been arguing, however, the normative text and
tables of ISO 639 capture only part of its meaning: it is normative
that "en" means "English", but there is no normative definition
of "English", nor is there ever likely to be one.

-- 
A mosquito cried out in his pain,               John Cowan
"A chemist has poisoned my brain!"              http://www.ccil.org/~cowan
        The cause of his sorrow                 [email-masked]
        Was para-dichloro-
Diphenyltrichloroethane.                                (aka DDT)

---

From: John Cowan <[email-masked]>
Date: Sat, 21 Apr 2007 02:17:21 -0400

Mark Davis scripsit:

> That is one interpretation, but it is not supported by the text.  

As I have argued repeatedly, the normative content of ISO 639 is
insufficient to properly interpret it.

> I think the best thing we could do at this point would be to strongly
> discourage the use of "mis" in 4646bis 

+1, as there will be a sufficient number of specific language codes
for almost anything, and "und" is satisfactory as a representation of
ignorance or indifference.

> The text doesn't, as far as I can see, support the contention, and I
> haven't seen anything from John or Peter to contradict that. Much as
> I respect their judgments on a whole variety of subjects, on this one
> I think they are interpreting the text as they think it should have

I am not interpreting the text at all: I am relaying the established
interpretive tradition.  By its very nature, an interpretive tradition
is outside the text.

> "Mis" as exclusionary makes sense any time you are in an environment
> that forces a disjunctive partition.

The partition need not be fully disjunctive; a hierarchical partition
works fine too, with "mis" as one of the top-level partitions, the one
which includes everything not included elsewhere.

> But ISO 639 does not require a partition.

It has traditionally been interpreted as one.

+1

> Or even better yet, a comment in the registry and the text explicitly

There is no reason to discourage "i-default" in its proper context.
For example, given a document available in French only, a request
for "i-default" might return the text "This document is available in
French only."  Alternatively, the "i-default" version of a document also
available in English might use simpler words or constructions so as to
be understood by a wider range of English-speakers.

-- 
Only do what only you can do.               John Cowan <[email-masked]>
  --Edsger W. Dijkstra's advice
    to a student in search of a thesis

---

From: John Cowan <[email-masked]>
Date: Sat, 21 Apr 2007 02:41:38 -0400

> can be identified using a regular subtag.

With the addition of i-default, I think that sums things up admirably,
and should be added to the next draft of 4646bis.

-- 
John Cowan    <[email-masked]>     http://www.ccil.org/~cowan
But no living man am I!  You look upon a woman.  Eowyn I am, Eomund's daughter.
You stand between me and my lord and kin.  Begone, if you be not deathless.
For living or dark undead, I will smite you if you touch him.

---

From: "Kent Karlsson" <[email-masked]>
Date: Sat, 21 Apr 2007 11:28:50 +0200

> http://www.ethnologue.com/14/show_iso639.asp?code=mis , probably.

That is just a snapshot of what was 'mis' but not "otherwise catalogued"
in a particular setting at a particular point in time. It's not the
definition of 'mis' in ISO 639 by any account.

> of "English", nor is there ever likely to be one.

True. And I would not expect that to ever happen for the individual
languages. But an explicit coverage list (admittedly with an implicit
"or any other variety that is not explicitly listed") is already given
for the "macrolanguages" (a concept not yet in ISO 639-2) for 639-3.
Are similar coverage lists (with implicit "or any other language in
this group not explicitly listed") planned for ISO 639-5?

The "macrolanguages" and "collections" are quite similar in the sense
that for both of them they cover all of the individual languages
(and for the collection, also all of the more specific collections)
that are (by convention) in their respective groups.

One of the differences between "macrolanguages" and "collections",
however, is that for "macrolanguages" it is quite ok to use the
macrolanguage code for any of the "sublanguages" covered, whereas
for "collections", the collection code *should not* be used
(even though it could) if there is a more specific code "available".
This goes for 'mis' too, as it is just one of the collection codes,
albeit the top level collection code.

		/kent k

---

From: John Cowan <[email-masked]>
Date: Sat, 21 Apr 2007 06:21:34 -0400

> this group not explicitly listed") planned for ISO 639-5?

Not as of the last time I saw a 639-5 draft, nor have I heard anything
to suggest it.  Which means that figuring out what a "Tai language" is
will depend on encyclopedic information outside the standard, just as
figuring out what "English" is.

FWIW, that draft is harmonized with 639-2, but does not have any
"(Other)" strings, so it takes the inclusive view of collections.

> (even though it could) if there is a more specific code "available".

Collection codes are useful for certain kinds of partial ignorance,
however.  For example, if you are classifying documents, you
may know that a particular document is written in some Saami
language, but may lack the resources to figure out which one.
In that case, the collection code "smi" is appropriate.

> albeit the top level collection code.

On that view of "mis" it represents total ignorance, which is
already well-represented by "und".

-- 
John Cowan  [email-masked]  http://ccil.org/~cowan
If I have seen farther than others, it is because I was standing on
the shoulders of giants.
        --Isaac Newton

---

From: "Kent Karlsson" <[email-masked]>
Date: Sat, 21 Apr 2007 12:49:58 +0200

> "(Other)" strings, so it takes the inclusive view of collections.

Hmm, the "(Other)" has no semantic significance as ISO 639-2 is written.
So the coverages for ISO 639-2 collection codes would be the same
as for the corresponding (i.e. same) collection codes of ISO 639-5.

> In that case, the collection code "smi" is appropriate.

That I think may be fine, given that "available" is also interpreted
along the dimension of "partial ignorance". Though that dimension
of "available" is not clear, as many other details, from the text
of ISO 639-2.

> already well-represented by "und".

Though not formally a collection code, if one wants to see it as
a collection code, 'und' is more encompassing than 'mis', as 'und'
covers not only what 'mis' covers, but it also covers 'zxx' and
'mul' which are not covered by 'mis'.

	/kent k

---

From: Peter Constable <[email-masked]>
Date: Sat, 21 Apr 2007 08:29:44 -0700

From: Kent Karlsson [mailto:[email-masked]]

> definition of 'mis' in ISO 639 by any account.

It isn't a normative definition, but it certainly is an analysis at that point in time that is consistent with the established interpretative tradition (as John so aptly put it).

Peter

---

From: Marion Gunn <[email-masked]>
Date: Sun, 22 Apr 2007 14:43:35 +0000

I agree with what Karen says below.

Even without the 'oxy-'.:-)

Having received 62 LTRU msgs in 24 hrs, all but 2 from the US, I only  
read her 2 (not because I expect the others to be relatively info- 
empty, but because non-US msgs are tending towards more favourable  
noise-2-content ratios recently).
mg

On 20 Apr 2007, at 17:24, scríobh [email-masked]:

> ...
> The Zxxx script tag and Zxx tag would have been better defined as  
> "Not Applicable" which is the semantic suggested below. The  
> concepts of "non-linguistic language" and "unwritten script" are  
> oxymoronic.

- -
Marion Gunn * EGTeo (Estab.1991)
27 Páirc an Fhéithlinn, Baile an
Bhóthair, Co. Átha Cliath, Éire.
* [email-masked] * [email-masked] *

---

From: Marion Gunn <[email-masked]>
Date: Mon, 23 Apr 2007 10:58:23 +0000

Dear Randy,

I am responding publicly to your off-list warning (below), because I  
am moved to make a public apology to any fellow-members of this list  
who found my msg below in any way offensive, as you have clearly  
found it so. The feeling here in the office was that the comment  
(below) is a witty way of pointing out how tiresome it is for some of  
us to be asked to tolerate so many msgs from one single country on a  
topic not related to cataloguing human languages (instead of many  
msgs from many different countries on topics which are).

mg

On 23 Apr 2007, at 05:50, scríobh Randy Presuhn:

> Hi -
>
> Please refrain from unprofessional commentary.
> See RFC 3934 for the procedural details.
> If necessary, your next warning may be public,
> followed by suspension if bad behaviour continues.
>
> Randy
> ltru co-chair
>
>
>> From: "Marion Gunn" <[email-masked]>
>> To: "LTRU Working Group" <[email-masked]>
>> Sent: Sunday, April 22, 2007 7:43 AM

- -

---

From: "David Dalby" <[email-masked]>
Date: Mon, 23 Apr 2007 11:51:53 +0100

Dear All,
I have refrained recently from responding to matters discussed on this and
its parallel forum, because I have felt so out-of-place among largely
trans-Atlantic experts in the rules of program(m)ing.  
Marion's humorous response is entirely appropriate and "on-topic", and I
look forward to the day when the coding of the world's languages may be
discussed and resolved by many more linguists and by many more speakers of
different languages from around the world.
Please forgive my intruding onto the bridge. I shall return to the
boiler-room.
Best wishes to all,
David

 _____________________________________________________
 
David Dalby
l'Observatoire Linguistique / the Linguasphere Observatory
Hebron, Whitland, Wales SA34 0XT
 
Tel: +44 (0)1994 419660

-----Original Message-----
From: Marion Gunn [mailto:[email-masked]] 
Sent: 23 April 2007 11:58
To: LTRU Working Group
Subject: Re: Off-list RFC 3934 warning ([Ltru] Re: Language Tag Special
Cases)

Dear Randy,

mg

- -

---

From: Misha Wolf <[email-masked]>
Date: Mon, 23 Apr 2007 13:29:25 +0100

Hi David,

I'm divided as to the wisdom of replying.  If this has 
reached you, then the urge to do so overcame the urge 
not to do so.

It seems self-evident that if you want people who are 
not currently on this list to join it in order to 
participate, you should be addressing yourself to them, 
rather than to the existing members of the list.  You 
could suggest the creation of a list [email-masked], 
whose membership would consist of those persons who are 
*not* members of the existing list.  You could then use 
the new forum to encourage them to join the ltru list.

Yours transatlantically,
Misha
London, UK

-----Original Message-----
From: David Dalby [mailto:[email-masked]] 
Sent: 23 April 2007 11:52
To: 'Marion Gunn'; 'LTRU Working Group'
Subject: RE: Off-list RFC 3934 warning ([Ltru] Re: Language Tag Special
Cases)

Dear All,
I have refrained recently from responding to matters discussed on this
and
its parallel forum, because I have felt so out-of-place among largely
trans-Atlantic experts in the rules of program(m)ing.  
Marion's humorous response is entirely appropriate and "on-topic", and I
look forward to the day when the coding of the world's languages may be
discussed and resolved by many more linguists and by many more speakers
of
different languages from around the world.
Please forgive my intruding onto the bridge. I shall return to the
boiler-room.
Best wishes to all,
David

This email was sent to you by Reuters, the global news and information company. 
To find out more about Reuters visit www.about.reuters.com

Any views expressed in this message are those of the individual sender, 
except where the sender specifically states them to be the views of Reuters Limited.

Reuters Limited is part of the Reuters Group of companies, of which Reuters Group PLC is the ultimate parent company.
Reuters Group PLC - Registered office address: The Reuters Building, South Colonnade, Canary Wharf, London E14 5EP, United Kingdom
Registered No: 3296375
Registered in England and Wales

---

From: "Randy Presuhn" <[email-masked]>
Date: Mon, 23 Apr 2007 11:25:40 -0700

Hi Marion -

You were warned privately.  This message constitutes a public warning.
Such unprofessional commentary as is in your "apology" has
no place in an IETF discussion.  As you well know, participation
in the IETF is by individuals, not by national representatives.

If bad behaviour continues, your posting privileges to the
ltru WG mailing list may be suspended. See RFC 3934
for the procedural details.

Randy
ltru co-chair

================================================================
Status:  U
From: Marion Gunn <[email-masked]>
Subject: Re: Off-list RFC 3934 warning ([Ltru] Re: Language Tag Special Cases)
Date: Mon, 23 Apr 2007 10:58:23 +0000
To: LTRU Working Group <[email-masked]>

Dear Randy,

I am responding publicly to your off-list warning (below), because I =20
am moved to make a public apology to any fellow-members of this list =20
who found my msg below in any way offensive, as you have clearly =20
found it so. The feeling here in the office was that the comment =20
(below) is a witty way of pointing out how tiresome it is for some of =20=

us to be asked to tolerate so many msgs from one single country on a =20
topic not related to cataloguing human languages (instead of many =20
msgs from many different countries on topics which are).

mg

On 23 Apr 2007, at 05:50, scr=EDobh Randy Presuhn:

>> On 20 Apr 2007, at 17:24, scr=EDobh [email-masked]:
>> ...
>> The Zxxx script tag and Zxx tag would have been better defined as =20
>> "Not Applicable" which is the semantic suggested below. The =20
>> concepts of "non-linguistic language" and "unwritten script" are =20
>> oxymoronic.

- -
Marion Gunn * EGTeo (Estab.1991)
27 P=E1irc an Fh=E9ithlinn, Baile an
Bh=F3thair, Co. =C1tha Cliath, =C9ire.
* [email-masked] * [email-masked] *

---

From: Marion Gunn <[email-masked]>
Date: Tue, 24 Apr 2007 11:02:59 +0000

I again offer my apology (without further comment) for your acceptance.
mg

- -

---

From: "Don Osborn" <[email-masked]>
Date: Tue, 24 Apr 2007 09:22:33 -0400

I understand David's point to be that there might be better outreach to
increase the level of involvement of experts with a range of language
expertise (and authority). 

To take the case of sub-Saharan African languages, I find that I am
practically the only person actively trying to focus on what the
implications of a lot of the discussions are for them, and what their needs
might imply for the form of language tagging. Not the only person to think
about it, and basically I know enough to know how little I really know, but
one of the very few to consistently raise African language issues. Africa
deserves better.

The simple answer to invite colleagues is sometimes too simple: some African
language/linguistics experts (wherever they are from or based) are clueless
about language tags. One (an American professor) recently asked on behalf of
colleagues who are setting up a workshop, "We don't know what is ISO-639?"
Outreach is more than just getting people on the wagon to take them to the
polling station (or whatever metaphor one may prefer). How can potentially
interested experts be educated about the language tagging issues from the
bottom up, in a succinct way, to facilitate their participation and input?
They know the languages and the issues, and could add important dimensions
to the discussions and decisions that we are missing.

Problem is that outreach requires some effort and resources, and here, like
the case with Unicode, there is no such. So, the system remains largely
North American and West European.

The problem with regard to Africa is a larger one. ISO published last year a
short document on increasing African government capacities and involvement
in ISO standards processes.* This is all ISO standards, not just language or
ICT related. But as a general rule, African involvement in language and ICT
standards is very low, especially taking into account LTRU, IETF-languages
etc.

I'm not sure what the answer is (again, the problem of lack of resources and
time usually cut such discussions short), but it is an issue that shouldn't
be lightly dismissed. And it does seem to be most appropriately addressed to
the kind of people who are already on [email-masked] .

Don Osborn

* International Organization for Standardization. 2006. "ISO and Africa."
June 2006. 
http://www.iso.org/iso/en/comms-markets/developingcountries/pdf/iso_and_afri
ca.pdf

> From: Misha Wolf [mailto:[email-masked]]
> Sent: Monday, April 23, 2007 8:29 AM
> To: LTRU Working Group
> Subject: OT: Attracting wider participation [was: RE: Off-list RFC 3934
> warning ([Ltru] Re: Language Tag Special Cases)]
> 
> Hi David,
> 
> Reuters Limited is part of the Reuters Group of companies, of which
> Reuters Group PLC is the ultimate parent company.
> Reuters Group PLC - Registered office address: The Reuters Building,
> South Colonnade, Canary Wharf, London E14 5EP, United Kingdom
> Registered No: 3296375