Subject: Re: [Ltru] xml:lang syntax

From: "Doug Ewell" <[email-masked]>
Date: Thu, 8 May 2008 23:05:12 -0600

Leif Halvard Silli <lhs at malform dot no> wrote:

>> Except that "mul" is a valid language tag and "mul-en-fr" isn't even 
>> well-formed.
>
> Of course. But the interesting thing is if it can be set up in 
> well-formed way.

A language tag denotes one language.  A protocol may define ways to 
combine multiple language tags into one field, as HTTP has done with 
Accept-Language by separating the tags with semicolons.  But that is 
defined by and specific to the protocol.

--
Doug Ewell  *  Arvada, Colorado, USA  *  RFC 4645  *  UTN #14
http://www.ewellic.org
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages  ˆ

_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Jukka K. Korpela" <[email-masked]>
Date: Fri, 9 May 2008 08:51:44 +0300

Doug Ewell wrote:

> A language tag denotes one language.

For suitable values for "one" and "language", yes.

> A protocol may define ways to

In the XML context, which is what the discussion is about, there is 
normally no need to consider the possibility of referring to several 
languages in one xml:lang attribute. The reason is that for 
mixed-language content, XML lets (and even encourages, so to say) you to 
use nested markup so that you can say exactly, for example, which parts 
of the text are in English and which are in French. This may require 
additional markup elements for the sole purpose of attaching xml:lang to 
some piece of text, but that's nothing odd. For a truly bilingual 
element (with almost equal amounts of text in two languages), you just 
need to select one language for it and override the language information 
in nested elements.

However, XML has no way to specify different languages for different 
_attributes_ of an element. If an element's attribute is in language 
other than than its content, you can use the workaround of an inner 
element, just for specifying the language of content, but this won't 
work for attributes in different languages. The problem is real because 
nowadays people often put textual data in attributes, as opposite to 
(what I see as) the original idea in generalized markup where attributes 
seldom contain text in a human language.

Jukka K. Korpela ("Yucca")
http://www.cs.tut.fi/~jkorpela/

---

From: Leif Halvard Silli <[email-masked]>
Date: Fri, 09 May 2008 14:57:25 +0200

Jukka K. Korpela 2008-05-09 07.51:
> languages in one xml:lang attribute.

But I may still use 'mul'. So it is allready somehow considered.

>  The reason is that for 

I would rather prefer to say

    <p xml:lang="mul">
        <span xml:lang="fi">22 Pistepirko.</span> 
        <span xml:lang="nn">22  Marihønor.</span>
    </p>

so that I can refer to *the paragraph* via CSS as p:lang(mul){}, rather 
than have to wrap that paragraph in to a div element, each with a 
different language tag,

    <div xml:lang="fi">
    <p xml:lang="nn">
        <span xml:lang="fi">22 Pistepirko.</span>
        <span xml:lang="nn">22  Marihønor.</span>
    </p>
    </div>

which I then would have to refer to/filter out via div:lang(fi) p:lang(nn){}

Then, if in iaddition I could - in a well formed way - could extend the 
'mul' tag to tell which languages it covered, then that would be great.

> seldom contain text in a human language.

This point about attributes is a very good point.

The alternative to extend 'mul' is to use several tags, in a legal way. 
But as we allready have seen, then we must specify what that list of 
language tags means: Are they there for the audience or for the content?
-- 
leif halvard silli
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 9 May 2008 07:46:57 -0700

does => does not allow one to

On Fri, May 9, 2008 at 7:46 AM, Mark Davis <[email-masked]>
wrote:

> Because the xml:lang only takes a single language tag, XML does easily mark
> one section of text as being equally well represented by either of two
> language tags. That is, it assumes that you know what the language is, and
> can pinpoint it as having exactly one tag. Reality is a bit messier... And a
> fallback to mul is useless, since that gives you far less information.
>
> This isn't formally a problem with XML, since you can always design a
> format that bundles together multiple xml:langs as applying to some text,
> and you also don't have to use xml:lang to mark text: you can use your own
> attribute, containing a list. But it would be less clumsy if xml:lang took a
> list, or we had xml:langs that did.
>
> Mark
>
>
> On Thu, May 8, 2008 at 10:51 PM, Jukka K. Korpela <[email-masked]>
> wrote:
>
>> Doug Ewell wrote:
>>
>> http://www.cs.tut.fi/~jkorpela/ <http://www.cs.tut.fi/%7Ejkorpela/>
>>
> Mark

-- 
Mark

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 9 May 2008 07:46:26 -0700

This isn't formally a problem with XML, since you can always design a format
that bundles together multiple xml:langs as applying to some text, and you
also don't have to use xml:lang to mark text: you can use your own
attribute, containing a list. But it would be less clumsy if xml:lang took a
list, or we had xml:langs that did.

Mark

---

From: "Phillips, Addison" <[email-masked]>
Date: Fri, 9 May 2008 10:59:22 -0700

To emphasize part of Mark’s note:

xml:lang is an attribute. If you want to declare languages or have an attribute that is a language list you should use your own and not abuse xml:lang. I wrote an article about this:

http://www.w3.org/International/questions/qa-when-xmllang

Just because xml:lang is available for declaring the language of an element doesn’t mean that it is the ONLY way to use language tags in markup languages.

As for Mark’s assertion:

Ø  But it would be less clumsy if xml:lang took a list, or we had xml:langs that did.

I think this would be bad. Lists of language (asserting the language of a span of content) make processing, parsing, validation, and other operations harder. They make it harder for users to tag content, since users have to think of potentially more than one tag. They make it harder to interpret content (since the language tags applied may not go together or even be contradictory). This idea that an item is in “multiple languages” is mostly wrong. If you need a list of languages or have some different application other than declaring the text processing language of an element then you should use your own element or attribute to convey this information.

Addison

Addison Phillips
Globalization Architect -- Lab126

Internationalization is not a feature.
It is an architecture.

From: [email-masked] [mailto:[email-masked]] On Behalf Of Mark Davis
Sent: Friday, May 09, 2008 7:46 AM
To: Jukka K. Korpela
Cc: LTRU Working Group
Subject: Re: [Ltru] xml:lang syntax

Because the xml:lang only takes a single language tag, XML does easily mark one section of text as being equally well represented by either of two language tags. That is, it assumes that you know what the language is, and can pinpoint it as having exactly one tag. Reality is a bit messier... And a fallback to mul is useless, since that gives you far less information.

This isn't formally a problem with XML, since you can always design a format that bundles together multiple xml:langs as applying to some text, and you also don't have to use xml:lang to mark text: you can use your own attribute, containing a list. But it would be less clumsy if xml:lang took a list, or we had xml:langs that did.

Mark
On Thu, May 8, 2008 at 10:51 PM, Jukka K. Korpela <[email-masked]<mailto:[email-masked]>> wrote:

Jukka K. Korpela ("Yucca")
http://www.cs.tut.fi/~jkorpela/<http://www.cs.tut.fi/%7Ejkorpela/>

_______________________________________________
Ltru mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/ltru

---

From: Leif Halvard Silli <[email-masked]>
Date: Fri, 09 May 2008 21:51:08 +0200

Phillips, Addison 2008-05-09 19.59:
> I think this would be bad. Lists of language (asserting the language of a span of content) make processing, parsing, validation, and other operations harder. They make it harder for users to tag content, since users have to think of potentially more than one tag.
>   

That it may be illegal with more thane one tag does not mean that tools 
and users must not know about more than one tag in order to understand 
what that single tag means.

You must for intance know that 'zh' covers 'yum' and vice versa. You 
must try to figure out what "mul" covers. You must figure out what "no" 
refers to. Etc.

>  They make it harder to interpret content (since the language tags applied may not go together or even be contradictory). This idea that an item is in “multiple languages” is mostly wrong. 

If it is only "mostly wrong", then it isn't wrong.

I think the key thing is _pairs_ as  Shawn spoke about or "different 
levels of specificity" as Peter called the same thing:

    "3. declaring a single variety for content/resources at multiple
    levels of specificity."

For instance "mul,en,fr" is a list which says the same thing, becuse 
"en" is encompassed in "mul, as is "fr". Likewise "zh,yum,cmn" is a list 
of the same thing because 'yum' and cmn' ar contained in 'zh'. "no,nn" 
is a list of the same thing."

It should be much better to give a list, in order to tell what you mean 
by that unclear tag "no", or that unclear tag "mul" or that unclear tag 
"zh", instead of  to simply write any of those tags, alone.

> If you need a list of languages or have some different application other than declaring the text processing language of an element then you should use your own element or attribute to convey this information.

One's own attribute or element is not an option in HTML. But I do 
understand the general advice you give here, which builds on the 
assumption that a paragraph language will be the same as its parent, and 
thus you can only mark those parts of it which are in a different language.

However, if you want to say something about that very paragraph as such, 
then you must say that it is multilingual "mul". Or that it is a 
wordlist for Nynorsk-Bokmål "no,nn,nb". For instance, the general 
punctuation rules for multlingual section might differ from that of the 
main language of that resource.
-- 
leif halvard silli
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: Felix Sasaki <[email-masked]>
Date: Mon, 12 May 2008 11:10:40 +0900

> seldom contain text in a human language.

The ITS "Language data category" provides a solutions for this. see
http://www.w3.org/TR/xml-i18n-bp/#EX-devlang-1
for an example of its usage. The problem you are mentioned could be 
solved like this:
<!-- Document is in English, @info is in German -->
<doc xml:lang="en" info="Information fuer den Uebersetzer">
...
<its:langRule  selector="/doc/@info" langPointer="'de'"/>
</doc>
(note that this usage is not 100% the intention of its:langRule, which 
is meant to point to existing values which are represented not via 
xml:lang, and pointing to them via langPointer . However, using the 
XPath expression 'de' as the value of langPointer will do the job for you.

Felix

---

From: Martin Duerst <[email-masked]>
Date: Mon, 12 May 2008 18:36:32 +0900

I have marked this contribution off-topic, because
this list is about LTRU, not XML.
I hope we can reduce the flow of data on the list
a bit, so that a bit more people (including the chairs :-()
have a chance to follow.

Regards,    Martin.

>This isn't formally a problem with XML, since you can always design a format that bundles together multiple xml:langs as applying to some text, and you also don't have to use xml:lang to mark text:…

It would be less clumsy for those very very rare cases where you really
think you need this. It would make things more clumsy for all other
cases, the vast majority.

xml:lang is very convenient in many cases. It is in particular convenient
if you use XML the way it is intended, i.e. for hierarchical structures,
with markup where needed (e.g. for language changes), without natural
text in attributes,... XML is less appropriate for data that cannot
be described hierarchically (there are for example quite a few approaches
to encoding parallel texts in XML, none optimal), or where you aren't
sure about the structure of the data or the significance of certain
pieces.

In more concrete terms, we have been talking about cases where e.g.
you don't know whether a snippet of text is French or English.
Now assume that you have an XML application that allows you to mark
up something as an emphasis or as a variable name (HTML <em> and <var>).
Assume you have some data where you don't know which to choose.
XML doesn't offer you a <em-or-var> tag (unless you create one on
your own). The fact that xml:lang accepts only one language
therefore seems to be very much in line with much more general
(so general that they are rarely even expressed) design principles
of XML, and therefore perfectly fine.

Regards,    Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]