Subject: RE: [Ltru] Language Ranges

From: "Addison Phillips" <[email-masked]>
Date: Thu, 29 Sep 2005 07:58:26 -0700

No, I think not. Here's why: RFC 3282 (Content-Language headers) defines the most common scheme for "language range lists" and it is semi-colon separated and has the whole q-weight thing.

In practice, the second and subsequent languages in a list are mostly useless and thus ignored (in part because current "basic" matching schemes are binary matching schemes and one tends never to get to the next language before the default content is located). 

This is especially true of "lookup" (fallback, locale-style), as compared to the "matching" (negotiated language style), since generally a mix of language resources is viewed as a bug or renders the content only semi-intelligible. There may be cases in which the second or subsequent language is useful (as when ranking results by language in order of preference), but these kinds of applications strike me as far into the application domain and not so much something we should be specifying here.

I do think that the message that Jefsey quoted the other day might make a useful example to include somewhere when discussing scored matching: scored matching is primarily a tool for ranking results (and not just identifying a matching set). Combining scores with an ordered list of ranges is a useful technique for ordering data according to a user's preference. 

But I also happen to think that such cases really shouldn't be part of the normative structure of the document. Once we've described matching (including, perhaps, better criteria for selecting a matching scheme) I think that the specific application will already be constrained enough to produce consistent results and we should leave some room for different applications to handle matching and thus preferences in a manner most useful to that application. Language range lists are exactly such a detail, IMO.

Note that if we were to include a section such as you describe, I would favor cloning Accept-Language (semicolon separated with q weights) out of respect for existing practice and a certain parsimony in wanting to create more protocols when a handy one is already available. But I note that RFC3282 already documents what you're looking for and there is no reason to change it in light of RFC 3066bis (except maybe to reference Basic Language Range or Extended Language Range in this document).

Addison

Addison P. Phillips
Globalization Architect, Quest Software
Chair, W3C Internationalization Core Working Group

Internationalization is not a feature.
It is an architecture. 
> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Mark Davis
> Sent: mercredi 28 septembre 2005 12:32
> To: Mark Davis
> Cc: [email-masked]
> Subject: [Ltru] Language Ranges
> 
> The main issue I have with the document as it stands is that it neglects a
> very important case. The general case of language matching involves having
> a *list* of language ranges. You take a list like "en-US en fr it" and use
> it for either matching or lookup. So I would suggest adding the following
> text. (It probably needs fleshing out a bit, but this is a start.).
> 
> 
> 2.3 Language Range Lists
> 
> A language range list is a space-delimited list of zero or more language
> ranges. An extended language range list contains extended language ranges;
> a basic language range list is limited to containing basic language ranges.
> In matching, anything that matches any element of the list according to
> the above options is included. (With distance metrics, the shortest
> distance from any element in the list is the one used for the overall
> distance).
> 
> In lookup, the list is examined from first to last for a match, according
> to the above options. For example, starting with the list range "zh-Hant-
> CN en-US fr", and using Lookup with truncation as in 2.1.1, the lookup
> would progressively search for content as shown below:
> 
> List Range to match: zh-Hant-CN en-US fr
> 1. zh-Hant-CN
> 2. zh-Hant
> 3. zh
> 4. en-US
> 5. en
> 6. fr
> 7. (default content or the empty tag)
> 
> 
> 
> 2.3 Meaning of Language Tags and Ranges [RENUMBER to 2.4...]
> 

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: Mark Davis <[email-masked]>
Date: Thu, 29 Sep 2005 09:17:34 -0700

> In practice, the second and subsequent languages in a list are mostly 
useless and thus ignored ...

I disagree. I have my browser set for the languages I can read, and it picks up my preferred secondary and tertiary choices for many web sites. Websphere certainly looks at second and subsequent languages; so does Struts, and many other technologies. Systems like Apple's set up a list of languages to lookup when getting localizations; that is implemented and works well (Deborah might be able to add more detail). We have found it necessary to add list lookup to CLDR to handle minority languages such as Sami.

This is not an edge case, and should be described in the matching document. I agree (as in my previous message) that Accept-Language should be there as well (although it can be mostly a pointer). But as I pointed out, Accept-Language doesn't really cover all the bases alone, just as Basic and Extended in the current document are not each alone sufficient.

Mark

---

From: r&d afrac <[email-masked]>
Date: Thu, 29 Sep 2005 18:25:15 +0200

At 16:58 29/09/2005, Addison Phillips wrote:
>I do think that the message that Jefsey quoted the other day might 
>make a useful example to include somewhere when discussing scored 
>matching: scored matching is primarily a tool for ranking results 
>(and not just identifying a matching set). Combining scores with an 
>ordered list of ranges is a useful technique for ordering data 
>according to a user's preference.

I am glad to see that we can agree once the ABNF issue is put aside. 
In this I suggest you keep scored matching totally independent from 
format, protocols, etc. I think in particular (see preceding mail) 
that OPES are a great place for that because they are front-end to 
the users, can be organised in community and adapted to each user. 
For example an OPES has the capacity to change the IP address in a 
packet header or a name in a DNS request depending on the language 
score between the caller and the available hosts. But there are other 
place for the scoring (or the OPES) to put in as a front-end for 
applications, and pick for example the locale matching the scoring. I 
am sure you worked on that.

This is why I would suggest that you consider a separate RFC on this 
issue which is common to all the filters, format, etc. and protocols. 
The advantage would be that every application could use the 
same/consistent filtering and you keep the same language when 
switching applications.

jfc

PS. OPES means Open Pluggable Edge Services. "edge" is for network 
edge. It is not used with the same meaning as Mark's remark in his 
mail. Actually one of the difficulties I find in the OPES concept is 
that "edge" is ill defined. For historical reasons and "end to end" 
process, it is understood as "on the proxy". But if you consider it 
from the computer, it is also on the computer and can serve 
computer/computer links (for example looking for a dictionnary, 
searching a cache, etc.)