Subject: RE: [Ltru] Language Ranges

From: "Addison Phillips" <[email-masked]>
Date: Thu, 29 Sep 2005 11:30:14 -0700

Mark,

My first language is almost never English. In fact, my list often doesn't include English. My list this morning (in Mozilla) does; it is: fr-fr, ja, sr-Cyrl, tlh-AQ, en.

English often comes up for me anyway because sites either don't do language negotiation or because that is their default language. I keep a long list lying around so I can just move up a value to see a page in a particular language quickly.

> If we are not talking about "specific mechanics of what an
> implementation might wish to consider", why do we have basic vs
> extended? Why don't we just document basic, and have an informative note
> to say that there are other mechanisms?

We are talking about the specific mechanics of matching tags, not the specific mechanics of implementing a content selection system. What content is returned for my peculiar Accept-Language is often amusing, but is usually logical and implementation specific... and based on the first entry. I'm not saying this is always true, especially since you apparently have counter examples in mind and since clearly that was the intent of Accept-Language in the first place.

I just note that I rarely, if ever, see French and Japanese mixed together, despite their (current) proximity at the top of my list.

Anyway, this is shouting at the forest fire. You're right that there are real use cases. I just want to draw a distinction between tag matching and content negotiation schemes... and point out that these are historically in separate RFCs which need modification anyway. It's not a big deal to any material about matching lists.

Addison

Addison P. Phillips
Globalization Architect, Quest Software
Chair, W3C Internationalization Core Working Group

Internationalization is not a feature.
It is an architecture. 

> -----Original Message-----
> From: Mark Davis [mailto:[email-masked]]
> Sent: jeudi 29 septembre 2005 11:08
> To: Addison Phillips
> Cc: [email-masked]; Deborah Goldsmith
> Subject: Re: [Ltru] Language Ranges
> 
> 
> 
> Addison Phillips wrote:
> 
> >There is a far difference between "looks at" and "uses". The fact that
> there is a "stack" of values doesn't mean that implementations go below
> the first item on the list! I too have a large list of languages in my
> browser preferences and cannot recall ever seeing a website return
> something other than the first language or a default language if that
> wasn't available.
> >
> >Struts, Weblogic, J2EE, etc. provide a mechanism for retrieving the list,
> but if one takes the first locale of the list and retrieves resources
> using it, one gets a result (even if it is the default language). The
> second language is rarely if ever queried in my experience. I'm not saying
> that you don't have a valid use case or that this isn't a real issue. Only
> that it is rare in my experience for this to have a meaningful
> implementation.
> >
> >
> That is because your first language is probably English. Try it with
> <Sami, English>  or <Finnish, English> and see how often your second
> item comes up!
> 
> >The real question in my mind is: how is your proposal any different than
> Accept-Language? Q-weights are used, as you say, to order the list. An
> ordered list, no matter what the separator, is an ordered list. Why
> pioneer a different way to express ordered lists *in this document*? I
> think that the text you've proposed wanders too far into the specific
> mechanics of what an implementation might wish to consider. A revision of
> RFC 3282 would be appropriate (to reference basic and extended language
> ranges, if nothing else) following this document on matching and this
> would be the place to deal with ordered lists.
> >
> >
> If we are not talking about "specific mechanics of what an
> implementation might wish to consider", why do we have basic vs
> extended? Why don't we just document basic, and have an informative note
> to say that there are other mechanisms?
> 
> >I also think that mentioning ordered lists is useful in this document (in
> fact, I just proposed in another thread to mention it!) but a document
> about matching schemes shouldn't necessarily say how an implementation
> forms its preference lists or weights the values for comparison purposes.
> >
> >
> I don't think we are talking about a lot of text here, in any event?
> 
> >Addison
> >
> >Addison P. Phillips
> >Globalization Architect, Quest Software
> >Chair, W3C Internationalization Core Working Group
> >
> >Internationalization is not a feature.
> >It is an architecture.
> >
> >
> >
> >>-----Original Message-----
> >>From: Mark Davis [mailto:[email-masked]]
> >>Sent: jeudi 29 septembre 2005 09:18
> >>To: Addison Phillips
> >>Cc: [email-masked]; Deborah Goldsmith
> >>Subject: Re: [Ltru] Language Ranges
> >>
> >> > In practice, the second and subsequent languages in a list are mostly
> >>useless and thus ignored ...
> >>
> >>I disagree. I have my browser set for the languages I can read, and it
> >>picks up my preferred secondary and tertiary choices for many web sites.
> >>Websphere certainly looks at second and subsequent languages; so does
> >>Struts, and many other technologies. Systems like Apple's set up a list
> of
> >>languages to lookup when getting localizations; that is implemented and
> >>works well (Deborah might be able to add more detail). We have found it
> >>necessary to add list lookup to CLDR to handle minority languages such
> as
> >>Sami.
> >>
> >>This is not an edge case, and should be described in the matching
> document.
> >>I agree (as in my previous message) that Accept-Language should be there
> >>as well (although it can be mostly a pointer). But as I pointed out,
> >>Accept-Language doesn't really cover all the bases alone, just as Basic
> >>and Extended in the current document are not each alone sufficient.
> >>
> >>Mark
> >>
> >>
> >>Addison Phillips wrote:
> >>
> >>
> >>
> >>>No, I think not. Here's why: RFC 3282 (Content-Language headers)
> defines
> >>>
> >>>
> >>the most common scheme for "language range lists" and it is semi-colon
> >>separated and has the whole q-weight thing.
> >>
> >>
> >>>In practice, the second and subsequent languages in a list are mostly
> >>>
> >>>
> >>useless and thus ignored (in part because current "basic" matching
> schemes
> >>are binary matching schemes and one tends never to get to the next
> >>language before the default content is located).
> >>
> >>
> >>>This is especially true of "lookup" (fallback, locale-style), as
> compared
> >>>
> >>>
> >>to the "matching" (negotiated language style), since generally a mix of
> >>language resources is viewed as a bug or renders the content only semi-
> >>intelligible. There may be cases in which the second or subsequent
> >>language is useful (as when ranking results by language in order of
> >>preference), but these kinds of applications strike me as far into the
> >>application domain and not so much something we should be specifying
> here.
> >>
> >>
> >>>I do think that the message that Jefsey quoted the other day might make
> a
> >>>
> >>>
> >>useful example to include somewhere when discussing scored matching:
> >>scored matching is primarily a tool for ranking results (and not just
> >>identifying a matching set). Combining scores with an ordered list of
> >>ranges is a useful technique for ordering data according to a user's
> >>preference.
> >>
> >>
> >>>But I also happen to think that such cases really shouldn't be part of
> >>>
> >>>
> >>the normative structure of the document. Once we've described matching
> >>(including, perhaps, better criteria for selecting a matching scheme) I
> >>think that the specific application will already be constrained enough
> to
> >>produce consistent results and we should leave some room for different
> >>applications to handle matching and thus preferences in a manner most
> >>useful to that application. Language range lists are exactly such a
> detail,
> >>IMO.
> >>
> >>
> >>>Note that if we were to include a section such as you describe, I would
> >>>
> >>>
> >>favor cloning Accept-Language (semicolon separated with q weights) out
> of
> >>respect for existing practice and a certain parsimony in wanting to
> create
> >>more protocols when a handy one is already available. But I note that
> >>RFC3282 already documents what you're looking for and there is no reason
> >>to change it in light of RFC 3066bis (except maybe to reference Basic
> >>Language Range or Extended Language Range in this document).
> >>
> >>
> >>>Addison
> >>>
> >>>Addison P. Phillips
> >>>Globalization Architect, Quest Software
> >>>Chair, W3C Internationalization Core Working Group
> >>>
> >>>Internationalization is not a feature.
> >>>It is an architecture.
> >>>
> >>>
> >>>
> >>>
> >>>>-----Original Message-----
> >>>>From: [email-masked] [mailto:[email-masked]] On Behalf
> Of
> >>>>Mark Davis
> >>>>Sent: mercredi 28 septembre 2005 12:32
> >>>>To: Mark Davis
> >>>>Cc: [email-masked]
> >>>>Subject: [Ltru] Language Ranges
> >>>>
> >>>>The main issue I have with the document as it stands is that it
> neglects
> >>>>
> >>>>
> >>a
> >>
> >>
> >>>>very important case. The general case of language matching involves
> >>>>
> >>>>
> >>having
> >>
> >>
> >>>>a *list* of language ranges. You take a list like "en-US en fr it" and
> >>>>
> >>>>
> >>use
> >>
> >>
> >>>>it for either matching or lookup. So I would suggest adding the
> >>>>
> >>>>
> >>following
> >>
> >>
> >>>>text. (It probably needs fleshing out a bit, but this is a start.).
> >>>>
> >>>>
> >>>>2.3 Language Range Lists
> >>>>
> >>>>A language range list is a space-delimited list of zero or more
> language
> >>>>ranges. An extended language range list contains extended language
> >>>>
> >>>>
> >>ranges;
> >>
> >>
> >>>>a basic language range list is limited to containing basic language
> >>>>
> >>>>
> >>ranges.
> >>
> >>
> >>>>In matching, anything that matches any element of the list according
> to
> >>>>the above options is included. (With distance metrics, the shortest
> >>>>distance from any element in the list is the one used for the overall
> >>>>distance).
> >>>>
> >>>>In lookup, the list is examined from first to last for a match,
> >>>>
> >>>>
> >>according
> >>
> >>
> >>>>to the above options. For example, starting with the list range "zh-
> >>>>
> >>>>
> >>Hant-
> >>
> >>
> >>>>CN en-US fr", and using Lookup with truncation as in 2.1.1, the lookup
> >>>>would progressively search for content as shown below:
> >>>>
> >>>>List Range to match: zh-Hant-CN en-US fr
> >>>>1. zh-Hant-CN
> >>>>2. zh-Hant
> >>>>3. zh
> >>>>4. en-US
> >>>>5. en
> >>>>6. fr
> >>>>7. (default content or the empty tag)
> >>>>
> >>>>
> >>>>
> >>>>2.3 Meaning of Language Tags and Ranges [RENUMBER to 2.4...]
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >
> >
> >

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru