Subject: [Ltru] wildcards in lookup

From: "Addison Phillips" <[email-masked]>
Date: Mon, 5 Dec 2005 09:26:35 -0800

Separately I sent an email responding to various small items Mark/Martin suggested I change. This item, though, I think needs its own thread.

Mark suggested:

=====

    Wildcard values in an extended language range are supposed to match
    any value that occurs in that position in a language tag. Since only
    one item can be returned for any given lookup request, the wildcards
    must be processed in a predictable manner (or the same request might
    produce widely varying results). Thus, for each range in the
    language priority list, the following rules must be applied to
    produce a basic language range for use in the fallback mechanism

    ...*
    Figure 10: Extended Language Range Fallback Example *

=>

    Wildcard values in an extended language range are supposed to match
    any value that occurs in that position in a language tag. Since only
    one item can be returned for any given lookup request, the wildcards
    must be processed in a predictable manner (or the same request might
    produce widely varying results). Where more than one piece of
    content would be returned, then the implementation MUST specify the
    method used to determine the result.
    [The method supplied in the document doesn't match reasonable user
    expections, as stated earlier. Since we can't agree, leave it up to
    the implemention.]

=====

And later Martin opined:

> 
> >
> > >    Where more than one piece of
> > >    content would be returned, then the implementation MUST specify the
> > >    method used to determine the result.
> >
> > 'Where', 'would', and 'then' don't make a reasonable sentence for me.
> > More general, the paragraph starts with "Since only one item can be
> > returned", so then speaking about more than one piece is confusing.
> 
> How about:
> 
> Where the use of a wildcard would cause matches with two different
> pieces of content, the implementation MUST specify the method used to
> choose between them. For example, the implementation could specify that
> the content tagged with the alphabetically-least language tag is returned.
[Addison Phillips] 

Mark and I don't agree about this and I dislike Martin's suggestion even more. Here's the problem: when I implemented a lookup scheme I didn't seek all of the possible matching content first. I used the tag to construct directory or file names, truncating until I found a matching bit of content.

When I consider Java's ResourceBundle class (which finds all matching content building out from root to the specified value before, returning content when the lookup fails to find a file), I see that the algorithm would have to change appreciably. Notably, it would have to load some content by name, with no way of knowing what that name actually was. 

That is, most implementations today actually would be forced to do what I propose: they rely on knowing the name tokens. A wild card works better if it matches nothing in that case.

I think it is problematic not to say anything about what to do here. We know clearly what the problem is. Is there some reason why we should not specify at least one mechanism (we do not have to make it normative, as it is now)?

To illustrate a bit more, if the range is "en-*-US", an implementation can either:

1. Try to enumerate all matching content. If content is represented as directory or file names, the complete list of file/directory names must be loaded from one or more locations on disk and then processed to find all of the matches. These can then be selected based on some algorithm.

2. Work as lookup has historically worked, by searching until there is a "miss" and return that item. If one works out from root, then the tree must be fully populated. 

So I don't favor making this change or I would prefer to make changes that spell some of this out.

Addison

Addison P. Phillips
Globalization Architect, Quest Software
http://www.quest.com

Internationalization is not a feature.
It is an architecture. 

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: Mark Davis <[email-masked]>
Date: Mon, 05 Dec 2005 09:59:21 -0800

The problem is that your method subverts all user expectations about 
what wildcards mean. If I supply the range "*-US", and the possible 
content is

es
es-MX
en
en-US
fr
fr-FR
fr-BE
[i-default]

then en-US is the obvious, clear choice. That is, after all, what is 
being asked for: a language tag consisting of language code = any + 
region code = 'US'.

The same goes for where the range is "en-*". Given the above tagged 
content, it would be bizarre not to return "en-US".

The implementation, for any system I know of, is not rocket science. 
Logically, you just enumerate through the available tags for the content 
you have, and see which (if any) matches.

The only problematic case is the one where the range matches 2 or more 
language tags. In that case, I don't think we have any choice but to 
leave this implementation dependent. (Actually, we do; my preference 
would be to have a fully specified specification that used 
alphabetically-least, with the proviso that implementations are free to 
customize this area at will. But I can live with the incomplete 
specification if you really want it.)

Mark

Addison Phillips wrote:

>------------------------------------------------------------------------
>