Subject: RE: [Ltru] Wildcards in filtering

From: "Addison Phillips" <[email-masked]>
Date: Mon, 21 Nov 2005 09:04:26 -0800

> When something has that big a hole in it, then we are then probably best
> off saying that *anything* with wildcards in lookup is
> implementation-defined, and not trying to specify it any further than
> that. (That has no impact on the use of wildcards in filtering, of course.)

Why not go with what we have and provide a MAY escape clause at the end? The current version of lookup is determinate and simple. Then, at the end of Section 3.3 we could add this text:

--
Applications or protocols MAY provide for expansion and matching of wildcards during lookup. Expanding wildcards in the lookup process can result in multiple pieces of content being an equally good match on any particular "level" of the fallback process. An implementation SHOULD specify how the content is chosen (even if it is random, first content found, or some other non-determinant mechanism).
--

We don't have to specify everything in here. I'd go further and say we need some running code to evaluate the results before we dabble in "extended range lookup" too far.

Addison

Addison P. Phillips
Globalization Architect, Quest Software

Internationalization is not a feature.
It is an architecture. 

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: Mark Davis <[email-masked]>
Date: Mon, 21 Nov 2005 09:39:48 -0800

I'm ok with that IF we take out the mechanisms for resolving wildcards 
in lookup. That is because of the problem with the current formulation 
that I pointed out. The problematic language is the following, and needs 
to be replaced, one way or another:

>
> The language range "*" matches any language tag. In the lookup scheme, 
> this language range does not convey enough information to determine 
> which content is most appropriate. If this language range is the only 
> one in the language priority list, it matches the default content. If 
> this language range is followed by other language ranges, it should be 
> skipped.

and

> In some cases, the language priority list might contain one or more 
> extended language ranges (as, for example, when the same language 
> priority list is used as input for both lookup and filtering 
> operations). Wildcard values in an extended language range are 
> supposed to match any value that occurs in that position in a language 
> tag. Since only one item can be returned for any given lookup request, 
> the wildcards must be processed in a predictable manner (or the same 
> request might produce widely varying results). Thus, for each range in 
> the language priority list, the following rules must be applied to 
> produce a basic language range for use in the fallback mechanism:
>
>    1. If the first subtag in the extended language range is a "*" then 
> entire range is converted to "*".
>    2. For each subsequent subtag, if the value is a "*" then that 
> subtag and its preceeding hyphen are removed.
>
> For example:
>
> *-US      becomes  *
> en-*-US   becomes  en-US
> en-Latn-* becomes  en-Latn
>
>  Figure 8: Transformation of Extended Language Ranges
>
> For the language priority list "*-US; fr-*-FR; zh-Hant", the fallback 
> pattern would be:
>
> 1. * (skipped)
> 2. fr-FR
> 3. fr
> 4. zh-Hant
> 5. zh
> 6. (default content)
>
>  Figure 9: Extended Language Range Fallback Example

 ===========================

Repeating my previous message, but with the language cleaned up:

Here is the problem I have with the current formulation of "*" in 
lookup. There is a conflict between the user expectations and the need 
for a determinant answer.

A. If a user has "sr-*-BA", and the actual content is one of:

sr
sr-Cyrl
sr-Cyrl-BA
sr-Latn

Then I believe the user's perfectly reasonable expectations would be 
that "sr-Cyrl-BA" would match. That is an exact, single match against 
the pattern.

B. However, if the content is

sr
sr-Cyrl
sr-Cyrl-BA
sr-Latn
sr-Latn-BA

then we can't get a determinant answer from that pattern: both 
sr-Cyrl-BA and sr-Latn-BA are perfectly reasonable matches.

The current draft has the advantage of getting a determinant answer, but 
*not* matching the user expectations. It transforms the sr-*-BA into 
sr-BA, then does the lookup. In both cases A and B above, it'd return 
"sr", which is *not* what the user would expect.

My view is that there are two possible alternatives.

=====

1. Continue the fallback unless there is a *single* match, replacing the 
problematic text with text to the effect that that:

   As usual, a wildcard matches against a subtag in that position. When
   filtering, however, the fallback process returns contents if and
   only if there is exactly one match against the language priority
   list. If there are two possible matches against the wildcard (or
   zero) then the fallback continues. Thus suppose we are filtering
   with the language priority list "sr-Cyrl-BA", and content for the
   following tags:

   sr
   sr-Cyrl
   sr-Cyrl-BA
   sr-Latn

   The result is the content for "sr-Cyrl-BA". When filtering with the
   language priority list "sr-*", and the same possible content, the
   result is the content for "sr".

=====

2. Leave out wildcards entirely, replacing the replacing the problematic 
text with text to the effect that:

   The use of wildcards are not defined in filtered lookup, because they 
allow
   for the possiblity of indeterminant answers. If, for example, the
   language priority list is "sr-*" and the content is the following,
   then the results would be indeterminant.

   sr
   sr-Cyrl
   sr-Cyrl-BA
   sr-Latn

3. Resolve the indeterminancy somewhat arbitrarily (but *not* randomly):

   As usual, a wildcard matches against a subtag in that position.
   If there are two possible matches against the wildcard (or
   zero) then contents for the language tag that is alphabetically first
   (with a caseless compare) is returned. Thus suppose we are filtering
   with the language priority list "sr-Cyrl-BA", and content for the
   following tags:

   sr
   sr-Cyrl
   sr-Cyrl-BA
   sr-Latn

   The result is the content for "sr-Cyrl-BA". When filtering with the
   language priority list "sr-*", and the same possible content, the
   result is the content for "sr-Cyrl", because that is alphabetically
   before the other possibility: "sr-Latn".

====
In *both* cases, add:

   However, a protocol MAY add additional specifications to filtered
   lookup to provide for wildcard matching, or for alternative choices.