Subject: RE: [Ltru] Scored Filtering: Expansion of tags and ranges

From: "Addison Phillips" <[email-masked]>
Date: Tue, 6 Dec 2005 09:37:32 -0800

See sample implementation at http://www.inter-locale.com/langtags/scored

The problem here is that wildcard expansion produces differently scored results, depending on what one does. Since the whole point of scored matching is to produce an ordered list, it makes sense that if the tags and ranges are not identical then the score is not 0. Yet, at the same time, users may find that certain combinations violate their sense of what is right in matching.

Using the draft-06 matching produces, for me, the least desirable results. Using the opposite expansion produces better results and no expansion produces the widest range of different scores altogether. The question is whether some items should score "0" (or at least a smaller amount) if a field is omitted.

In other words, is range "de-DE" equivalent to tag "de-Brai-DE"? Or is it just a "pretty good" match?

Addison

Addison P. Phillips
Globalization Architect, Quest Software

Internationalization is not a feature.
It is an architecture. 

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Martin Duerst
> Sent: Tuesday, December 06, 2005 1:01 AM
> To: LTRU Working Group
> Subject: [Ltru] Scored Filtering: Expansion of tags and ranges
> 
> Why do we need to make distinctions between ranges and tags,
> and between internal and trailing components in ranges? It
> seems it would all be the same if every missing piece (after
> suppress-script expansion, see separate mail) were converted
> into a '*'. And it would simplify the explanation considerably.
> 
> Thinking about it a bit more, there are examples where this
> wouldn't work. Take
> 
> tag: de-DE-1996, expanded: (de, *, DE, 1996)
> range: de-DE, expanded: (de, *, DE, "")
> 
> For the last field, neither are they identical, nor is
> one of them "*", so we would get a mismatch, with distance 1.
> It's very unclear to me why we should get 1 here, and 0
> in the reverse case. In filtering, shorter ranges are supposed
> to match longer tags, not the other way round.
> 
> So I think this should be changed to work the other way round.
> 
> Regards,   Martin.
> 
> 
> _______________________________________________
> Ltru mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ltru
_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "John.Cowan" <[email-masked]>
Date: Tue, 6 Dec 2005 12:57:34 -0500

> In other words, is range "de-DE" equivalent to tag "de-Brai-DE"? Or is it just a "pretty good" match?

Since you are not allowed to say "de-Latn-DE" thanks to the Suppress-script
setting, we have to interpret "de-DE" as *meaning* "de-Latn-DE", in which
case "de-Brai-DE" is a match in language only, same as "az-Cyrl" vs. "az-Arab".

-- 
John Cowan  www.reutershealth.com  www.ccil.org/~cowan  [email-masked]
The Penguin shall hunt and devour all that is crufty, gnarly and
bogacious; all code which wriggles like spaghetti, or is infested with
blighting creatures, or is bound by grave and perilous Licences shall it
capture.  And in capturing shall it replicate, and in replicating shall
it document, and in documentation shall it bring freedom, serenity and
most cool froodiness to the earth and all who code therein.  --Gospel of Tux