Subject: RE: Suppressed and non-suppressed scripts (was: Re: [Ltru]Comments on draft-matching-07)

From: "Addison Phillips" <[email-masked]>
Date: Mon, 21 Nov 2005 10:05:09 -0800

Here I'll expand on the remarks buried in my response to Doug.

If you look at my little implementation, which seems to be relatively bug-free, you'll see that expanding wildcards produces some odd-seeming results. They are consistent with the text of the draft. But I would view them as odd as a user.

Ideally, I think, a distance metric should produce as many different levels of score as possible. That is "en-Brai-US" should have a lower score than "en-US" for the range "en-US"... but a higher score than, say, "fr" or "en-GB". It seems more sensible.

Thus I propose that we remove wildcard expansion from scored filtering unless the wildcard is explicitly in the range. Thus "en-*-US" matches "en-Brai-US" with a score of zero, while "en-US" matches it with a score of 128.

Consider this result set for en-US:

- en-US (0)
- en-US-x-foo (1)
- en-US-boont (4) // fictional
- en (32)
- en-GB (32)
- en-Latn-GB (36)
- en-Brai-US (128)
- en-Brai (160)
- fr-US (256)
- fr-FR (288)
- fr-Brai (416)

Compare this to the wildcard expanded list:

- en-US (0)
- en-US-x-foo (0)
- en-US-boont (0)
- en-Brai-US (0)
- en (32)
- en-GB (32)
- en-Latn-GB (32)
- en-Brai (32)
- fr-US (256)
- fr-Brai (256)
- fr-FR (288)

Hmm... that list isn't as useful. I understand that the values are consistent with the algorithm, but if the point is that the list is ordered, it makes sense that the "missing" values in the range are scored. Now consider this result with en-*-US:

- en-US (0)
- en-Brai-US (0)
- en-US-x-foo (1)
- en-US-boont (4)
- en (32)
- en-GB (32)
- en-Latn-GB (32)
- en-Brai (32)
- fr-US (256)
- fr-FR (288)
- fr-Brai (288)

Addison

Addison P. Phillips
Globalization Architect, Quest Software

Internationalization is not a feature.
It is an architecture. 

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Martin Duerst
> Sent: 2005年11月21日 1:05
> To: Doug Ewell; LTRU Working Group
> Subject: Suppressed and non-suppressed scripts (was: Re: [Ltru]Comments on
> draft-matching-07)
> 
> At 12:45 05/11/21, Doug Ewell wrote:
>  >Here are my comments on draft-matching-07.  Most of these are editorial.
> 
> Thanks, great. I'm picking out one (maybe the only one,
> apart from the fact that in some places, you planned
> to look at the new version of the document anyway)
> issue that doesn't look editorial.
> 
>  >"en-Latn-US"       0 (all fields match)
>  >
>  >-> In reading this, it appears that "en-Brai-US" would also give a
> perfect 0 score, and for that matter so would "en-Egyp-US".  Is that
> correct?  We aren't looking at Suppress-Script here, so that doesn't
> factor
> into anything (unless we are one of the "implementations or protocols"
> that
> customize things).
> 
> I got the same impression. It doesn't look like that would make sense.
> I was thinking that maybe how this should be handled is as follows:
> 
> - In a range, a '*' in script position stays a '*'; an absent
>    script field gets converted into the 'suppress-script' value
>    for that lanugage if such a value is available. Otherwise, this
>    expands into a '*'.
> 
> - The corresponding content tags are also expanded by suppress-script
>    information where available.
> 
> - The matching is done on the resulting expanded tags, with a '*'
>    matching anything.
> 
> This would give the following:
> 
> Range: en-US -> en-Latn-US
> Tag: en-Brai-US -> en-Brai-US
> -> match on language and region, but not on script
> 
> Range: en-*-US -> en-*-US
> Tag: en-Brai-US -> en-Brai-US
> -> match on all fields
> 
> Range: en-US -> en-Latn-US
> Tag: en-US -> en-Latn-US
> -> match on all fields
> 
> I'm rather sure somebody will find a hair in this soup,
> if not some bigger obstacle, but at least for the cases
> above, it would work reasonably as expected.
> 
> 
> Regards,    Martin
> 
> 
> _______________________________________________
> Ltru mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ltru
_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru