Subject: RE: [Ltru] scored filtering and supress script

From: "Addison Phillips" <[email-masked]>
Date: Wed, 7 Dec 2005 09:08:46 -0800

There is the problem of text here. I propose the following:

<t>For language tags that have no script subtag but whose language subtag's record in the IANA Language Subtag Registry contains the field "Suppress-Script", the script element in the quintuple MUST be set to the script subtag in the Suppress-Script field. This is necessary because <xref target="RFC3066bis"></xref> strongly recommends that users not use this subtag to form language tags and this document recommends that users not use them to form ranges. For example, if the script were not expanded in this manner, a range such as "de-DE" would produce a more-distant score for content that happened to be labeled "de-Latn-DE" than users would expect that it should. Note that languages which have a "Suppress-Script" field in the registry are predominantly written in a single script.</t>

I also modified the following sentence to being with "Any remaining missing components..."

Addison P. Phillips
Globalization Architect, Quest Software

Internationalization is not a feature.
It is an architecture. 

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Martin Duerst
> Sent: Tuesday, December 06, 2005 10:34 PM
> To: Harald Tveit Alvestrand; LTRU Working Group
> Subject: Re: [Ltru] scored filtering and supress script
> 
> At 21:53 05/12/06, Harald Tveit Alvestrand wrote:
>  >
>  >
>  >--On tirsdag, desember 06, 2005 17:39:58 +0900 Martin Duerst
> <[email-masked]> wrote:
>  >
>  >> [co-chair hat off]
>  >> This is a proposal that both Doug and I liked, and that
>  >> I don't understand why it shouldn't work, so I'm bringing
>  >> it up again here. Doug also pointed out that we need
>  >> have the necessary information anyway.
>  >>
>  >> For scored filtering, I propose to exand both ranges
>  >> and tags with suppress-script. This means that
>  >> de-DE becomes de-Latn-DE, and thus doesn't match
>  >> de-Brai-DE, in contrast to de-*-DE, which of course
>  >> would match de-Brai-DE.
>  >
>  >This of course demands access to the registry, and the results will vary
> based on which date the registry was incorporated/consulted.
>  >But since this is already implicit in section 3.2.3:
> 
> Yes, exactly. This is what I meant when I wrote "Doug also pointed
> out that we need have the necessary information anyway." above.
> 
>  >   First both the extended language range and the language tags to be
>  >   matched to it must be canonicalized by mapping grandfathered and
>  >   obsolete tags into modern equivalents.
>  >
>  >I see this as a reasonable thing to do.
>  >Note: draft-ietf-ltru-initial-06 has 125 "Suppress-Script" tags, and 481
> entries with "Type: Language". It's possible that there'll be a number of
> updates requested if this proves popular and useful.
> 
> Yes indeed. There are a lot of languages that in practice are written
> only or highly predominantly with one script, but do not yet have the
> appropriate "Suppress-Script" information.
> 
> Regards,    Martin.
> 
> 
> _______________________________________________
> Ltru mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ltru
_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: Martin Duerst <[email-masked]>
Date: Thu, 08 Dec 2005 14:57:16 +0900

Great, thanks. I'll comment on any details (if I find some) in the
next round.                    Regards,    Martin.

At 02:08 05/12/08, Addison Phillips wrote:
 ><t>For language tags that have no script subtag but whose language subtag's
 >record in the IANA Language Subtag Registry contains the field
 >"Suppress-Script", the script element in the quintuple MUST be set to the
 >script subtag in the Suppress-Script field. This is necessary because <xref
 >target="RFC3066bis"></xref> strongly recommends that users not use this
 >subtag to form language tags and this document recommends that users not
 >use them to form ranges. For example, if the script were not expanded in
 >this manner, a range such as "de-DE" would produce a more-distant score for
 >content that happened to be labeled "de-Latn-DE" than users would expect
 >that it should. Note that languages which have a "Suppress-Script" field in
 >the registry are predominantly written in a single script.</t>
 >