Subject: RE: [Ltru] filtering and extended filtering...

From: "McDonald, Ira" <[email-masked]>
Date: Wed, 15 Feb 2006 14:35:51 -0800

Hi,

+1

A nice solution to avoid Registry access need for the corner
cases of Suppress-Script.  Because a missing script could be
either due to Suppress-Script or simply legacy document is
just plain ambiguous.

Cheers,
- Ira

Ira McDonald (Musician / Software Architect)
Blue Roof Music / High North Inc
PO Box 221  Grand Marais, MI  49839
phone: +1-906-494-2434
email: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of
> Addison Phillips
> Sent: Wednesday, February 15, 2006 12:57 PM
> To: [email-masked]
> Subject: [Ltru] filtering and extended filtering...
> 
> 
> I have followed the current debate with some interest, but 
> this is the first time I've been able to craft a complete 
> response before more messages have rendered it obsolete.
> 
> I have a few observations.
> 
> 1. Frank is correct that the "*" is redundant in all but the 
> first position in an extended language range (since "missing" 
> or unspecified subtags expand to star). I think we ought to 
> remove them because it simplifies implementation and 
> interoperability with basic ranges.
> 
> 2. There seems to be an assumption that extended filtering 
> should require registry access. This is based on the thought 
> that a range such as "en-Latn" should match all "en-*" 
> content that has obeyed the "suppression" of the 'Latn' 
> subtag. This is, I think, wrong for three reasons:
> 
>    i. There is no other reason to require registry access or 
> a validating processor in order to do extended filtering.
> 
>    ii. Operations in which the user specifies something 
> should produce recognizable matches. If I wish to select all 
> content in my document that is labeled with the Latin script 
> subtag (perhaps I am going through the content to remove the 
> subtag!), the matching set from the filter should look 
> maximally like my range.
> 
>    iii. The use cases for extended filtering does not include 
> language negotiation (see below).
> 
> 3. The only valid use cases for extended filtering deal with 
> selecting content in which the user is interested in specific 
> subtag values, not with the semantic meaning of the resulting 
> tag. The example I use most often is applying fonts to 
> Chinese texts in CSS:
> 
>    :lang(*-Hant)  # use a trad font
>    :lang(*-Hans)  # use a simpl font
> 
> I believe that all examples of this sort produce sets of 
> content that are not required to be (and usually actually are 
> not) linguistically related or mutually understandable at 
> all. This suggests that the extended filtering is mostly to 
> do with document or content processing. Thus, we might wish 
> to do selections like "*-CH" (find all the content that is 
> purported to be in a Swiss dialect) or "de-1996" (find any 
> content that is labeled as using the 1996 orthography, 
> regardless of script or region). Extended filtering is simply 
> a tool used by higher level protocols to select specific 
> language tags based on their attributes and not directly on 
> the language itself.
> 
> 4. Frank's points about variants, while technically correct, 
> I don't believe are cause for worry in practice. Variants can 
> appear in unlimited multiples and they can appear in any 
> order in theory. However, variants *do* have Prefix fields 
> and the text of 3066bis is quite clear about when variants 
> may be used together. To wit:
> 
> --
> Most variants that share a prefix are mutually exclusive. For 
> example, the German orthographic variations '1996' and '1901' 
> SHOULD NOT be used in the same tag, as they represent the 
> dates of different spelling reforms. A variant that can 
> meaningfully be used in combination with another variant 
> SHOULD include a 'Prefix' field in its registry record that 
> lists that other variant. For example, if another German 
> variant 'example' were created that made sense to use with 
> '1996', then 'example' should include two Prefix fields: "de" 
> and "de-1996".
> --
> 
> I would suggest that very few if any variants will ever be 
> used *correctly* together in any sequence and I'd be willing 
> to wager that we won't seen a "correct" three-variant 
> sequence at any time in the foreseeable future. 
> 
> 5. I wrote the original extended filtering grammar based on 
> the idea that the wildcard must come at the end of a subtag 
> type. This complicates the code for variants quite a bit (if 
> the variant is expressed in the range, you have to check that 
> no variant subtags appear before that subtag). Given the use 
> cases above, I think now that this is actually wrong. 
> Extended range filtering should work like regexp, I think: 
> there is a match if and only if all subtags in the range 
> appear (in the same order) in the tag. But all tags that 
> contain the same subtags in the same order match. 
> Implementation of this scheme is easy and requires *no* 
> access to the registry at all. It *also* means we can get rid 
> of the complicated ABNF and move to:
> 
> Ext-range = ("*" / Subtag) *["-" Subtag]
> Subtag = 1*8alphanum
> 
> An implementation of this is *very* simple, simple enough to 
> explain to actual users, which I think is also a benefit.
> 
> So I've written some opinions down. I think my main proposals 
> would be:
> 
> A. Include some text in draft-matching pointing out that an 
> explicit script subtag in the range does not match a missing 
> implied one in the tag and that the reason why is so that 
> users can select content that has the subtag visible.
> 
> B. Change the text and grammar to suppress wildcards except 
> in the initial position.
> 
> C. Spell out usage scenarios for extended filtering.
> 
> D. Use the simple ABNF and treat extended filtering as a form 
> of regexp.
> 
> Addison
> 
> Addison Phillips
> Internationalization Architect - Yahoo! Inc.
> 
> Internationalization is an architecture.
> It is not a feature. 
> 
> 
> _______________________________________________
> Ltru mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ltru
> 

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: [email-masked]
Date: Wed, 15 Feb 2006 15:10:07 -0800

Or the missing script could be because it's entirely inappropriate.  ;)

I do not allow Traditional or Simplified Mandarin in lists used to 
identify spoken languages in audiovisual content. In spoken contexts, I 
allow classification of content as Mandarin or Cantonese, but an audio 
track cannot be in Traditional Mandarin. That's bad data.

My two cents,

Karen Broome
Metadata Systems Designer
Sony Pictures Entertainment
310.244.4384

"McDonald, Ira" <[email-masked]> 
Sent by: [email-masked]
02/15/2006 02:35 PM

To
"'Addison Phillips'" <[email-masked]>, [email-masked]
cc

Subject
RE: [Ltru] filtering and extended filtering...

Hi,

+1

Cheers,
- Ira