Subject: [Ltru] RE: comments on matching-8.txt - markus

From: "Addison Phillips" <[email-masked]>
Date: Mon, 12 Dec 2005 17:04:58 -0800

Comments follow. Thanks again for your comments. As noted, I have copied the list on this message, which I hope you don't mind.

Addison

Addison P. Phillips
Globalization Architect, Quest Software

Internationalization is not a feature.
It is an architecture. 

> -----Original Message-----
> From: Markus Scherer [mailto:[email-masked]]
> Sent: 2005年12月11日 15:38
> To: [email-masked]; Addison Phillips
> Subject: comments on matching-8.txt - markus
> 
> (Not to the list. Feel free to forward to there if you think it's useful.)
> Reading matching-8.txt by invitation from Mark...
> 
> 1. Introduction
> ...Protocols that
>    implement a matching scheme either need to choose a particular option
>    or indicate that the particular options is left to the specific
>    implementation to decide.
> 
> Remove plural 's' from last "options".
[Addison Phillips] 

It really should be edited more thoroughly than that. Actually, I question putting this quasi-normative text into the introduction, but for now am suggesting we change it to:

--
Protocols that implement a matching scheme either need to specify each particular choice or indicate the options that are left to the implementation to decide.
--
> 
> 2.1.  Basic Language Range
> ...The basic
>    language range is defined by the following the ABNF[RFC4234]:
> 
> The last "the" doesn't belong - just remove? replace by "in"? say "grammar
> in"??
> Usage in 2.2 suggests just dropping the "the".
[Addison Phillips] 

It's the result of a poorly implemented edit. Removed the extra article.

> 
> 3.1.  Choosing a Type of Matching
> ... For example, if process were to...
> 
> -> if _a_ process were to...
[Addison Phillips] 

Done.
> 
> 3.2.  Filtering
>    Filtering is used to select the set of content that matches a given
>    language priority list. ...
> 
> There seems to be some arbitrary jumping between language priority
> lists and language ranges here.
[Addison Phillips] 

Done.
> 
> Attempted clarified paragraph for a local fix:
> 
>    Filtering is used to select the set of content that matches a given
>    language priority list.  It is called "filtering" because this set of
>    content may contain no items at all or it may return an arbitrarily
>    large number of matching items--
> as many items as whose language tags match any of the language ranges
> in the language priority list,
> thus filtering out the non-matching
>    content.
[Addison Phillips] 
 
My edit says:

<t>Filtering is used to select the set of content that matches a given language priority list. It is called "filtering" because this set of content may contain no items at all or it may return an arbitrarily large number of matching items: as many items as match the language priority list, thus "filtering out" the non-matching items.</t>

> 
> 3.2.1.  Filtering with Basic Language Ranges
> The special range "*" in a language priority list matches any tag. ...
> 
> ->"*" was not previously introduced as a special range. If should be
> introduced as such (with semantics) in 2.3.
[Addison Phillips] 

It doesn't say it, as you say, early enough. I inserted this paragraph into Section 2.0:

<t>Language ranges are similar in structure and content to language tags: they consist of alphanumeric "subtags" separated by hyphens, plus a special subtag consisting of the character "*" (%2A, ASTERISK), which is used in ranges as a "wildcard", that is, a value that matches any subtag.</t>
> 
> 
> 3.2.2.  Filtering with Extended Language Ranges
>    For example, the extended language range "de-*-DE" matches all of the
>    following tags because the unspecified variant field is expanded to
>    "*":
> 
> -> Remove the "because" - it is incomplete and therefore misleading:
> Not just the variant field (better: subtag) is unspecified and
> expanded, but also every other subtag except language and region.
> (extlang, script, extension, privateuse...)
[Addison Phillips] 

The text, as written, is correct, since the examples all match the explicit wildcard "*" to various subtags and the unspecified variant to a number of variants. No extensions are presented (one private-use example is). 

In any case, I used:

--
For example, the extended language range "de-*-DE" matches all of the following tags, in part because the unspecified variant, extension, and private-use subtags are expanded to "*":
--
> 
> 
> 3.2.3.  Scored Filtering
>    Each extended language range and language tag MUST first be
>    canonicalized by mapping grandfathered and obsolete tags into modern
>    equivalents. ...
> 
> I suggest to add a parallel "should" clause to earlier filtering sections.
[Addison Phillips] 

One might argue that this prevents certain kinds of matching ("I want content labeled 'i-klingon' and NOT content labeled 'tlh'"). It also requires access to the subtag registry, which is something that we want to avoid.

It would be useful to permit it, but SHOULD is stronger than just permission. Instead, perhaps at the end of section 3.1 we could put:

"Regardless of the matching scheme chosen, protocols and implementations MAY canonicalize language tags and ranges by mapping grandfathered and obsolete tags or subtags into modern equivalents. If an implementation canonicalizes..."
> 
>    Language subtags 'und', 'mul', and the script subtag 'Zyyy' are
>    converted to "*": ...
> 
> I believe Mark recently suggested the script code "Zzzz" for use with
> unassigned Unicode code points (i.e., as an "unknown" script). Should
> it be added here?
[Addison Phillips] 

I haven't seen that suggestion previously or I missed it. If he's suggested it for inclusion into ISO 15924 (a better place, perhaps?), then draft-matching will pick it up by default.
> 
>    For language tags that have no script subtag but whose language
>    subtag's record in the IANA Language Subtag Registry contains the
>    field "Suppress-Script", the script element in the quintuple MUST be
>    set to the script subtag in the Suppress-Script field.  This is
>    necessary because [RFC3066bis] strongly recommends that users not use
>    this subtag to form language tags and this document recommends that
>    users not use them to form ranges.
> 
> Does this document recommend that? If so, I missed it (entirely
> possible). I thought this document was mostly meant to be used without
> the registry.
> [Ah, I see it now in 4.1.  Choosing Language Ranges - bad sequence for
> education...]
[Addison Phillips] 

Some sections got moved around quite a bit during the last edit, making the text less logical. In this case, there should be a forward reference, combined with some editing of the paragraph in question.
> 
> Figures: I can't find Figures 1, 2, 4, 5 - or at least no captions
> with those "Figure x" values.
[Addison Phillips] 

This was a comment on draft-07. I haven't gone through and suppressed the figure numbers yet or determined which, if any, should be numbered. Thanks for noticing.
> 
> Misspelling in 4.1: "interefer" -> "interfere"
[Addison Phillips] 

Corrected. 
> 
> 
> 4.4.  Length Considerations in Matching
> This seems to say an implementation must be able to handle 33, then
> later 42, characters. Pick one...
[Addison Phillips] 

No, it says that 33 characters are required, but "at least 42" is a better minimum. I am not fond of this section: I think it applicable only in the most arcane cases as well as difficult to read. It might be an alternative subject of this old joke:

Q: What is a camel?
A: It is a horse designed by a committee.

> 
> What's the rationale for the recommended 62 characters? Why not 42,
> 47, 63, whatever?
[Addison Phillips] 

33, 42, and 62 are each "magic numbers" for language tags, indicating the longest possible tag containing specific combinations of subtags (or the maximum length). There is an extensive thread discussing these numbers in the LTRU archive, although I can't find it right now (gmane is down??)

> 
> 
> Truncation: I propose that, if an extension or privateuse consists of
> a singleton followed by 2 or more subtags, the whole part including
> the singleton should always be removed. Removing just the last of
> multiple subtags inside an extension will likely produce nonsense.
[Addison Phillips] 

Actually, it shouldn't. Badly designed extensions will produce that effect (read the next-to-last paragraph in RFC 3066bis, section 3.7). Mostly, though, matching extensions is a waste of time: they are supposed to be orthogonal to tags, after all.

> 
> Authors:
> 
>    Mark Davis (editor)
>    IBM
> 
>    Email: mark dot davis at ibm dot com
> 
> -> Mark needs an "us dot" after the "at".
[Addison Phillips] 

I us-dotted him.
> 
> 
> markus
> 
> --
> Opinions expressed here may not reflect my company's positions unless
> otherwise noted.
_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: Mark Davis <[email-masked]>
Date: Mon, 12 Dec 2005 17:38:54 -0800

The comments and dispositions look good. One item: Zzzz already exists 
as a 15924 value, and is thus already allowed, so it is orthogonal to 
this group. The UTC proposal is to tag reserved (not yet allocated to 
characters) code points with it.

Mark

---

From: Markus Scherer <[email-masked]>
Date: Tue, 13 Dec 2005 12:11:34 -0800

Thanks for your consideration of my comments. Dispositions look good.

A couple more comments:

> Comments follow. Thanks again for your comments. As noted, I have copied the list on this message, which I hope you don't mind.

I don't mind at all.

> Actually, it shouldn't. Badly designed extensions will produce that effect (read the next-to-last paragraph in RFC 3066bis, section 3.7).

I was thinking of POSIX/CLDR-like extensions where one might have
keyword-value pairs as in
de-US-markus-c-coll-phone-strength-secondry-case-upperfst

Since a vanilla truncation wouldn't know the structure of the
extension, removing the whole extension seems better than removing
trailing pieces.

> Mostly, though, matching extensions is a waste of time: they are supposed to be orthogonal to tags, after all.

I agree.

markus