Subject: [Ltru] schemes based on list or on range?

From: "Addison Phillips" <[email-masked]>
Date: Fri, 18 Nov 2005 14:08:46 -0800

> >>[General Observation.
> >>
> >>We need to give a mechanism that will always give a precise, determinant
> >>answer. We then let protocols specify alternatives (if they need to) by
> >>additional specification. In particular, in each section we have to make
> >>clear what exactly happens with a language range, and what happens with
> >>a language priority list.
> >>
> >>For filtering, this can be done by moving the discussion of the language
> >>priority list up into 3.2. ]
> >>
> >>
> >[Addison Phillips]
> >
> >Actually, I'd go further and suggest that all of the matching schemes be
> defined in terms of language priority lists (a language range by itself
> being the collapsed case of a language priority list with only one item--
> inside the specific scheme we describe how to match a single range to a
> single tag, but the scheme overall matches the user's language
> preferences--a list--against content). Lookup now reads this way.
> >
> That works too. Then we just say at the very top that all the mechanisms
> treat a single language range as a single element language priority list.

Yep. I think we should finish harmonizing the text to this approach. Implementers can make certain assumptions when they only allow or only have a single range, but it is more important to describe the full algorithm in a single consistent manner. 

Addison

Addison P. Phillips
Globalization Architect, Quest Software

Internationalization is not a feature.
It is an architecture. 

> -----Original Message-----
> From: Mark Davis [mailto:[email-masked]]
> Sent: 2005年11月18日 13:42
> To: Addison Phillips
> Cc: [email-masked]
> Subject: Re: [Ltru] draft-06 comments: choice of range
> 
> 
> 
> Addison Phillips wrote:
> 
> >Comments interlinearly below. I have some strong reservations about some
> of these comments.
> >
> >Addison
> >
> >Addison P. Phillips
> >Globalization Architect, Quest Software
> >
> >Internationalization is not a feature.
> >It is an architecture.
> >
> >
> >
> >>-----Original Message-----
> >>From: Mark Davis [mailto:[email-masked]]
> >>Sent: 2005年11月18日 10:15
> >>To: Addison Phillips
> >>Cc: [email-masked]
> >>Subject: Re: [Ltru] draft-06 comments: choice of range
> >>
> >>
> >>
> >
> >>----
> >>
> 
> >This section 3.2 text:
> >
> >--
> >Filtering is used to select the set of content that matches a given
> prefix.
> >--
> >
> >Should be modified to say:
> >
> >--
> >Filtering is used to select the set of content that matches the user's
> language preferences (see Section 2.1, Language Priority Lists).
> >--
> >
> >Before objecting to this change because it removes the critical point
> about the range being a prefix, consider that the entire second paragraph
> of this section covers that.
> >
> >
> >
> >>3.2.
> >>If the language priority list (see Section 2.1 (Lists of Language
> >>Ranges)
> >><http://www.inter-locale.com/ID/draft-ietf-ltru-matching-07.html#lists>)
> >>contains more than one range, the content returned is typically ordered
> >>in descending level of preference.
> >>=>
> >>If the language priority list (see Section 2.1 (Lists of Language
> >>Ranges)
> >><http://www.inter-locale.com/ID/draft-ietf-ltru-matching-07.html#lists>)
> >>contains more than one range, the contents that match that range are
> >>added to the set of results.
> >>
> >>
> >[Addison Phillips]
> >
> >The word content in this sentence is already plural. "Contents" is wrong.
> If we want to be explicit about it, we should use the previously quoted
> formulation "set of content".
> >
> >
> set of content works.
> 
> >
> >
> >>[At the end of the section, after we've introduced the discussion of an
> >>ordered list and distance metric, add]
> >>
> >>When the set of matches is returned, the items are in descending order
> >>where the weight for each item is the priority in the language priority
> >>list divided by the distance metric. (If the filtering method does not
> >>supply a distance metric, then the distance metric is 1.) If the item
> >>matches multiple ranges, then its weight is the square root of the sum
> >>of squares of the individual weights for each range. If two items have
> >>the same weight, then their ordering is based on a case-insensitive
> >>ordering of the language tags.
> >>
> >>
> >[Addison Phillips]
> >
> >Ugh. Icky. Many items will have the same weight even in spite of this
> complex scheme.
> >
> >
> Can't get around that. If priorities are equal and there is no distance
> metric, then you essentially have an unordered list.
> 
> >I note that q weights in Accept-Language are always less than or equal to
> one and greater than or equal to zero. Your algorithm won't work correctly
> with those values (instead multiplication of the score works better).
> >
> >
> 
> You're right that there is a numeric problem, but you can't multiply,
> because smaller distances are better, but lower priorities are *worse*.
> So you have to divide one way or another. Look at an example.
> 
> X, q=0.5; Y, q=0.3;
> 
> Z matches X with a distance of, say, 2. W matches Y with a distance of
> 10. If you multiply, you get
> 
> Z -> (0.5 * 2) = 1
> W -> (0.3) * 10) =  3
> 
> That would rank Y better than Z! What you want is a division.
> 
> Now, as I think about it, the simplest way is to still think of the
> results as distances, with the smallest distance being the best. Then
> the algorithm is
> 
> a) toss out any q weights that are zero. After all, that means that you
> give them *no* priority.
> b) divide the distance by the priority.
> c) order in ascending order
> 
> Z -> (2 / 0.5) = 1
> W -> (10 / 0.3) = 33.33
> 
> which ranks them correctly.
> 
> The square rooty stuff was just because distance metrics with multiple
> fields weight better when you treat them as dimensional vectors.
> However, in retrospect it is probably just simpler to take the minimum
> of the possible weights. So if Q matches X with a distance of, say, 2. Q
> matches Y with a distance of 10. Then the result would be the minimum, or
> 1.
> 
> >>A protocol which uses language priority lists MAY specify an alternative
> >>mechanism for computing the ordering.
> >>
> >>----
> >>
> >>A protocol which uses language ranges MAY specify additional rules about
> >>the semantics of "*";
> >>=>
> >>A protocol which uses language ranges MAY specify additional rules about
> >>the semantics of "*" or other tags;
> >>[no reason to restrict it]
> >>
> >>----
> >>
> >>Implementations or protocols sometimes might wish to use more
> >>sophisticated weights that
> >>=>
> >>Implementations or protocols sometimes MAY specify the use of
> >>alternative weights that
> >>
> >>
> >[Addison Phillips]
> >
> >Either "...protocols MAY..." or "... protocols sometimes might..."
> >
> >
> sometimes MAY => MAY
> 
> >I chose non-normative language initially to allow things without turning
> the whole section into RFC 2119 soup.
> >
> >
> >>----
> >>[Make subsections in Lookup. First language range lookup, then language
> >>priority list lookup, then extended language range lookup.]
> >>
> >>
> >[Addison Phillips]
> >
> >I think that's an awful idea.
> >
> >And make up your mind: we REMOVED extended language range lookup because
> it doesn’t make sense. Lookup is now cast as I suggested above: it is
> strictly a language priority list beast, but it starts by defined the
> match between a tag and a range.
> >
> >
> Lemme relook at it, but it seems to me that you are doing it, but *via*
> transforming it. (but more in a separate message anyway).
> 
> >>----
> >>Other examples of lookup might include:
> >>=>
> >>Other examples of lookup include:
> >>[the examples *do* include. no need to be tentative ;-)]
> >>
> >>
> >[Addison Phillips]
> >
> >Okay.
> >
> >
> >>----
> >>
> >>If this language range is the only one in the language priority list, it
> >>matches the default content. If this language range is followed by other
> >>language ranges, it should be skipped.
> >>[doesn't address  *;*]
> >>
> >>
> >[Addison Phillips]
> >
> >Yes, it does... iteratively.
> >
> >
> You're right.
> 
> >
> >
> >>=>
> >>All "*" language tags are removed from language priority list. If there
> >>is nothing left, then it matches the default content.
> >>
> >>
> >[Addison Phillips]
> >
> >That's "ranges" in your first sentence.
> >
> >
> >>----
> >>
> >>5. (default content or the empty tag)
> >>=>
> >>5. (default)
> >>[explain below what this is. it is also otherwise odd that 1-4 are tags,
> >>but then this is 'content']
> >>
> >>
> >[Addison Phillips]
> >
> >Okay.
> >
> >
> >>----
> >>
> >>When performing lookup using a language priority list, the progressive
> >>search MUST proceed to consider each language range before finding the
> >>default content or empty tag. The default content might be content with
> >>no language tag (or with an empty value, as with xml:lang in the XML
> >>specification), or it might be a particular language designated for that
> >>bit of content.
> >>
> >>One common way to provide for default content is to allow a specific
> >>language range to be set as the default for a specific type of request.
> >>This language range is then treated as if it were appended to the end of
> >>the language priority list, rather than after each item in the language
> >>priority list.
> >>
> >>=>
> >>[reword for clarity. Funny thing: when I first read 'might be content
> >>with no language tag' I read it as the verb "be content with", like "be
> >>happy with". The empty tag is not a valid language tag, so we should say
> >>"und", then say that a system can use alternatives. Here is the text.]
> >>
> >>
> >[Addison Phillips]
> >
> >No, not 'und' (and I *hate* 'und'). 'Und' would be an example of an
> implementation specific decision. Personally I would prefer never to see
> 'und' content being used as a default.
> >
> >
> Well, "" is illegal as a language-tag, so I was just using it as a
> stand-in because we don't have a language tag for "root".
> 
> >>When performing lookup using a language priority list, the progressive
> >>search MUST consider each language range before finding the default. The
> >>default might be specified as "und" (or be indicated with an empty
> >>value, as with xml:lang in the XML specification, or with "root" as in
> >>some systems), or it might be specified as a particular language range.
> >>This language range is thus treated as if it were appended to the end of
> >>the language priority list, rather than after each item in the language
> >>priority list.
> >>
> >>
> >[Addison Phillips]
> >
> >I prefer not to make this change. I think it delves too closely into the
> implementation specific.
> >
> >
> It can't be left as it is. Other alternatives?
> 
> >>----
> >>
> >>5. (return default content)
> >>a. ja-JP
> >>b. ja
> >>c. (empty tag or other default content)
> >>
> >>5. (default)
> >>a. ja-JP
> >>b. ja
> >>c. und
> >>
> >>
> >[Addison Phillips]
> >
> >How about:
> >
> >5. (default content lookup)
> >   a. ja-JP
> >   b. ja
> >   c. (implementation defined)
> >
> >
> That works.
> 
> >>----
> >>
> >>Wildcard values in an extended language range are supposed to match any
> >>value
> >>=>
> >>Wildcard values in an extended language range match any value
> >>
> >>
> >[Addison Phillips]
> >
> >No, they are *supposed* to match, but in this case they don't. They match
> the empty value.
> >
> >
> I read that differently. When you say "supposed to match", it sounds
> like there is a choice, that there are some values that they don't
> match. Like the US is "supposed to" disallow torture.
> 
> >>----
> >>
> >>Add
> >>
> >>
> >>      3.3.x. Distance Metric Lookup
> >>
> >>Distance metric lookup is performed by producing an ordered list of
> >>filtered results as specified in 3.2.2, then returning the first item.
> >>
> >>
> >[Addison Phillips]
> >
> >I think we'd best not go down that path. Among other things: it won't
> work. Distance metric filtering can produce items with identical scores...
> >
> >
> Ok, drop it.
> 
> >
> >
> >>----
> >>
> >>Mark
> >>
> >>P.S. I still think that the extended range lookup doesn't quite work,
> >>but I'll send that in a separate message.
> >>
> >>
> >[Addison Phillips]
> >
> >No, it doesn't work and we eliminated it. We've mapped extended ranges
> onto basic ranges, which do work, producing different results than the
> language priority list might lead one to expect.
> >
> >
> Let me look this over again.

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru