Subject: RE: [Ltru] draft-06 of matching prototype...

From: "Addison Phillips" <[email-masked]>
Date: Tue, 15 Nov 2005 17:42:42 -0800

To illustrate... I updated draft-06 on the inter-locale site. Take a look at that text and see how you'd change it.

Addison

Addison P. Phillips
Globalization Architect, Quest Software

Internationalization is not a feature.
It is an architecture. 

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Addison Phillips
> Sent: 2005年11月15日 15:50
> To: Mark Davis
> Cc: LTRU Working Group
> Subject: RE: [Ltru] draft-06 of matching prototype...
> 
> You're missing what I said, I think. We are saying the same thing. I'm
> merely suggesting that how we handle the language priority list be
> different. Your suggestion results in two behaviors that I think are
> undesirable:
> 
> 1. The primary language subtag's record has to be looked up in the
> registry (to see if it has a Suppress-Script).
> 2. Most ranges that contain a "*" are skipped instead of being included in
> the fallback.
> 
> I said, indeed, get rid of "Extended Range Lookup" and have only (what
> amounts to) "Basic Range Lookup".
> 
> The tag "*-US" matches whichever among the set "en"/"no"/"fr"/"pt" is the
> default (i.e. it matches "").
> 
> I don't think I'm looking at it as filtering. I'm looking strictly to
> write what the fallback pattern is. This works better (I'm trying to show)
> if you do NOT skip all values that contain a wildcard (except in the first
> position).
> 
> Let's state it a different way. Imagine I request:
> 
>    zh
> 
> and you have:
> 
> en
> no
> fr
> pt
> 
> Q: What gets returned then?
> A: Whatever the default content is.
> 
> So a request for "*-US" in my proposal is the same as "*" (essentially,
> nothing). We don't define what the default content is, because it is
> probably user defined.
> 
> Does that make more sense?
> 
> Addison
> 
> Addison P. Phillips
> Globalization Architect, Quest Software
> 
> Internationalization is not a feature.
> It is an architecture.
> 
> > -----Original Message-----
> > From: Mark Davis [mailto:[email-masked]]
> > Sent: 2005年11月15日 15:30
> > To: Addison Phillips
> > Cc: LTRU Working Group
> > Subject: Re: [Ltru] draft-06 of matching prototype...
> >
> > My main concern is that I don't think it is very well defined. It is
> > still approaching it from a filtering point of view, and not a lookup
> > point of view. What happens when you have
> >
> > *-US
> >
> > and you have
> >
> > en
> > no
> > fr
> > pt
> >
> > What do you get? If you can come up with a reasonable answer, then I can
> > see keeping the section. But I don't see an obvious answer.
> >
> > Mark
> >
> >
> > Addison Phillips wrote:
> >
> > >The range "*" means "default". Since it matches anything, there is no
> > other basis on which to choose. Default is implementation defined. Both
> of
> > these things really should be added to the Basic Range Lookup scheme
> (that
> > "*" means "default" and that the default is implementation defined or
> the
> > empty tag).
> > >
> > >I strongly dislike your #2 and #3. I think my approach was cleaner and
> > more logically consistent.
> > >
> > >If a tag has a primary language subtag and zero or more other subtags,
> > this is enough to get into the logical fallback pattern in a consistent
> > manner. Notice that a "*" can be omitted in an Extended Language Range.
> > "en-US" is considered equivalent to "en-*-US".
> > >
> > >Consider this language priority list:
> > >
> > >   en-*-US, fr; q=0.8
> > >
> > >Clearly I would expect to see "en-US" or "en" content before seeing
> > French. My algorithm produces both (but not "en-Brai" or "en-Latn-US").
> > Your algorithm produces the same result, so then let's consider:
> > >
> > >  en-*, fr; q=0.8
> > >
> > >My version still produces "en" before "fr" (but not "en-US"). Your's
> > produces "fr". Mine is logical to the end user: they wanted "any flavor
> of
> > English" and they got the generic English.
> > >
> > >Let's dive deeper still. Consider:
> > >
> > >  de-*-1901, fr; q=0.8 // * in both script and region position
> > >
> > >Now my version produces "de-1901" and "de" before "fr" (but not, for
> > example, "de-CH-1901"), while your's produces "fr" immediately.
> > >
> > >Thus the same request can be used to fulfill both lookup and filtering
> > application needs with results consistent for *both*.
> > >
> > >Now let's consider this request:
> > >
> > >  *-US, fr; q=0.8
> > >
> > >Here is where we would need to add one twist to my proposal: if the
> range
> > is "*" and a language priority list has been supplied, go to the next
> item
> > before returning the default content.
> > >
> > >If you think about it, the request above is "give me any dialect used
> in
> > the USA and, if not available, give me French". That request could just
> as
> > easily return "zh-Hant-US" or "sgn-US" as "en-US": it is essentially a
> > hopeless request for anything except filtering. In this case, treating
> "*-
> > US" as "default" and skipping to French gives a fallback pattern of:
> > >
> > >  1. fr
> > >  2. (default or empty tag)
> > >
> > >What do you think?
> > >
> > >Addison
> > >
> > >Addison P. Phillips
> > >Globalization Architect, Quest Software
> > >
> > >Internationalization is not a feature.
> > >It is an architecture.
> > >
> > >
> > >
> > >>-----Original Message-----
> > >>From: Mark Davis [mailto:[email-masked]]
> > >>Sent: 2005年11月15日 12:07
> > >>To: Addison Phillips
> > >>Cc: LTRU Working Group
> > >>Subject: Re: [Ltru] draft-06 of matching prototype...
> > >>
> > >>We could consider that, but...
> > >>
> > >>If I lookup "*" and "fr" and "en" are available, which gets returned?
> > >>Can't have that kind of indeterminacy in lookup in the spec. So if we
> > >>want to take that approach, then I suggest the following mapping:
> > >>
> > >>1. Remove all trailing "-*" strings.
> > >>2. Remove "-*" for the script if there is Suppress Script
> > >>3. If the tag has any remaining "*", then it cannot be mapped. If the
> > >>range was part of a priority list, then ignore it for lookup. If there
> > >>are no values left, then the lookup fails (or returns the default
> value).
> > >>
> > >>An implementation may (MAY?) chose not to do #3 if it has a mechanism
> > >>for prioritizing lookups.
> > >>
> > >>[Add note in Security section that such behavior that causes
> differences
> > >>between implementations may be open to security problems.]
> > >>
> > >>Mark
> > >>
> > >>Addison Phillips wrote:
> > >>
> > >>
> > >>
> > >>>>I really like the restructuring. Interesting how much value it is
> just
> > >>>>to introduce terminology like 'filtering'; it will make the
> situation
> > >>>>much clearer to users.
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>[Addison Phillips]
> > >>>
> > >>>I thought it worked out pretty well.
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>>What I think it also makes clear is that we shouldn't be including
> > >>>>3.2.2. Extended Range Lookup. A lookup has to give a precise, single
> > >>>>answer whenever there are a number of possible matches. The
> > formulation
> > >>>>that the draft gives does not distinguish in priority among matching
> > >>>>items.
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>[Addison Phillips]
> > >>>
> > >>>I agree, although I would take a different approach.
> > >>>
> > >>>For HTTP in particular, the Accept-Language header might be used for
> > both
> > >>>
> > >>>
> > >>filtering and lookup applications. It would be useful I think to
> specify
> > a
> > >>mapping from Extended to Basic Language Ranges in the (sole) Lookup
> > scheme.
> > >>
> > >>
> > >>>I think the mapping works thusly:
> > >>>
> > >>>1. If the Extended Language Range starts with "*", then the Basic
> > >>>
> > >>>
> > >>Language Range is "*".
> > >>
> > >>
> > >>>2. Otherwise, remove any "*" subtags and their associated trailing "-
> "
> > if
> > >>>
> > >>>
> > >>it exists.
> > >>
> > >>
> > >>>Thus a request for:
> > >>>
> > >>> "*-US" => "*"
> > >>> "en-*-US" => "en-US"
> > >>> "sr-*-CS" => "sr-CS"
> > >>> "sl-*-*-nedis" => "sl-nedis"
> > >>> "de-*-1901" => "de-1901"
> > >>> "de-*" => "de"
> > >>>
> > >>>Best Regards,
> > >>>
> > >>>Addison
> > >>>
> > >>>Addison P. Phillips
> > >>>Globalization Architect, Quest Software
> > >>>
> > >>>Internationalization is not a feature.
> > >>>It is an architecture.
> > >>>
> > >>>
> > >>>
> > >>>
> > >
> > >
> > >

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: Martin Duerst <[email-masked]>
Date: Wed, 16 Nov 2005 18:52:17 +0900

[This is mostly a technical contribution, plus
a tad of 'trying to sit back' to focus the discussion.]

In general, I like the improved structure very much.

Here, I'd like to closely look at what's the fundamental
difference between what's currently called filtering and
what's currently called lookup.

This is in part motivated by the observation that HTTP
language negotiation quite closely fits the "usage profile"
(one solution, potential default/fallback) of lookup, but
uses the matching pattern of filtering (general ranges
match against (potentially) more specific tags).

In the draft as currently written,
the fundamental difference is presented as:
filtering: zero to many results
lookup: one result

I think that this is actually not the most fundamental difference.

I think the most fundamental difference is the following:
filtering: most general language range, more specific ones are okay
lookup: most specific language range, more general ones are okay
(note that with this distinction, the terms may no longer be
completely accurate, but I'll still use them for the moment).

Remembering the discussion about language negotiation in HTTP
(about 8 or 10 years ago), when we worked on the details, I
think we came to the conclusion that there were two generally
workable ways to do things:

1) Negotiation would work with Accept-Language containing specific
ranges matched against potentially more general tags. I.e.
requesting en-US would also return en, but requesting en
wouldn't return en-US.

2) Negotiation would also work with Accept-Language containing
general ranges matched against potetially more specific tags.
I.e. requesting en would also return en-us, but requesting
en-US wouldn't return en.

In both cases, some advice is needed for users. For 1),
document providers have to be told to not use overly specific
tags (or in case multiple tagging is possible, not overly
specific tags only). For 2), end users (or browser makers)
have to be told to not request overly specific ranges
(or in case multiple ranges can be used, not overly
specific ranges only).

The HTTP spec in the end went for 2). I don't remember
exactly why, probably because some implementations did
it that way, or it seemed more natural to the people involved.
It may be somewhat more natural, but it's in no way
overwhelmingly better.

The important conclusion is that you can have it one way or
the other, but not both ways. You could say that e.g. a request
for en-US matches both en and (made up) en-US-boont, but this
means that a lot of stuff is being matched, and the selectivity/
expressiveness of the matching scheme is reduced.

In my eyes, whether in the end you chose only one result or
whether you end up with a (potentially prioritized) list of
results is important for the users of our draft (in their
protocol or application, which one they need will be pretty
clear), but it is not an intrinsic property of the matching
schemes we are defining.

Regards,   Martin.

At 10:42 05/11/16, Addison Phillips wrote:
 >Content-class: urn:content-classes:message
 >Content-Type: text/plain;charset="UTF-8"
 >
 >To illustrate... I updated draft-06 on the inter-locale site. Take a look
 >at that text and see how you'd change it.
 >
 >Addison
 >
 >Addison P. Phillips
 >> Sent: 2005$Bj;%((B11$Bk|/(B5$Bkw!&(B 15:50
 >> To: Mark Davis
 >> Cc: LTRU Working Group
 >> > Sent: 2005$Bj;%((B11$Bk|/(B5$Bkw!&(B 15:30
 >> > To: Addison Phillips
 >> > Cc: LTRU Working Group
 >> > >>Sent: 2005$Bj;%((B11$Bk|/(B5$Bkw!&(B 12:07
 >> > >>To: Addison Phillips
 >> > >>Cc: LTRU Working Group