Subject: [Ltru] Martin's WG last call comments on matching

From: Martin Duerst <[email-masked]>
Date: Thu, 30 Mar 2006 18:53:46 +0900

[co-chair hat off]

I have read the -11 version of the matching draft carefully.
Compared to previous drafts, the current draft is quite a bit
cleaner and more straightforward, and I think we should not
loose much more time on it.

However, I think it would be good to check/fix the following few
issues.

A) editorial, check

   Section 2.1:
     Basic language ranges (originally described by HTTP/1.1 [RFC2616] and
     later [RFC3066]) have the same syntax as an [RFC3066] language tag or
     are the single character "*".  They differ from the language tags
     defined in [RFC3066bis] only in that there is no requirement that
     they be "well-formed" or be validated against the IANA Language
     Subtag Registry (although such ill-formed ranges will probably not
     match anything).

   I'm not totally sure the second [RFC3066] on the second line is
   what we want, or whether it should not be [RFC3066].

B) editorial, move

   Section 2.1: The last paragraph made sense here earlier, it looked
   very much out of context, because the concept of prefix is not
   mentioned anymore beforehand. I propose to move it to Section 4.2,
   after the current single paragraph in that section. The pararaph
   reads:

     Use of a basic language range seems to imply that there is a semantic
     relationship between language tags that share the same prefix.  While
     this is often the case, it is not always true and users should note
     that the set of language tags that match a specific language range
     may not represent mutually intelligible languages.

C) potentially substantial, but very minor

   In Section 2.2, there is a sentence saying:

      However, wildcards outside the first position in an extended
      language range are ignored by most matching schemes.

   I find the 'most' inappropriate. It suggests that there are a
   considerable number of such schemes in the first place. I suggest
   changing this to:

      However, wildcards outside the first position in an extended
      language range are ignored by some matching schemes, in
      particular by Extended Filtering (see Section 3.2.2).

D) editorial or substantial, depending on why we have the current
    text

   There are a few instances where only "Implementations" are mentioned,
   where I'd have assumed it would say something like "Applications,
   protocols and implementations" (or whatever our preferred phrase was).

   If the uses of just "Implementation" in these cases is just an
   editorial oversight, I suggest we fix it. If it is not an editorial
   oversight, I still would prefer to fix it.

   The places I have found are as follows:

   D-1: Section 2.2:
     Implementations that specify basic ranges MAY map extended language
     ranges to basic language ranges: ...

   D-2: Section 3.3, just after Figure 4:
     Implementations SHOULD ignore extensions and unrecognized private-use
     subtags when performing lookup, since these subtags are usually
     orthogonal to the user's request.

   D-3: Section 3.3, second-to-last para:
     Implementations that accept extended language ranges MUST define
     which content is returned when more than one item matches the
     extended language range.

   D-4: Section 3.3, last para (two instances):
     For example, an implementation could return the matching tag that is
     first in ASCII-order.  If the language range were "*-CH" and the set
     of tags included "de-CH", "fr-CH", and "it-CH", then the tag "de-CH"
     would be returned.  Another possibility would be for an
     implementation to map the extended language ranges to basic ranges.

E) editorial (or at least I think so)

   Section 3.2.2 is entitled "Extended Filtering", and starts:

     When filtering using extended language ranges, each extended language
     range in the language priority list is considered in turn, according
     to priority.  A particular language range is compared to each
     language tag using the following process:
     ...

   This mixes up the general concept of filtering using extended language
   ranges and the specific concept of extended filtering as defined in
   our draft. It is probably a remnant of the earlier days, when our draft
   was more along the lines of "this is it" rather than "here are some
   reasonable ways to do things". I suggest to change the above text to:

     Extended filtering uses extended language ranges. Each extended language
     range in the language priority list is considered in turn, according
     to priority.  A particular language range is compared to each
     language tag using the following process:
     ...

F) explanatory

   I still remember having been surprised by how de-*-DE was used
   for extended filtering, and wondering whether the range of de-DE
   would lead to the same results (as it did).

   To help future readers, I'd like to modify the final paragraph of
   3.2.2, which currently reads:

     Note: The structure of language tags defined by [RFC3066bis] defines
     each type of subtag (language, script, region, and so forth)
     according to position, size, and content.  This means that subtags in
     a language range can only match specific types of subtags in a
     language tag.  For example, a subtag such as 'Latn' is always a
     script subtag (unless it follows a singleton) while a subtag such as
     'nedis' can only match the equivalent variant subtag.

   to something like the following:

     Note: The structure of language tags defined by [RFC3066bis] defines
     each type of subtag (language, script, region, and so forth)
     according to position, size, and content.  This means that subtags in
     a language range can only match specific types of subtags in a
     language tag.  For example, a subtag such as 'Latn' is always a
     script subtag (unless it follows a singleton) while a subtag such as
     'nedis' can only match the equivalent variant subtag. The only
     exception to this is that two-letter subtags in initial position
     have a different type (language) than two-letter subtags in
     later positions (country). This is the reason why in Extended
     Filtering, "*" characters in extended language ranges are
     significant in the first position, but insignificant if they
     appear later.

G) editorial, minor

   The first sentence in 3.3 Lookup reads:

     Lookup is used to select the single language tag that best matches
     the language priority list for a given request and return the
     associated content.

   I think it would be marginally easier to read if it read:

     Lookup is used to select the single language tag that best matches
     the language priority list for a given request and to return the
     associated content.

H) editorial, readability

   In Section 4.1, second para, please change

     Most matching schemes make no attempt to process the semantic meaning
     of the subtags and the language range is compared, in a case-
     insensitive manner, to each language tag being matched, using basic
     string processing.

   to make two sentences, as follows:

     Most matching schemes make no attempt to process the semantic meaning
     of the subtags. The language range is compared, in a case-
     insensitive manner, to each language tag being matched, using basic
     string processing.

I) explanatory, but somewhat major

    Section 4.1, 4th para, says:

      Users SHOULD avoid subtags that add no distinguishing value to a
      language range.  When filtering, the fewer the number of subtags that
      appear in the language range, the more content the range will
      probably match, while in lookup unnecessary subtags might cause
      "better", more-specific content to be skipped in favor of less
      specific content.  For example, the range "de-Latn-DE" would return
      content tagged "de" instead of content tagged "de-DE", even though
      the latter is probably a better match.

    This works in the case above, but there are other cases where the
    choice is not so easy. zh-Hans would be a good example. A user who
    reads both Simplified and Traditional Chinese, but who prefers
    simplified would use zh for filtering (getting all the content
    she can read) but zh-Hans for lookup (making sure she gets the
    preferred form if it's available, but the fallback to zh will
    still work). On the other hand, content in this case should be
    labeled as zh-Hans (or zh-Hant if that applies) for filtering,
    but for lookup, if there is either zh-Hans content or zh-Hant
    content, then one of them (the one considered 'default') should
    also be looked up under a simple zh.

    I think that if we do not explain this case where ranges and
    taggings for filtering and lookup differ, users of our document(s)
    will not be well served. I therefore think it is important to
    add this explanation.

J) editorial, sentence structure/word usage

   The first para of Section 4.3 says:

     Private-use subtags require private agreement between the parties
     that intend to use or exchange language tags that use them and great
     caution SHOULD be used in employing them in content or protocols
     intended for general use.  Private-use subtags are simply useless for
     information exchange without prior arrangement.

   The first sentence is difficult to understand. 'use' is used too much.
   I suggest the following rewrite:

     Private argeement is necessary between the parties that intend to use
     or exchange language tags that contain private-use subtags.
     Great caution SHOULD be used in employing private-use subtags in
     content or protocols intended for general use.  Private-use subtags
     are simply useless for information exchange without prior arrangement.

K) <flame>To say it mildly, I'm rather upset that Section 6, Changes,
    is still around. I wouldn't mind if it were there  and contained
    useful information, or if it were almost up to date or there were
    any chance to get it up to date. As it is, it simply totally useless,
    and should go immediately, and I have said so previously and
    not received any satisfactory reason for its presence.</flame>

[co-chair hat on]
As a co-chair, I will not forward this draft to the AD and the IESG
as long as it still contains section 6 in its present form. While
there is some cleanup that the RFC editor has to do (such as
replacing RFC3066bis with the real number), I do not want to
bother anybody higher up or farther down the line with this
section in its present form.
[co-chair hat off]

Regards,     Martin. 

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: John Cowan <[email-masked]>
Date: Thu, 30 Mar 2006 13:03:44 -0500

> However, I think it would be good to check/fix the following few

+1 to all specific points.

-- 
A mosquito cried out in his pain,               John Cowan
"A chemist has poisoned my brain!"              http://www.ccil.org/~cowan
        The cause of his sorrow                 http://www.ap.org
        Was para-dichloro-                      [email-masked]
Diphenyltrichloroethane.                                (aka DDT)

---

From: "Addison Phillips" <[email-masked]>
Date: Thu, 30 Mar 2006 11:06:10 -0800

Responses follow.

Addison Phillips
Internationalization Architect - Yahoo! Inc.

Internationalization is an architecture.
It is not a feature. 

> -----Original Message-----
> From: Martin Duerst [mailto:[email-masked]]
> Sent: 2006?3?30? 1:54
> To: LTRU Working Group
> Subject: [Ltru] Martin's WG last call comments on matching
> 
> 
> A) editorial, check
> 
>    Section 2.1:

Do you mean "it should not be [RFC3066bis]"?

If so, the answer is "no, we want RFC 3066 here". The tag syntax for ranges
is explicitly more relaxed and is compatible with RFC2616 and friends, not
the more stringent 3066bis rules.
>      may not represent mutually intelligible languages.

The real issue here is not the range syntax but rather that syntax's use in
filtering (it doesn't apply to lookup). I propose we move this text to
section 3.2 and rephrase as:

--
Filtering seems to imply that there is a semantic relationship between
language tags that share the same prefix. While this is often the case, it
is not always true and users should note that the set of language tags that
match a specific language range may not represent mutually intelligible
languages.
--

>       particular by Extended Filtering (see Section 3.2.2).

How about (including a minor mod to the following sentence):

--
However, wildcards outside the first position are ignored by Extended
Filtering (see Section 3.2.2). The use or absence of one or more wildcards
SHOULD NOT be taken to imply that a certain number of subtags will appear in
the matching set of language tags.
--
>    oversight, I still would prefer to fix it.

It isn't an editorial oversight in some cases and is in others. The
rationale for the non-oversight instances has subsequently been removed,
though. The only one I object to fixing is the last one, which is a
>      language tag using the following process:

I agree.
>    would lead to the same results (as it did).

That comment resulted in large scale wording changes to the algorithm, as I
recall.
>      significant in the first position, but insignificant if they

The main problem with your suggested text is the word "exception".
Position/size/content are what separates the subtag items. The problem is
the need to indicate the position if you want an 'emtpy' or "don't care"
initial subtag. Note that the algorithm does not permit "*-Latn" to match
tags "x-Latn" or "de-f-Latn"!!

Thus I would suggest this modified version of your text:

--
Note: The structure of language tags defined by [RFC3066bis] defines each
type of subtag (language, script, region, and so forth) according to
position, size, and content.  This means that subtags in a language range
can only match specific types of subtags in a language tag.  For example, a
subtag such as 'Latn' is always a script subtag (unless it follows a
singleton) while a subtag such as 'nedis' can only match the equivalent
variant subtag. One such difference is that two-letter subtags in initial
position have a different type (language) than two-letter subtags in later
positions (region). This is the reason why, in Extended Filtering, wildcards
in extended language ranges are significant in the first position, but can
be ignored if they appear later.
--

>      the language priority list for a given request and to return the

I disagree. Your wording separates this single action into two actions.
>      insensitive manner, to each language tag being matched, using basic

+1
>     will not be well served. I therefore think it is important to

The key phrase is "distinguishing value". I agree that we should add text
based on your example.
>      are simply useless for information exchange without prior
arrangement.

+1
>     not received any satisfactory reason for its presence.</flame>

I didn't remove it from draft-11 since I intended to (and have been
recording) the WGLC changes in the document. Since idnits doesn't care, I
have removed it from my editor's copy instead.

---

From: "Addison Phillips" <[email-masked]>
Date: Thu, 30 Mar 2006 11:42:54 -0800

I updated the editor's copy on:

   http://www.inter-locale.com/ID/draft-ietf-ltru-matching-12.txt
   http://www.inter-locale.com/ID/draft-ietf-ltru-matching-12.html
   http://www.inter-locale.com/ID/draft-ietf-ltru-matching-12.xml

... with the changes discussed below.

Addison

Addison Phillips

> From: Addison Phillips [mailto:[email-masked]]
> Sent: 2006?3?30? 11:06
> To: 'Martin Duerst'; 'LTRU Working Group'
> Subject: RE: [Ltru] Martin's WG last call comments on matching
> 
> Responses follow.
> 
> Addison Phillips