Subject: Re: Layer 2 and "idn identities" (was: Re: [idn] what are the IDN identifiers?)

From: liana Ye <[email-masked]>
Date: Fri, 30 Nov 2001 13:21:11 -0600

Thanks John for you are addressing what I am looking
for.   And I do think bottom up discussion hitting the 
wall after the introducing TC/SC, because it is a part of  the CJK 
problem that JET has been facing.   It is a good time to look 
at top-down wise.  And we will hit CJK problem
too as codepoint usage conflicts. 

So I think CJK code points usage conflicts should be 
resolved before this group can discuss solutions effectively.
Thus the introduction of the term "equivalent character set" 
is to capture the broader "case folding" or normalization.

For example: 
One Latin case from [nameprep]:
0048; 0068; Case map

210B; 0068; Additional folding
210C; 0068; Additional folding
210D; 0068; Additional folding

1D407; 0068; Additional folding
1D43B; 0068; Additional folding
1D46F; 0068; Additional folding
1D4D7; 0068; Additional folding
1D573; 0068; Additional folding

And Chinese TC/SC example:
<wind> has four code points in Chinese:
TC, SC, TC radical, SC radical.

I believe there are other code points of <wind>  have 
been allowed in UCS.  Isn't  it a time to use the 
"equivalent character set" term? Or may we use 
"equivalent codepoints" to stay away from the lanugage
connotation?

If the "equivalent character set" is not a good term to group
this codepoint normalization problem, I'd like to hear it. 
No feed-back means agreeable  to the term.   

Liana

On Fri, 30 Nov 2001 06:42:41 -0500 John C Klensin <[email-masked]>
writes:
> --On Wednesday, 28 November, 2001 21:53 -0600 liana Ye
> <[email-masked]> wrote:
> 
> >...
> >  In one word, I am arguing for John's layer
> > two stuff to be dealt with in his layer one, is this 
> > multilingual?  I think this is the difference we have 
> > started from beginning, from allowing TC/SC in
> > [nameprep].  This is nothing to do with "multilingual".
> 
> But, Liana, these sorts of things are precisely why there is a
> layer 2, so I'm not sure I understand what you are suggesting.
> 
> To go through that once again:
> 
> (i) In the general case, we cannot handle TC-SC, or any other
> language-sensitive, or context-sensitive, translations or
> matchings in the DNS.  A number of special cases and/or subsets
> can be made to work.  But special cases and subsets are very
> confusing to users who expect the generalizations if anything
> appears to work.   And, without a clear demonstration that there
> is a solution for the [more] general case on the near horizon,
> it would be irresponsible for the IETF to approve a "this
> handles the easy case, we will deal with hard cases later" style
> of solution -- that sort of approach is almost always bad
> systems design, and demonstrably so here.
> 
> (ii) Language tagging in the DNS in some form would help a bit,
> but doing such tagging has gotten, as far as I can tell, not
> even a sign of consensus in the WG.  It is reasonable that it
> has not, if only because nothing the WG has done or agreed to
> would change the fundamental character-by-character nature of
> the DNS or the assumption that the characters of a label can be
> chosen from any valid character from the selected subset of the
> relevant CCS, in any order (with the exception of the legacy
> rules about hyphen-minus).
> 
> One way to look at the above is that the DNS just doesn't have
> enough information available during matching.  The matching
> algorithms don't have access to language information, country
> information, or other things than could be used to sort out
> similarities and variants.  And the DNS does exact matches -- no
> ambiguities permitted.  If the needed information isn't there,
> no matching tricks or "preparation" is going to help -- there is
> no place in the DNS or either magic of "do what I mean"
> capabilities either.
> 
> One can, again, get partially around those problems with
> client-side code --nameprep is precisely such a partial solution
> (or a potentially-complete solution to a narrower problem).  But
> complex client-side solutions have a history of causing
> interoperability problems and user confusion.  Client software
> suppliers find it just too tempting to provide codes that are
> "just a little better" than required by the standard.  In this
> case, that means that user-supplied names will be interpreted
> differently depending on which client software package is used,
> and that is very bad news.  Worse, interoperability testing in
> the usual IETF sense is impossible in these cases, since any
> damage has been done before a single bit crosses the wire.
> 
> Search Layer 2 (and 3, actually) are responses to these specific
> issues, not exercises in protocol design.  The documents didn't
> appear two years ago because it was important to understand how
> far a DNS-based solution could be extended.  I suggest that we
> now know, and that the answer is "not into these language
> problems, or context problems, or even script problems that
> involve considering characters more than one at a time".
> 
> "DNS Search", or, now, "IRNSS", is also, so far, just a
> framework.  We have proposals for sublayer 2 on the table, but
> neither is complete and more proposals are welcome.  But that
> framework explicitly provides for:
> 
> 	* Language information
> 	
> 	* Country information
> 	
> 	* Matching functions that can consider complete name
> 	strings, not just individual characters and one label at
> 	a time.
> 	
> 	* Matching functions that can do parallel table lookups,
> 	apply spelling differentiators, and do other "fuzzy"
> 	things and do them as predictable, server-side
> 	operations.
> 
> Isn't that the set of tools you are looking for?
> 
> See following note on IRNSS.
> 
>     john
>

---

From: John C Klensin <[email-masked]>
Date: Fri, 30 Nov 2001 17:54:37 -0500

--On Friday, 30 November, 2001 13:21 -0600 liana Ye
<[email-masked]> wrote:

> wall after the introducing TC/SC, because it is a part of  the
> CJK  problem that JET has been facing.   It is a good time to
> look  at top-down wise.  And we will hit CJK problem

But there is very little top-down where the DNS itself is
concerned.  Despite upward-facing uses, it is fundamentally
downward-facing, and that is, in different language, what the
"identifier" discussion is all about. 

> resolved before this group can discuss solutions effectively.

I think that statement is equivalent to "the IDN WG should never
finish its work, or at least should take years to do so". With
the understanding that I don't read Han characters, I think
there have been ample illustrations over the last several months
that completely resolving "CJK code point(s) usage conflicts"
will require that the Japanese and Korean languages be reformed,
possibly to use a different character set base entirely.

> TC, SC, TC radical, SC radical.

Liana, this is just not helpful.  I wrote a fairly careful
explanation of why a particular approach would not work and what
at least one alternative was.  You apparently ignored that
discussion (I'm not even sure that you read it) and instead
appear to just be repeating what you have been saying all along.

And trying to create terminology confusion doesn't help either:
suppose we see an animal with four legs that gives milk and
makes sounds transcribed in some languages as "moo", and I say
to you "that is a duck".  That might conceivably result in some
confusion on your part, but it isn't going to cause the animal
to start quacking or to take wing and fly away.  Case-folding is
a well-defined operation for scripts associated with languages
with cases and that don't use diacritical marks.  It is somewhat
less well-defined for scripts associated with languages that
have cases but that do use diacritical marks.   It is
meaningless for languages without case distinctions.

It is completely appropriate for you (or others) to say "if it
is reasonable to do case-folding for scripts with cases, then it
is reasonable to map TC<->SC".   It is possible to debate the
reasonableness of that statement and the feasibility of doing
the mapping (that debate has been going on for some time now).
But the statement itself is appropriate to make.  

By contrast, when you try to equate case-folding with TC<->SC
mapping, you are either making a statement that is false or you
are playing language games in the hope of convincing people of
something analogous to cows being ducks.  Please stop it; it
doesn't help, and it lowers your credibility.

I deliberately haven't used "normalization" in the discussion
above, since I'm not sure I know what that word means or, more
specifically, since I think it can be stretched to cover enough
meanings to not provide a useful definition.

> "equivalent codepoints" to stay away from the lanugage

I don't think this is helpful either, at least unless you want
to associate distance functions to the concept of "equivalence".
That, of course, is another version of the problem: the DNS is
binary -- things either match on a codepoint by codepoint basis
or they do not -- while distance functions between (StringI,
LanguageI, CountryI) and (StringJ, LanguageJ, CountryJ) are what
is needed here.  But, had you read my previous notes, you would
presumably understand that already.
 
> No feed-back means agreeable  to the term.   

Please do not, ever, send a note to an IETF mailing list that
buries an "if you don't respond, you agree with me" statement at
the end of a long note (or in the middle of one).  We just don't
work that way around here and, again, trying it damages your
credibility.

    john