Subject: Re: [idn] hostname history hell

From: Kenneth Whistler <[email-masked]>
Date: Wed, 21 Nov 2001 09:08:28 -0800 (PST)

Tim said:

> I think Eric's proposal may have legs. Or at least something along these
> lines. I agree with John's point that we need to start conservative and
> expand from that base-line. To be too inclusive at first means it would be
> nearly impossible to go back.
> 
> We must though be very careful not to inadvertently exclude
> scripts/characters that are used by some languages even though we thought
> they were merely symbols.

The list you are looking for is provided by the Unicode Consortium:

http://www.unicode.org/Public/UNIDATA/Scripts.txt

That gives script assignments for Unicode characters (Latin, Greek,
Cyrillic, Devanagari, Bengali, Han, ...), and provides guidance for
what not to leave out if you are simply trying to make a conservative
decision without leaving some languages essentially unrepresentable.

Note that many scripts inherently include combining characters. I
absolutely agree with Kent that a blanket prohibition of combining
characters is unacceptable. In a discussion dominated by English,
Chinese, and Korean speaker/writers, it might seem o.k., but I
assure you that if there were as many Arabic, Urdu, Hindi, and
Bengali speaker/writers participating, it would *not* seem o.k.

Otherwise, deciding to omit punctuation, space characters, format
control characters, and symbols is fine as a conservative approach
to the problem, however.

--Ken

---

From: John C Klensin <[email-masked]>
Date: Wed, 21 Nov 2001 17:10:57 -0500

--On Wednesday, 21 November, 2001 09:08 -0800 Kenneth Whistler
<[email-masked]> wrote:

>> scripts/characters that are used by some languages even
>> though we thought they were merely symbols.
> 
> That gives script assignments for Unicode characters (Latin,
> Greek, Cyrillic, Devanagari, Bengali, Han, ...), and provides
>...
> Note that many scripts inherently include combining
> characters. I absolutely agree with Kent that a blanket
> prohibition of combining characters is unacceptable. In a
> discussion dominated by English, Chinese, and Korean
> speaker/writers, it might seem o.k., but I assure you that if
> there were as many Arabic, Urdu, Hindi, and Bengali
> speaker/writers participating, it would *not* seem o.k.

Ken, I may not have been reading closely enough, but I don't
believe this discussion has included a proposal to ban combining
characters.  I do have an issue with them, but I think it is
separable (see below).

> Otherwise, deciding to omit punctuation, space characters,
> format control characters, and symbols is fine as a
> conservative approach to the problem, however.

Good to hear this.

The combining character problem (if it is a problem) is that, so
far, we have no proposals on the table that would require that a
DNS label be a valid name in any particular language, or even
that it be drawn, homogeneously, from any particular script.
Until and unless one of those rules is made (my guess is that it
would be nearly impossible to do so, but this is not my area of
expertise), we are thrown back on the traditional DNS rule that,
subject to the hyphen-placement rule, any valid character of the
chosen CCS can appear in any relationship to any other valid
character of the CCS.  In particular, there is no way to require
or assume script-homogeniety.  

If, to use your example, we have a selection of Arabic, Urdu,
Hindi, and Bengali, which characters from each script designed
by its first character,  
   AUHBBHUA
ought to be a valid label.  While we know how to construct AAAA,
UUUU, HHHH, and BBBB, regardless of whether a given character is
combining or non-combining, I worry about interpretation and
ambiguity if combining characters (or partial breaks, etc.) are
taken from one of these scripts and surrounded by characters
from an unrelated script.  Maybe it is not a problem, but I'd
like someone to assure me that is the case.

     john

---

From: Roozbeh Pournader <[email-masked]>
Date: Mon, 26 Nov 2001 23:40:25 +0330 (IRT)

> Bengali speaker/writers participating, it would *not* seem o.k.

I cannot speak for Indic scripts/languages, but from what I have found in 
the technical mailing list of AINC (Arabic Internet Names Consortium), and 
the Urdu Computing mailing list (where the current agenda is Urdu domain 
names), the consensus is not allowing combining marks in Arabic script 
internet names.

Just my two cents,

roozbeh