Subject: [idn] San Diego Meeting Notes

From: "James Seng/Personal" <[email-masked]>
Date: Sun, 14 Jan 2001 13:51:55 +0800

Dear all,

Here is the minutes of the IDN WG Meeting at San Diego IETF. Thanks to David
(again) scribing 43Kb of text notes :P

Some important highlight: (feel free to object if you think I am wrong).

1. Requirements I-D is almost ready for last call with some minor changes. It
is
   likely we will do one more version. Any last comments, please do so ASAP.

2. General feeling that Nameprep sub-team is working in the right direction.

3. Strawpoll agrees with the Protocol sub-team recommendation that we will
focus
   on "ACE on Application" now but do not reject the possibiity to do
something
   else longer term. What is the longer term solution is left for future
discussion.

-James Seng

---

From: "D. J. Bernstein" <[email-masked]>
Date: 14 Jan 2001 09:09:11 -0000

Don't confuse protocols with software.

The ACE _protocol_ extensions are isolated in user-interface protocols,
while the UTF-8 _protocol_ extensions are not. But it is a serious
mistake to conclude that the ACE _software_ changes are more isolated
than the UTF-8 _software_ changes.

> ACE solutions, positive
> - they are easier to implement than the long term solutions,

That's simply not true.

The most easily implemented and deployed solution is straight UTF-8.
Nameprep can be handled in the way I've described. This solution allows
many existing 8-bit-clean programs, such as hundreds of thousands of
qmail installations, to handle IDNs properly.

(Yes, many UNIX users have to get the Unicode fonts and the UTF-8 xterm,
but ACE won't avoid that. These tools are included in current systems.)

The ACE solutions require more extensive code changes in more programs.
They then require more deployment effort. They will, for example,
require that qmail perform ACE conversions when

   * reading domain names from command lines,
   * reading domain names from environment variables,
   * reading domain names from configuration files,
   * printing domain names in logs, and
   * printing domain names in mail-queue summaries,

because all of these are part of qmail's user interface. After these
changes, qmail will have to be redeployed.

UTF-8 needs changes in most MUAs, some browsers, and certain servers,
notably Sendmail; deployment will not be easy. But ACE needs changes in
every MUA, every browser, every MTA, every web server, and many other
programs; deployment will be vastly more difficult.

In short, the costs of making ACE work are much higher than the costs of
making UTF-8 work. And that's not counting the future costs of switching
from ACE to UTF-8 later or inflicting ACE on future implementors.

> - the obvious advantage is that update just applications will go faster 
> than updating applications and the entire naming infrastructure

This concept of ``applications'' is bogus. The OSI model does not match
the reality of low-level programs talking directly to users.

---Dan

---

From: "James Seng/Personal" <[email-masked]>
Date: Wed, 24 Jan 2001 05:38:48 +0800

Dear all,

I would like to re-emphasis a few 'consensus' in the meeting in San
Diego.

a) Requirements I-D to move to Last Call after last minor edition.

b) Strawpoll agrees with Protocol Design Team recommendation to focus
   on "ACE on Application" now but do not rejecting a longer term
   solution in future.

c) Strawpoll agrees that the Nameprep Design Team is working in the
   right direction. (See Nameprep-02)

If you have any objection (I know a few of you do), please raise them
now.

If there is no major objection, then the next step would include

1. Zita will do the Requirements -04 and we will move to Last Call.

2. Protocol Design Team (or someone) should start working on writing
   an I-D for the IDN Protocol. As it stands now, IDNA seem the closest
   we have.

3. An ACE Design Team may be form to investigate what is the correct
   ACE to use for (2).

In additional, I would also like to propose to update the Goals and
Milestone section of the WG Charter to reflect our progress more
accurately.

Your comments would be appreciated.

-James Seng

---

From: "James Seng/Personal" <[email-masked]>
Date: Wed, 24 Jan 2001 06:29:36 +0800

I apologise. Yes, Shim is right. The relevant portion of the minutes as
follows.

-James Seng

--- CUT HERE ---
DL: is the consensus of the working group to be focusing on an
application
area approach.  Should we stop looking at infrastructure approaches.

<half the room>

HA: this does not mean the infrastructure need replacing.  just that
this
WG is not the one to do it.

----- Original Message -----
From: "Sung Jae Shim" <[email-masked]>
To: "James Seng/Personal" <[email-masked]>; <[email-masked]>
Sent: Wednesday, January 24, 2001 6:27 AM
Subject: Re: [idn] San Diego Meeting Notes

> I don't recall that straw poll agrees to focus on "ACE on
Application". But
> I recall that straw poll agrees to focus on "Application Area
Approach."
> Please take a look at the minutes.
>
> Sung Shim
>
>

---

From: "Sung Jae Shim" <[email-masked]>
Date: Tue, 23 Jan 2001 17:27:33 -0500

>    on "ACE on Application" now but do not rejecting a longer term

I don't recall that straw poll agrees to focus on "ACE on Application". But
I recall that straw poll agrees to focus on "Application Area Approach."
Please take a look at the minutes.

Sung Shim

---

From: "D. J. Bernstein" <[email-masked]>
Date: 23 Jan 2001 23:32:43 -0000

I object to ACE because the initial switch involves massive unnecessary
costs. The costs are discussed in http://cr.yp.to/proto/idn.html. See
below for a transliterated copy of idn.html.

I also object to the idea of specifying a short-term plan without a
long-term plan. How can we rationally evaluate the costs of ACE if we
don't also evaluate the likely costs of a future ACE-to-UTF-8 switch?

I also object to the characterization of ACE as an ``application-only''
solution. Is an MTA an ``application''? How about a DNS server? Unless
we can settle on a clear definition of ``application,'' we shouldn't
even be using the word, let alone making decisions that involve it.

As for nameprep: When are we going to see some working software? I have
a bunch of examples that I'd like to try; it's difficult to evaluate
nameprep without software.

---Dan

   D. J. Bernstein
   Protocols

                        Internationalized domain names

   We want to let people use domain names like abg.com (``alpha beta
   gamma dot com''). They should be able to register the name, set up
   computers under the name, connect to those computers by name, set up
   web pages under the name, set up links to those web pages, browse
   those web pages given the name or a link, send email from an address
   under the name, receive email at that address, etc.

   The big question is how these domain names will be encoded. This web
   page describes two proposals, and tallies the costs of each proposal
   for UNIX. (The costs for Windows are of similar types but generally
   smaller; Microsoft has supported Unicode for years.)

   Important note: An uppercase ABG.com (``Alpha Beta Gamma dot com'') is
   guaranteed to cause confusion: when the uppercase Alpha is printed
   properly, it looks just like an uppercase A. Some other strings are
   also guaranteed to cause confusion. There's a complicated definition
   of good names among all Unicode strings. Good names won't be confused
   with each other. Registrars won't allow registration of bad names.

Base costs

   Users need to be able to see common Unicode characters. The necessary
   fonts are available, as is a version of xterm that displays the
   characters given UTF-8 input. These are all included with the current
   version of XFree86, so they are being deployed as part of regular OS
   upgrades.

   Users will sometimes need to type strange addresses from business
   cards. Finding an unusual character in a huge font display is
   difficult, so I expect business cards to provide more information,
   such as Unicode numbers in small type. Keyboard interfaces will have
   to improve to accept this information. (The ISO standard method is
   Shift-Ctrl-222E for character 222E.)

Costs of ACE with slow nameprep

   What it means. Domain names are encoded as 7-bit strings in the
   following contexts:
     * DNS registration forms.
     * DNS queries and responses.
     * Mail message header fields: From, To, Received, etc.
     * POP USER commands. (POP usernames typically include domain names.)
     * Various parts of IMAP.
     * HTTP fields: Host, etc.
     * URLs.

   Domain names are encoded as UTF-8 strings in the following contexts:
     * The argument to gethostbyname.
     * h_name and h_aliases.
     * /etc/hosts and many other network configuration files.
     * /etc/named.boot and zone files.
     * Command lines for ndc, nsupdate, etc.
     * BIND log files.
     * /service/dnscache/root/servers.
     * dnscache log files.
     * /service/tinydns/root/data.
     * Command lines for add-host, add-ns, etc.
     * tinydns log files.
     * /etc/resolv.conf, $LOCALDOMAIN, etc.
     * Command lines for dig, host, etc.
     * Output of dig, host, etc.
     * Command lines for telnet, ssh, etc.
     * /etc/hosts.allow, .ssh/known_hosts, etc.
     * httpd.conf.
     * /public/file.
     * More HTTP server configuration files.
     * lynx.cfg.
     * Command line for lynx.
     * .fetchmailrc.
     * Pine interface: message displays, command line, etc.
     * Mutt interface: message displays, command line, etc.       
     * More mail clients.

   A domain name encoded as a UTF-8 string is permitted to be a bad name
   if it looks just like a good name. It is interpreted as that good
   name.

   Making it work. BIND, tinydns, and other DNS servers need to be   
   upgraded. Domain names in configuration files need to be converted  
   from possibly bad to good, and from UTF-8 to 7-bit. 7-bit domain names
   in queries need to be converted to UTF-8 for logs.

   The gethostbyname DNS client library needs to be upgraded. The input
   domain name needs to be converted from possibly bad to good, and from
   UTF-8 to 7-bit, before it is sent as a DNS query. The output domain
   names in h_name and h_aliases need to be converted from 7-bit to
   UTF-8.

   The tcpclient networking tool needs to be upgraded. The input domain
   name needs to be converted from possibly bad to good, and from UTF-8
   to 7-bit, before it is sent as a DNS query.

   telnetd, sshd, tcpserver, etc. need to be upgraded. Domain names in
   configuration files need to be converted from possibly bad to good,
   and from UTF-8 to 7-bit. 7-bit domain names need to be converted to 
   UTF-8 for logs.

   Many low-level networking tools need to be upgraded. Domain names in
   configuration files need to be converted from possibly bad to good,
   and from UTF-8 to 7-bit. 7-bit domain names need to be converted to
   UTF-8 for logs.    

   Pine, Mutt, and other text-mode mail clients need to be upgraded.
   7-bit domain names need to be converted to UTF-8 when messages are
   displayed. UTF-8 needs to be spaced properly. Addresses in
   configuration files (for example, ``flag messages to [email-masked]'')
   need to be converted from possibly bad to good, and from UTF-8 to
   7-bit.

   fetchmail and other POP clients need to be upgraded. Domain names
   embedded in POP usernames need to be converted from possibly bad to
   good, and from UTF-8 to 7-bit.

   Netscape Mail and other graphical mail clients need to be upgraded.
   7-bit domain names need to be displayed as Unicode glyphs. Addresses
   in configuration files need to be converted from possibly bad to good,
   and from UTF-8 to 7-bit.

   Apache, publicfile, and other HTTP servers need to be upgraded. Domain
   names in configuration files need to be converted from possibly bad to
   good, and from UTF-8 to 7-bit. 7-bit domain names need to be converted
   to UTF-8 for logs.

   Lynx and other text-mode browsers need to be upgraded. Domain names in
   configuration files need to be converted from possibly bad to good,
   and from UTF-8 to 7-bit. 7-bit domain names need to be converted to
   UTF-8 for internationalized URL displays.

   Netscape and other graphical browsers need to be upgraded. 7-bit
   domain names need to be displayed as Unicode glyphs. Domain names in
   configuration files need to be converted from possibly bad to good, 
   and from UTF-8 to 7-bit.

Costs of UTF-8 with fast nameprep

   What it means. Domain names are encoded as UTF-8 strings in all of the
   above contexts.

   Bad names are not allowed to appear. (Exception: Users can send bad
   names in DNS registration forms; the registrar will send back a
   rejection notice showing the closest good name.)

   Making it work. Sendmail needs to be upgraded. Current versions     
   discard bytes \200 through \237 in mail message headers.

   The gethostbyname DNS client library needs to be upgraded. Many
   current installations, in violation of RFC 2181, reject DNS answers 
   that contain unusual characters. (However, some versions will work 
   correctly with options allow_special all or options no-check-names in
   /etc/resolv.conf.) 

   Pine, Mutt, and other text-mode mail clients need to be upgraded.
   UTF-8 needs to be spaced properly.

   There's one report that an obsolete version of the Netscape mailer
   crashes under Solaris when it reads UTF-8 messages. I need verifiable
   details.

   Are there more programs that need to be upgraded? Let me know.

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Tue, 23 Jan 2001 17:19:42 -0800

>    an I-D for the IDN Protocol. As it stands now, IDNA seem the closest

The protocol design team report that was sent to the mailing list by 
your co-chair last week said:

"Because the proposed IDN solution is for the presentation-layer of 
applications, the protocol design team proposes that the IDN WG allow 
the Applications Area to finish the work on the specific protocol 
proposal. The people in the Applications Area are most familiar with 
the problems associated with changing the presentation layer."

The protocol design team did not volunteer to do the work. The 
Applications Area Directors have said they are willing to take the 
work on in the Apps area. Obviously, anyone from this WG would be 
invited to help.

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Tue, 23 Jan 2001 17:39:04 -0800

>below for a transliterated copy of idn.html.

That document appears to detail the costs (but does not quantify 
"massive") for upgrading applications to any form of IDN. It does not 
differentiate the costs between upgrading to an ACE or to UTF8. It 
does not say why going to an ACE is "unnecessary".

The fact that some programs can today display UTF-8 but cannot check 
nameprepped UTF8 shows that all programs are going to need to be 
updated for input of DNS names. If you believe that the cost of going 
to ACE is massive but going to UTF8 is not, that implies that the 
UTF8-to-ACE conversion cost is massively larger than the cost of 
doing nameprep. It is likely that the opposite is true: writing the 
normalization step in nameprep is probably much, much harder than 
coding an ACE, particularly LACE.

>don't also evaluate the likely costs of a future ACE-to-UTF-8 switch?

Wouldn't those costs be constant regardless of what the long-term 
plan is? Put another way, which two of the long-term plans proposed 
have different ACE-to-long-term costs?

>solution.

Good point. It might have more carefully worded as the design team 
did: it is for the presentation layer of applications.

>  Is an MTA an ``application''? How about a DNS server?

If it has a presentation layer, it will be affected by IDNA; if not, no.

>As for nameprep: When are we going to see some working software?

Many months ago.

>  I have

Some testbeds have already been announced on this list. Mine (which 
is written in Perl and the source code is available) is at 
<http://www.imc.org/nameprep/>. Please feel free to use the web form 
or, better yet, grab the source and run it locally. By all means 
report any bugs you find to me (you don't need to send them to the 
whole mailing list). Open source implemenatations in C or C++ would 
probably be more appreciated by the folks on this list than my 
non-optimized Perl hack^H^H^H^Hscript.

--Paul Hoffman, Director

---

From: Keith Moore <[email-masked]>
Date: Tue, 23 Jan 2001 21:27:36 -0500

Dan,

your analysis assumes that most sites are or will be using UTF-8 locally.  
this seems like quite a stretch as it is very different from current reality.
IMHO it would be much more realistic to assume that folks who are currently 
using some other non-ASCII charset in their locale will continue to do so 
for quite some time - each will convert to UTF-8 in his/her own timeframe
(if ever).  

if you make that assumption, the costs of upgrading those applications to 
convert between local representation and either ACE or UTF-8 are quite 
similar.  ACE wins not because it costs significantly less for conversion
but because it allows each site to upgrade in its own timeframe, thus
lowering the barrier to adoption of IDNs.

of course, nobody can predict the future with 100% accuracy, and reasonable 
people will disagree on how things will unfold.   but for this very reason
it makes sense to choose a IDN deployment strategy that seems likely to 
work in a variety of future scenarios - i.e. whether or not people convert 
to UTF-8.  

ACE seems to have a lower risk of failure.

Keith

---

From: "D. J. Bernstein" <[email-masked]>
Date: 24 Jan 2001 02:29:59 -0000

Paul Hoffman / IMC writes:
> differentiate the costs between upgrading to an ACE or to UTF8.

False. The web page describes in detail one ACE proposal and its costs,
and then one UTF-8 proposal and its costs. Many programs need updates
and redeployment only for ACE.

> all programs are going to need to be updated

False. The UTF-8 proposal analyzed on my web page involves redeployment
of only a few programs.

> If it has a presentation layer

Please give a clear definition of ``presentation layer,'' and explain
which of the following programs have ``presentation layers'': named;
dents; tinydns; tcpclient; telnetd; sshd; tcpd; tcpserver; fetchmail;
apache; boa; fhttpd; gn; mathopd; publicfile; roxen; thttpd; webfs; wn.
Those are some of the programs that have to be upgraded and redeployed
for ACE.

> http://www.imc.org/nameprep

Is that (1) up to date and (2) normative?

---Dan

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Tue, 23 Jan 2001 19:00:56 -0800

>Is that (1) up to date and (2) normative?

(1) Yes.
(2) What do you mean?

--Paul Hoffman, Director

---

From: Marc Tamsky <[email-masked]>
Date: 24 Jan 2001 04:29:03 -0800

>>>>> On Tue, 23 Jan 2001 21:27:36 -0500, Keith Moore <[email-masked]> said:

>  Dan,
[...]
>  ACE seems to have a lower risk of failure.

I guess we'll now need an examination of whether:

 (a) There is a need to assess the risk of failure for each proposal.
 
and

 (b) What the criteria for measuring the "risk of failure" is.

The arguments that follow are obvious, if (a) then (b) needs to be
answered.  If (a) is false, then forget (b) and stop waving that flag.

Marc.

---

From: Keith Moore <[email-masked]>
Date: Wed, 24 Jan 2001 08:30:23 -0500

>  (b) What the criteria for measuring the "risk of failure" is.

actually we don't have to do that.  we need to reach rough
consensus on a proposal.  that doesn't mean we have to have
consensus on the reasons for supporting that proposal.

however it does seem like we'd prefer to support a proposal that
seems likely to be adopted, and to be deployable.  if adoption
of an IDN solution causes people too much more pain than not doing
so, they'll be less likely to deploy it.

perhaps "risk of failure" isn't the best way to describe this,
but I hope my meaning is sufficiently clear anyway.

I suppose one could contrast this with the "cost of inelegance"
(also insufficiently defined) of having to use ACE on the wire
for an arbitrary number of years,  and the potential "cost of
multiple conversions" associated with having to convert from
ASCII to ACE and then to UTF-8 (or something else) later.
I don't deny the first cost or the potential for the second cost;
I'm just trying to state the nature of the design choice.

Keith

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 25 Jan 2001 00:38:27 +0100

At 17:19 23/01/2001 -0800, Paul Hoffman / IMC wrote:
>Applications Area Directors have said they are willing to take the work on 
>in the Apps area. Obviously, anyone from this WG would be invited to help.

I think this working group should be the one that reviews the proposal for 
what is loosely termed "the IDN protocol" (IDNA or something very like it) 
before sending it to IETF-wide Last Call.

The work of specifying how (and if) SMTP, NNTP, SNMP and so on and so forth 
make special efforts to deal with the IDN-ized DNS names is obviously a 
task for the relevant working groups, most of which are in the Applications 
area.

I would like to encourage the authors of the IDNA document to update that 
document to be consistent with the design team report and submit it to the 
working group as a proposal for this WG's final output.

--
Harald Tveit Alvestrand, [email-masked]
+47 41 44 29 94
Personal email: [email-masked]

---

From: "Brian W. Spolarich" <[email-masked]>
Date: Thu, 25 Jan 2001 14:34:02 -0500

| I think this working group should be the one that reviews the proposal
| for what is loosely termed "the IDN protocol" (IDNA or something very
| like it) before sending it to IETF-wide Last Call.

  What is the opinion of the working group on IDNA vs. IDNRA?  Paul and
Patrik have proposed two alternative ACE implementation approaches:  IDNRA
calls for an application and resolver-based approach, putting most of the
IDN work in the resolver, while IDNA calls for an application-only approach.

  One of the premises of the ACE approach is the belief that the
infrastructure will take too long to update (I'm purposefully simplifying
here) and the IDN handling needs to happen at the edge of the network.

  As I read IDNA, the authors believe that the bulk of IDN processing cannot
happen in the resolver (or perhaps immediately in front of the resolver) and
must happen solely in the application.  If that is so, is it reasonable to
assume that a significant percentage of application will be updated very
quickly?  Certainly there are market pressures that might make this so, but
I'm not convinced.  The authors do not state in IDNA why they believe IDNA
is a better approach than IDNRA, which would be a helpful clarification of
the draft.

  I briefly discussed this with Paul at IETF49 but we didn't have an
opportunity to discuss this in detail (except that we disagreed :-).

  It seems to me that in most cases, particularly with most of the
relatively simple applications, as long as the application "stays out of the
way" and passes the IDN intact to the resolver the IDN work can happen
completely in the resolver layer.  Some applications and protocols will have
to be reviewed and updated(and has already been discussed there will need to
be a review of the impact by the various WGs in the applications area), but
certainly by all means not every application.

  -bws

---

From: Keith Moore <[email-masked]>
Date: Thu, 25 Jan 2001 15:41:07 -0500

> IDN work in the resolver, while IDNA calls for an application-only approach.

I don't think there's much difference between the two; IDNA is just a 
cleaner way of expressing what would end up as more-or-less the same 
architecture as IDNRA.  Here's why:

- since ACEs are compatible with ASCII domain names, a resolver API 
  that accepts and returns ACEs already exists; it's not going to
  go away anytime soon even if other resolver APIs accepting and/or
  returning UTF-8 were to appear.

  even if the resolver API always accepts and returns ACE, there will
  inevitably be wrapper routines around the resolver which perform
  simple kinds of lookups and which accept and/or return UTF-8.  

  so either way, we'll have one interface that deals in terms of ACE 
  and another that deals in terms of UTF-8.  the only real question
  is the degree to which the latter interface will be consistent across
  different platforms.    if not, folks writing portable software
  will need to supply their own conversion routines.  however, the
  algorithms (and probably code) to do this will be published, so 
  even this isn't a huge barrier.

- many applications will need to compare IDNs with one another for 
  equivalence.  for reasons of backward compatibility they will see
  both ACE IDNs and UTF-8 IDNs, and they'll have to know whether
  they are equivalent.  in addition, applications will have to 
  convert from UTF-8 format to ACE when IDNs are used as elements 
  in protocols which do not (yet?) tolerate non-ascii DNS names, 
  and they'll have to convert from ACE to UTF-8 for display.

  so either way, there will need to be direct access to the nameprep
  and ACE<->UTF-8 conversion routines. they cannot merely be embedded
  in the resolver API.

so either way, you have pretty much the same set of interfaces, and
the same kind of layering.

Keith

---

From: "D. J. Bernstein" <[email-masked]>
Date: 25 Jan 2001 20:16:38 -0000

Brian W. Spolarich says that ``in most cases'' the IDN work can happen
``completely in the resolver layer.''

For the record: My ACE upgrade list in http://cr.yp.to/proto/idn.html
takes this idea into account. It explicitly assumes that the argument to
gethostbyname() is in UTF-8. ACE conversion is needed in gethostbyname()
but not in applications that merely call gethostbyname().

But the list is still very long, because there are many other uses of
names. For example, DNS servers and SMTP servers and web servers need to
be upgraded for ACE, so that they can compare incoming ACE names to
UTF-8 names in configuration files.

---Dan

---

From: "D. J. Bernstein" <[email-masked]>
Date: 25 Jan 2001 20:46:20 -0000

> this seems like quite a stretch as it is very different from current reality.

For extensive evidence that it's happening:

http://www.cl.cam.ac.uk/~mgk25/unicode.html
ftp://ftp.ilog.fr/pub/Users/haible/utf8/Unicode-HOWTO.html
http://czyborra.com/utf/#UTF-8

---Dan

---

From: "James Seng/Personal" <[email-masked]>
Date: Fri, 26 Jan 2001 06:58:46 +0800

Dan,

When is the last time you travel to non-English speaking country and ask
the people there when they are going to switch over to UTF-8?

-James Seng

---

From: Keith Moore <[email-masked]>
Date: Thu, 25 Jan 2001 18:19:15 -0500

> UTF-8 names in configuration files.

they'd have to be upgraded for nameprep anyway, otherwise attempts to
compare different UTF-8 representations of the same IDN would fail.

Keith

---

From: "D. J. Bernstein" <[email-masked]>
Date: 25 Jan 2001 21:24:50 -0000

I'd like to draw more attention to this example from Dan Oscarsson,
because I find it extremely worrisome.

According to Paul's nameprep script, uppercase Alpha Beta is accepted in
domain names, and matches lowercase alpha beta, not ab.

But uppercase AB---which, when properly displayed, has the same glyphs---
matches ab, not alpha beta.

I thought the point of nameprep was to prevent confusion: in particular,
to guarantee that two valid host names will match inside the computer if
they match visually. If Alpha Beta and AB are valid, they have to match.

> > Is that (1) up to date and (2) normative?

The final nameprep specification should be working software; anything
else is unnecessarily error-prone and unnecessarily difficult to review.
My question is whether your software is the current specification.

---Dan

---

From: Keith Moore <[email-masked]>
Date: Thu, 25 Jan 2001 19:00:48 -0500

> For extensive evidence that it's happening:

that's not anything of the sort.  you cited evidence that a few energetic 
folks are implementing and modifying existing tools to support UTF-8.

that's a far cry from widespread adoption of UTF-8 by real users.

Keith

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 25 Jan 2001 19:53:35 -0800

Keith Moore wrote:
> that's not anything of the sort.  you cited evidence that a few
> energetic folks are implementing and modifying existing tools to
> support UTF-8.
> 

I think it's pretty clear that UTF-8 is the direction that most
environments are heading to, if they aren't there already.

 - Solaris 7 and higher use UTF-8 for the local (Unicode) interfaces
 - Windows 2000 and CE use Unicode and UTF-8
 - MacOS 9 (or X?) and higher same
 - etc... don't discount the Linux i8n efforts either, it is an extreme
   likelihood at the very least if not a statement of direction

End-user apps that support UTF-8 include Office 2000 and of course most of
the modern web/mail front-ends

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

---

From: Keith Moore <[email-masked]>
Date: Thu, 25 Jan 2001 23:06:47 -0500

> environments are heading to, if they aren't there already.

no real argument about general direction.  my point is that users who 
have an investment in existing tools and existing files using other
charsets  may not follow that lead as quickly as some imagine.   

MIME has been the obvious direction for email for about ten years, and 
it's amazing how many people are still using user agents that don't 
support it.

Keith

---

From: "Carl S. Gutekunst" <[email-masked]>
Date: Thu, 25 Jan 2001 20:28:59 -0800 (PST)

>  - Windows 2000 and CE use Unicode and UTF-8

Again, the native character set of Windows 2000 is UCS-2, not UTF-8. There are
a few applications that partially support UTF-8, notably Internet Explorer and
Outlook Express, but that's only in some places, and only because a developer
made a conscious decision to offer that support. Run-of-the-mill Windows APIs
do not support UTF-8 and will not reliably pass it without damage.

<csg>

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 25 Jan 2001 20:52:06 -0800

"Carl S. Gutekunst" wrote:
> Again, the native character set of Windows 2000 is UCS-2, not UTF-8.
> There are a few applications that partially support UTF-8, notably
> Internet Explorer and Outlook Express, but that's only in some places,

Their DNS implementation allows UTF-8 encoded names, so I assumed that the
APIs for the network interfaces supported it as well. Mea culpa if this is
not the case (I am left wondering how names get resolved if that is truly
the situation however).

> Run-of-the-mill Windows APIs do not support UTF-8 and will not
> reliably pass it without damage.

The interface to DNS would seem to be the important part for this group,
and apparently it does support UTF-8.

Good question to ask though, does Solaris' UTF-8 support extend to the
resolver APIs?

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 25 Jan 2001 21:08:57 -0800

People are clearly moving to Unicode. Exactly which UTF they choose (8, 16,
32) is not as important, since they all can be converted to each other very
efficiently and without loss.

It is however an overstatement to say that all environments are headed
towards UTF-8. Certainly Windows, JavaScript, and Java use UTF-16; these are
not unimportant. I'm not sure about the Mac, but I believe they are also
UTF-16.

Mark

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 25 Jan 2001 21:14:57 -0800

A huge number of people in non-English speaking countries are using Unicode,
*and don't know it*. Anyone using Windows NT/2000, or Microsoft Office is,
as well as many other products. Many websites use Unicode internally, and
just convert to the user's codepage, etc.

Mark

---

From: "J. William Semich" <[email-masked]>
Date: Fri, 26 Jan 2001 00:29:00 -0500

Microsoft is moving UNICODE/UTF-8 into many other of its applications and
systems, and not just Exchange and IE:

FROM http://www.microsoft.com/Office/ORK/2000/Five/70ct_15.htm

The Microsoft Office 2000 MultiLanguage Pack includes plug-in language
features that allow users to change the language of the user interface.
When you use Office Web features in an international environment, you can
do the following:

Conduct Web Discussions in any available language 
All collaboration data is sent over the wire in Universal Character Set
Transformation Format 8-bit (UTF-8). UTF-8 is an ISO standard for
transmitting character codes for all languages.
-------------------------

FROM http://www.microsoft.com/TechNet/win2000/dnsover.asp

Unicode character support. While conventional DNS server implementations
limit allowed characters to the set described in RFC 1123, this
implementation of DNS accommodates a wider range of characters—including
the UTF-8 character encoding (RFC 2044), a superset of ASCII and a
translation of the UCS-2 (or Unicode) character encoding. This saves
administrators the chore of renaming devices using a non-strict set of
characters on a machine-by-machine basis. 
------------------------------------

FROM http://msdn.microsoft.com/library/psdk/winui/richedit_9d80.htm

Rich Edit 3.0 includes the following features in addition to the features
provided by Rich Edit 2.0. 
...
UTF-8 RTF 
This is recommended for cutting, copying, and pasting operations. This file
format is more compact than ordinary RTF, faster, and compatible with
Unicode. 

And many more ....

Bill Semich

>do not support UTF-8 and will not reliably pass it without damage.
Bill Semich
President and Founder
WorldNames, Inc.
http://www.worldnames.net
[email-masked]

---

From: "Edmon Chung" <[email-masked]>
Date: Fri, 26 Jan 2001 00:39:33 -0500

While I am pro-non-ACE-as-final-solution, I must say that although UTF8 is
being deployed readily, some implementations such as Win2000 as I have
pointed out in my known issues document
(http://www.openidn.org/issues.html), is sometimes a bit shaky especially
when used to package DNS requests.

Edmon

---

From: Keith Moore <[email-masked]>
Date: Fri, 26 Jan 2001 00:45:32 -0500

> efficiently and without loss.

And since an ACE is just another encoding of Unicode, you can add ACE to
that set.
 
> towards UTF-8.

And if there's not a single preferred encoding of Unicode that's being
widely supported, using ACE for IDNs makes about as much sense as anything.
The fact that ACE doesn't happen to use the 0x80 bit doesn't strike me
as a particularly good reason to rule it out  - especially when (for a 
carefully-designed ACE) the encoding will often be more efficient than 
either UTF-16 or UTF-8.

If UTF-8 is optimized for ASCII-compatibility, the C language (so
that for instance NUL-terminated strings still work), and to
minimize the amount of state that must be maintained while decoding; 
ACE can be optimized for space-efficiency, compatibility with protocols 
that were designed for ASCII-only DNS names, and (with nameprep) 
ease of comparison.  It's not as if one is right and the other wrong.
They are different encodings to accomodate different sets of 
transition issues.

Keith

---

From: Keith Moore <[email-masked]>
Date: Fri, 26 Jan 2001 01:00:30 -0500

> just convert to the user's codepage, etc.

sure.  but just because someone is using microsoft office that happens
to use some encoding of Unicode doesn't mean they can use it to edit 
a "plain text" config file that expects IDNs to be in UTF-8 format.

---

From: "James Seng/Personal" <[email-masked]>
Date: Fri, 26 Jan 2001 14:47:27 +0800

Mark,

I didnt argue about adoption of Unicode. Most people do use it without
realising it. Yes.

I am arguing about adoption of UTF-8.

-James Seng

----- Original Message -----
From: "Mark Davis" <[email-masked]>
To: "James Seng/Personal" <[email-masked]>; "D. J. Bernstein"
<[email-masked]>; <[email-masked]>
Sent: Friday, January 26, 2001 1:14 PM
Subject: Re: [idn] UNIX moving to UTF-8

> > > > this seems like quite a stretch as it is very different from
current
> > reality.

---

From: Patrik Fältström <[email-masked]>
Date: Fri, 26 Jan 2001 09:43:18 +0100

>and apparently it does support UTF-8.

No, DNS have very little to do with what this group is doing!

This group is about internationalization of domain names.

   paf

---

From: Patrik Fältström <[email-masked]>
Date: Fri, 26 Jan 2001 09:37:45 +0100

At 14.34 -0500 01-01-25, Brian W. Spolarich wrote:
>The authors do not state in IDNA why they believe IDNA

My take is that it is better to do this in the application as the 
domainname is not only used in DNS but also in application protocols 
like SMTP. If we "only" fix the DNS protocol, and applications 
interaction with DNS, then we have yet another large problem to 
solve, and that is how to encode domainnames which are used as 
protocol parameters in each of the protocols using it.

By doing nameprep and encoding in the application, the same "output" 
of this transformation can be used in all communication on the net.

Also, if an application is not updated, regardless of how ugly 
leakage is, the user _can_ use the encoded string directly in the 
applications which today use as protocol parameter domainnames. I.e. 
I personally see this as an important feature just because 
replacement of software will not be as fast as some people might 
think.

I can add some more text on why it is better to have transformation 
in the application, but not a comparison between the two proposals. 
They are just I-D's and when one of them (or none) is published, the 
other (or both) will go away, and the reference to the non-existing 
document is no longer making sense.

    Patrik

---

From: Patrik Fältström <[email-masked]>
Date: Fri, 26 Jan 2001 09:40:00 +0100

>UTF-8 names in configuration files.

This is exactly why IDNA say that the encoding into an ACE should be 
done on a higher level in the application, and not in the resolver.

   paf

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 26 Jan 2001 06:42:45 -0800

I am not at all arguing against using an ACE. I agree that if the particular
ACE in question is well-defined, and fits DNS constraints better than UTF-8,
it is perfectly appropriate to use it.

All I meant to point out is that
(a) Unicode is in wide, and rapidly increasing use,
(b) the particular form of Unicode that is used depends on the environment,
it is not exclusively UTF-8.

Mark

----- Original Message -----
From: "Keith Moore" <[email-masked]>
To: "Mark Davis" <[email-masked]>
Cc: "Eric A. Hall" <[email-masked]>; "Keith Moore" <[email-masked]>; "D.
J. Bernstein" <[email-masked]>; <[email-masked]>
Sent: Thursday, January 25, 2001 21:45
Subject: Re: [idn] UNIX moving to UTF-8

...

---

From: Steve Hanna <[email-masked]>
Date: Fri, 26 Jan 2001 10:18:11 -0500

Java uses 16-bit Unicode internally and UTF-8 for most external
exchanges (such as class files). This has been true since JDK 1.0.
Support for adding other encodings has been present since JDK 1.1. Not
that it makes much difference, but I wanted to set the record straight.

-Steve

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 26 Jan 2001 08:18:10 -0800

Just to set the record straight,
- Java uses a variant of UTF-8 in class files.
- That variant is not conformant to the UTF-8 spec.
- It is only for internal use within the "compiled" class files and in
serialization. (And while it is theoretically possible for a non-Java
program to pick apart serialized Java classes, it would be extremely rare.)
- UTF-8 is supported by Java for interchange, of course, but the API all
uses UTF-16 interfaces.

Mark
----- Original Message -----
From: "Steve Hanna" <[email-masked]>
To: "Mark Davis" <[email-masked]>
Cc: "Eric A. Hall" <[email-masked]>; "Keith Moore" <[email-masked]>; "D.
J. Bernstein" <[email-masked]>; <[email-masked]>
Sent: Friday, January 26, 2001 07:18
Subject: Re: [idn] UNIX moving to UTF-8