Subject: RE: [idn] Reality Check

From: "Russ Rolfe" <[email-masked]>
Date: Sat, 14 Jul 2001 02:43:35 -0700

>> we can't expect people to upgrade their applications en masse. nor=20
>> would this make good operational sense.

> We are expecting them to do it for ACE (so that it is usable anyway).
> We might as well let them move to recent state-of-the-art when they
> do it.

All thou I have been most of the time a silent observer, I agree here
with Eric, and would to like to state the following:

1. Since IDN is a new feature being added to the internet, most people
   will accept the premise that to use a new feature, you will most
   likely need to upgrade to some application or system that=20
   supports the use of this feature.

2. Before the Internet became the default carrier for E-mail, many
   companies had some internal system that allowed electron mail.
   With these E-mail packages people could send e-mail to some=20
   people but not to all the people they wanted.  As the internet
   became more accesses able to companies, people began to=20
   request the ability to send to others out side of their systems.
   A lot of these companies had to install whole new systems in order
   to accommodate this request.  And while doing so, a lot of us had
   to live with two or more E-mail systems to do our work until every
   thing was right.

   I see no difference in this analogy then what we are trying to do
   with the introduction of an IDN standard.  With the introduction
   of a UTF-8 based IDN, those who wish to use it will upgrade
   to the applications and systems that can handle it.  While in
   the process of upgrading, people may not be able to communicate=20
   with everyone they want to via the new feature, but if it is
   important enough, these people will use both IDN and non-IDN
   address until every they need to communicate can, but because
   it is important all parties will upgrade there systems ASAP.

3. I have read a lot of comments on this list that state that ACE
   is just an intermediate step to help us get to a UTF-8 solution.
   IMHO most intermediate steps become long term solutions (for
   example how many of you have seen supposable TEMPORARY taxes =20
   that have actually been repealed -- not very many).

Thus I would suggest that a UTF-8 solution that in the beginning may
cause some a little irritation, will actually help us get to the UTF-8
solution quicker than using ACE as an intermediate step.  For it is
human nature to remove or change that which is uncomfortable first
before addressing things that don't seam to be causing problems.

Regards, Russ Rolfe ([email-masked])
Microsoft

---

From: "Adam M. Costello" <[email-masked]>
Date: Sat, 14 Jul 2001 21:18:16 +0000

Russ Rolfe <[email-masked]> wrote:

> ...people will use both IDN and non-IDN address until every they need
> to communicate can...

Are you arguing that ACE is unnecessary?  If so, please explain how
email is going to work.  What is supposed to happen when IDN-capable
person A sends a message to IDN-capable person B, who then forwards the
message to IDN-incapable person C, who would like to reply to A?  What
is supposed to happen when IDN-capable people send email to mailing
lists?  Are they supposed to use their non-IDN address if there are any
IDN-incapable subscribers?  How will they know?

> I have read a lot of comments on this list that state that ACE is just
> an intermediate step to help us get to a UTF-8 solution.

I don't agree with those comments.  While I can see ACE becoming less
common over time, I wouldn't expect it to completely die out in the
forseeable future.

AMC

---

From: "Soobok Lee" <[email-masked]>
Date: Sun, 15 Jul 2001 13:11:09 +0900

DUDE and AMC-ACE-Z (not RACE,ACE37) already  avoid 
using '0', 'o', '1', 'l'  in its base 32 encoding 
for security reasons. It is a nice feature. 
 
----- Original Message ----- 
From: "Martin Duerst" <[email-masked]>
To: "Soobok Lee" <[email-masked]>; <[email-masked]>
Sent: Sunday, July 15, 2001 12:54 PM
Subject: Re: [idn] IDN security and ACE leakage

> 'l' and '1' and 'I' ('ell' and 'one' and 'upper-case i') are
> almost indistinguishable. For security reasons, DNS-capable email
> programs (i.e. every email program) may display an email
> address using hexadecimal (ACE won't work).
> 
> How many programs actually do this?
> 
> Regards,   Martin.
> 
> At 12:16 01/07/15 +0900, Soobok Lee wrote:
> >Latin 'o' and Greek 'o'  are almost indistinguishable ,
> >(and I can list up hundreds of such examples.)
> >but their ACE labels  often look very different.
> >
> >For security reasons, IDN-capable email programs
> >may display an IDN email address
> >  both in its original scripts and in its ACEed form
> >  to encourage  instant verification  like this:
> >"FullName <[email-masked]>" [[email-masked]]
> >
> >It's more secure but looks ugly. shorter ACE labels may help.
> >ACE labels are better than appended hexadecimal dump of utf8 lables
> >for this purpose.
> >
> >Soobok Lee
>

---

From: Martin Duerst <[email-masked]>
Date: Sun, 15 Jul 2001 12:54:46 +0900

Regards,   Martin.

---

From: "Soobok Lee" <[email-masked]>
Date: Sun, 15 Jul 2001 13:30:57 +0900

For 1 and 'l', choosing a good latin display font set is enough.
and the ambiguity is well known.

But, In IDN context, we have hundreds or more of such ambiguities 
that are too many to educate.

How can we distinguish  katakana 'ro-to.com' and hangul 'ma.com' 
if we haven't learn japanese and korean ?

---

From: Martin Duerst <[email-masked]>
Date: Sun, 15 Jul 2001 12:51:48 +0900

>IDN-incapable subscribers?  How will they know?

Assuming that the sender has some clue about how the address will
show up at the other end, no sender will want ACE to show up.
If mail systems don't get some functionality that will automatically
only use the ACE address if the other systems can display the actual
characters, ACE will get very bad press. So there is no real argument
here to favor ACE over UTF-8.

Regards,   Martin.

---

From: "Soobok Lee" <[email-masked]>
Date: Sun, 15 Jul 2001 12:16:43 +0900

Soobok Lee

---

From: "Adam M. Costello" <[email-masked]>
Date: Sun, 15 Jul 2001 06:31:11 +0000

Martin Duerst <[email-masked]> wrote:

> > What is supposed to happen when IDN-capable people send email to
> > mailing lists?  Are they supposed to use their non-IDN address if
> > there are any IDN-incapable subscribers?  How will they know?
>
> show up at the other end, no sender will want ACE to show up.  If
> mail systems don't get some functionality that will automatically

Suppose I have a Chinese email address and an ASCII email address, and
I want to send mail to a mailing list.  Let's compare what happens in a
world without ACE, versus a world with ACE:

In a world without ACE, if I send mail using my Chinese email address,
then all IDN-incapable subscribers will be completely unable to reply to
me.  No thanks, I'll use my ASCII address.  Looks like IDN is going to
have a hard time catching on in this world.

In a world with ACE, if I send mail using my Chinese email address,
everyone will be able to reply to me.  Some subscribers will be
IDN-capable and will see my Chinese address in its proper form, while
others will be IDN-incapable and will see the ACE.  If I use my ASCII
address things will be prettier for the IDN-incapable subscribers who
are familiar with ASCII letters, but less pretty for the IDN-capable
subscribers who are familiar with Chinese characters.  But since
everyone can reply to me in any case, I might prefer to reward the
IDN-capable folks and let the IDN-incapable folks see the ACE.  This way
I can encourage the IDN-incapable folks to upgrade without completely
cutting them off.  IDNs have an easier time catching on in this world.

AMC

---

From: "D. J. Bernstein" <[email-masked]>
Date: 15 Jul 2001 19:24:26 -0000

Adam M. Costello writes:
> IDN-incapable subscribers will be completely unable to reply

Are you saying that there are mail readers today that can't reply to a
UTF-8 address? Please identify the specific software bugs that you're
talking about.

As far as I know, there are only two relevant bugs anywhere in the
entire mail path: the UNIX gethostbyname() 8-bit bug, which in current
versions is trivially worked around with a one-line change to
/etc/resolv.conf, and the sendmail 8-bit bug.

There are certainly some programs that don't _display_ UTF-8 nicely, but
that's a separate issue. (They don't display ACE nicely either.)

---Dan

---

From: "D. J. Bernstein" <[email-masked]>
Date: 15 Jul 2001 20:30:04 -0000

> I don't agree with those comments.

I believe that the IDN WG has consensus on the following point: ``The
long-term IDN solution will encode Unicode characters as UTF-8 on the
wire.'' I find the other possibilities absurd, and I am dismayed at the
amount of time that has been wasted discussing them on the IDN list.

On 2001-05-29 I requested that the the IDN WG chairs run a poll to see
whether we do, in fact, have consensus on this point.

``I would like to hear some support on this before we run this poll,''
James Seng said, speaking as chair.

He promptly heard some support, but he did not run the poll. I asked why
the chairs were not running the poll. There was no response.

I accuse the chairs of bias in their selection of polls. They appear to
be running polls that they believe will promote their personal agendas,
and refusing to run any other polls. Does the IETF have any procedures
to fix this problem?

In a legitimate standards organization, when 2/3 of the members agree on
something, they can easily force a binding vote---not just a wishy-washy
``poll''---over the chair's opposition. They can even expel the chair;
I've never heard of that actually happening, but it's comforting to know
that the group has final control over the chair, instead of vice versa.

---Dan

---

From: "Adam M. Costello" <[email-masked]>
Date: Mon, 16 Jul 2001 00:13:19 +0000

> UTF-8 address?  Please identify the specific software bugs that you're

I'm not talking about any bugs.  I'm talking about software that
conforms to the standards.

> entire mail path: the UNIX gethostbyname() 8-bit bug, which in
> current versions is trivially worked around with a one-line change to

Out of dozens (if not hundreds) of mail user agents, and dozens of mail
transfer agents, and various resolver and DNS server implementations,
you think there are only two that rely on hostnames being what the
standard says they are?

I think that's very unlikely, but suppose these were indeed the only
two.  Then the only people who can't reply to my IDN address are UNIX
users who haven't customized their /etc/resolv.conf and anyone whose
mail would have to pass through sendmail on its way to me.  Hmmm, if I
care about getting replies, I think I'd stick with ASCII names rather
than stuff UTF-8 into message headers.  But with ACE, everyone can
reply, even people behind "buggy" (standard-conformant) software, so I'd
use that.

> wire.'' I find the other possibilities absurd, and I am dismayed at
> the amount of time that has been wasted discussing them on the IDN
> list.

I think we have consensus on "New protocols should use UTF-8 for
hostnames."  We might have strong support for "Existing application
protocols should try to migrate toward UTF-8."  I don't think there is
anything approaching consensus on "ACE will eventually be eradicated
from all protocols."

Here is one end-game that I don't find absurd:  UTF-8 everywhere
except in DNS messages, which use ACE.  DNS already uses a different
representation of domain names than everything else does:  Everything
else uses dots to separate the labels, while DNS does not, so the
resolver already has to translate back and forth.  There are very few
things that speak DNS directly (like DNS servers, resolvers, and MTAs)
and everything else could be oblivious to ACE.

Why don't I think it's important to eliminate ACE from DNS?  Because
I don't think it's even practical.  We wouldn't be able to remove ACE
support from DNS until all other message/file formats support UTF-8 IDNs
and all ACEs in all data files have been decoded.  Even if that ever
came to pass, how would we know?

Why don't I think it's important to add a UTF-8 option to DNS?  Because
ACE gets the job done, and adding a UTF-8 option would increase the
complexity of DNS servers (which would then have to handle both UTF-8
and ACE) while helping only a few entities (the ones that speak DNS
directly).  While many applications might appreciate a UTF-8 resolver
interface, most have no way to know how DNS messages are encoded, and no
reason to care.

AMC

---

From: Martin Duerst <[email-masked]>
Date: Mon, 16 Jul 2001 08:53:44 +0900

>and the ambiguity is well known.

Yes, that's the main point. The ambiguities are again well known
in the other scripts where they turn up.

>if we haven't learn japanese and korean ?

Well, the main point will be spacing. There are also other font
differences. But these are made for people who write katakana or
hangul, not for those that don't.

Regards,   Martin.

---

From: "Soobok Lee" <[email-masked]>
Date: Mon, 16 Jul 2001 10:26:10 +0900

----- Original Message ----- 
> hangul, not for those that don't.
In most cases, the problems come from the similarities, not from spacing.

katakana 'ka' and chinese letter 'power(U+529B)' look the same.
Begali numeral '4'  and latin '8' looks the same.

What if japanese or korean customers send us with their IDN-email
addresses  and we don't know japanese and korean letters and theri
spacing sematics ?

Still I cannot distinguish between Bengali numerial '4' and latin '8'.

How can we prevent malicious attempts to 
forge similiarly-looking but different domains ?

---

From: "James Seng/Personal" <[email-masked]>
Date: Mon, 16 Jul 2001 10:10:56 +0800

> wire.'' I find the other possibilities absurd, and I am dismayed at
the
> amount of time that has been wasted discussing them on the IDN list.

I speak for myself in this note. (Marc may have different opinion)

The working groups works based on ideas documented in internet drafts
and not a broad statement/claim.

> the chairs were not running the poll. There was no response.

First, there is a response: I requested you to point to an internet
draft which you are referring or write a draft yourself.

Secondly, strawpoll are not fun events which we conduct -- it is very
time-consuming. Surprisingly, I still have a day-time job.

Thirdly, strawpoll are useful, but not neccessary, when the chairs wants
to know if there is a consensus on a particular *draft*.

And last, but not the least, there is a existing announced process to
split the wg drafts into Pool W for drafts with 7 supporters and Pool T
for others. If you strongly feel about your idea, please write a draft,
gather 7 supporters, then come back to the working group with your
draft.

> and refusing to run any other polls. Does the IETF have any procedures

You are free to form any conspiracy theory you like in your personal
thoughts. But making these theory into public statement requires
substained evidences.

If anyone feels the wg chairs is act inappropriatly, a private note to
IESG would usually do the job. Given that a number of IESG and IAB
members are on this list, any wrongdoing would be note almost
immediately.

Once in a while, certain individual in the wg have louder voice than
others and repeat his opinion often enough. Fortunately, the IETF
process does not guage consensus based on decibel level or number of
emails sent. Instead, we work on Internet Drafts and consensus, ie. each
and every wg members' views are taken into consideration. A louder voice
does not make anyone more important than others.

And the wg chairs are here to assist the process, not to conduct
strawpoll for individual whims. Thank you.

-James Seng

---

From: "D. J. Bernstein" <[email-masked]>
Date: 16 Jul 2001 03:13:18 -0000

James Seng/Personal writes:
> I requested you to point to an internet draft

Your request is noted.

In the meantime, please run the poll. A two-week poll will leave enough
time for people to get back from vacation. The agree/disagree statement,
as I've told you repeatedly, is as follows: ``The long-term IDN solution
will encode Unicode characters as UTF-8 on the wire.''

I think we have consensus on this one aspect of what's going to happen.
If I'm right, then we can stop wasting time discussing silly ACE-forever
scenarios that aren't going to happen. Please stop stonewalling.

> strawpoll for individual whims

Excuse me? Many people have announced their support for UTF-8 as the
long-term solution. Only a few people have objected. When you stalled by
saying ``I would like to hear some support on this before we run this
poll,'' you immediately saw some public messages of support. For
example, here's what Bruce Thomson said:

   It is likely that we have consensus on this point, and it would be
   well worth the effort of a straw poll to establish this, so we can
   move on.

How you can call this an ``individual whim'' is beyond me.

---Dan

---

From: "Eric A. Hall" <[email-masked]>
Date: Sun, 15 Jul 2001 22:35:12 -0500

> anything approaching consensus on "ACE will eventually be eradicated

I concur on this assessment, even though the third point weakens my
arguments. Some protocols which are dead and buried are very unlikely to
be upgraded.

As an aside, it is possible to upgrade many of the legacy protocols to
UTF-8 without actually changing the protocol semantics, simply by using
out-of-band negotation features. Finger, Echo, Gopher, etc., all of them
could be upgraded. Not by this WG of course, but the work that this WG
performs will dictate how that work eventually happens.

For example, if there's an out-of-band negotiation for "use UTF-8" that is
glued onto finger, then the output can be internationalized, but the query
inputs will also need to be internationalized for maximum functionality.
This will include IDNs (with Finger in particular).

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

---

From: "James Seng/Personal" <[email-masked]>
Date: Mon, 16 Jul 2001 12:22:31 +0800

Pardon me, but I have to eat my own words. I do not think running a
strawpoll based on a broad statement as you proposed is useful to the
process.

It would be more appropriate when you have an draft (or point to a draft
which you support) and if there is sufficient support for that draft, a
strawpoll maybe conducted to evaluate consensus.

Strawpoll is useful tool for wg chairs to guage consensus of a draft
when the chairs is unable to determine consensus within the group. It is
not an "voting tool" for a statement. Neither is it actually neccessary
for any process so long the chairs can determine consensus.

Using it as an affirmation of a statement as you request would be
abusing the strawpoll. It is not only time-consuming, it is also a waste
of time for wg members if it is not neccessary.

ps: speaking for myself again, I can say it is not neccessary because I
have suspected there is some agreement (given the discussion of UTF-8 in
the wg) with your statement already. Nevertheless, the wg cannot proceed
based on ideas, we need drafts.

-James Seng

---

From: Dave Crocker <[email-masked]>
Date: Mon, 16 Jul 2001 20:29:17 -0700

>and refusing to run any other polls. Does the IETF have any procedures

Dan, you have raised similar complaints many times in recent years.  You 
have had IETF procedures and procedural documents explained to you many times.

Please follow those procedures or please refrain from disrupting working 
group processes.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.273.6464

---

From: Erik Nordmark <[email-masked]>
Date: Wed, 18 Jul 2001 23:11:29 +0200 (CEST)

Russ,

>    it is important all parties will upgrade there systems ASAP.

You omit one important issue - the possibility that upgrading the software
makes it impossible (or harder) to communicate with those that haven't
upgraded.

For instance, if as part of upgrading the user ends up changing their email
address to be an IDN email address, then the user might no longer be
able to communcate with peers that don't support IDN.

One way to avoid this is to define an ACE encoding to be used in that case.
Another would be for the user to have two different email address - one
IDN and one following current email address syntax.
In the latter case the user and/or the applications need to be able to
pick which sender email address to use in a given case.

  Erik