Subject: RE: [idn] pool deadline imminent

From: "Russ Rolfe" <[email-masked]>
Date: Mon, 16 Jul 2001 11:06:26 -0700

Martin Duerst wrote:

> At 17:46 01/07/12 +0000, Adam M. Costello wrote:
>> I didn't really understand until this morning that any proposal that 
>> doesn't have seven supporters by tomorrow will be out of the running.

> Just for the record: If anybody is working on a UTF-8 based
> (or even ACE/UTF-8 mixed) solution, they herewith all have
> my support (I won't tell them one-by-one).

I know this is late, but was unable to reply until now.  UTF-8 based
solutions also have my support.

Regards, Russ Rolfe
Microsoft -- [email-masked]

-----Original Message-----
From: Martin Duerst [mailto:[email-masked]] 
Sent: Thursday, July 12, 2001 7:49 PM
To: idn working group
Subject: Re: [idn] pool deadline imminent

At 17:46 01/07/12 +0000, Adam M. Costello wrote:
>I didn't really understand until this morning that any proposal that 
>doesn't have seven supporters by tomorrow will be out of the running.

Just for the record: If anybody is working on a UTF-8 based
(or even ACE/UTF-8 mixed) solution, they herewith all have
my support (I won't tell them one-by-one).

Regards,   Martin.

---

From: *click* <[email-masked]>
Date: Mon, 16 Jul 2001 14:27:52 -0400 (EDT)

Better late then never...
I am in support of any UTF-8 based solution and am willing to help
contribute if anyone is working on such a draft.

-Bill

---

From: Keith Moore <[email-masked]>
Date: Mon, 16 Jul 2001 15:59:02 -0400

> contribute if anyone is working on such a draft.

without details of such a solution, seems like such a statement is
fairly meaningless...or at least difficult to evaluate.

---

From: David Hopwood <[email-masked]>
Date: Tue, 17 Jul 2001 14:01:53 +0100

-----BEGIN PGP SIGNED MESSAGE-----

> >> doesn't have seven supporters by tomorrow will be out of the running.

Did that only apply to ACE proposals? I've been reading the list archives
to get an idea of where the group is up to, and I have some ideas that
haven't been proposed in any of the current drafts.

> > my support (I won't tell them one-by-one).

I'm working on such a solution; for lack of a better name I'll call it
IDN-8. It is ACE + UTF-8, but with ACE restricted to only the places
where it is needed: RFC 2822 headers and SMTP commands/responses.
UTF-8 is used almost everywhere else, including on the wire for DNS
queries. The most important changes that need to be made are to resolvers,
although applications can link with a compatibility library so that they
work even if the resolver hasn't been changed.

In practice ACE names will leak, and so there are various measures to
minimise any problems caused by that. nameprep is used for
canonicalization and to define which names are valid. The exact choice
of ACE is not critical: maximum lengths are mainly determined by
the efficiency of UTF-8. New or updated software is required to
understand UTF-8 everywhere (including RFC 2822 headers and SMTP
commands/responses).

Can I take it that there is sufficient support for a solution along
these lines that it is worth me writing it up as a formal I-D?

- -- 
David Hopwood <[email-masked]>

Home page & PGP public key: http://www.users.zetnet.co.uk/hopwood/
RSA 2048-bit; fingerprint 71 8E A6 23 0E D3 4C E5  0F 69 8C D4 FA 66 15 01
Nothing in this message is intended to be legally binding. If I revoke a
public key but refuse to specify why, it is because the private key has been
seized under the Regulation of Investigatory Powers Act; see www.fipr.org/rip

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Jul 2001 18:39:30 -0400

> where it is needed: RFC 2822 headers and SMTP commands/responses.

which is out of scope for this working group.

> work even if the resolver hasn't been changed.

for your  proposal to be credible you need to explain how (for instance)
how a legacy email client talking to a legacy resolver can lookup an ACEd
IDN if the on-the-wire protocol uses raw UTF-8.

and for that matter, you should explain why using UTF-8 on the wire in
the DNS query provides any significant benefit over more compact encodings
such as DUDE.  nobody has explained this yet, beyond some vague appeals 
to idealism.

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Jul 2001 19:09:11 -0400

> 1) DUDE isn't particularly compact, at least in comparison to UTF-8 or some
> of the other ACE alternatives.

sorry, I misread some figures.
 
> 2) Does "idealism" include wanting to simplify the system so one form of
> presentation encoding is in use?

if the operational need for backward compatibility is ignored, yes.

> 3) What implements DUDE now?  What implements UTF-8 now?

since nameprep is a necessary part of any UTF-8 scheme, nobody implements
either one...or at least the number of deployed implementations are so 
small relative to the number of things that need to be upgraded that there
is no practical difference in deployment.
 
> Put me in the camp of those who see similarities between these discussion
> and the NAT/end-to-end holy wars.

me too, I suppose - since both arguments are being made for entirely 
practical reasons.  but the comparison isn't exactly illuminating.

Keith

---

From: "David R. Conrad" <[email-masked]>
Date: Tue, 17 Jul 2001 15:58:48 -0700

Keith,

>such as DUDE.  nobody has explained this yet, beyond some vague appeals

Not choosing sides, but:

Rgds,
-drc

---

From: "David R. Conrad" <[email-masked]>
Date: Tue, 17 Jul 2001 16:24:57 -0700

Keith,

>if the operational need for backward compatibility is ignored, yes.

There are other operational needs beyond backwards compatibility.  One of 
these is providing useful functionality that meets market demands in a 
timeframe the market wants.

>since nameprep is a necessary part of any UTF-8 scheme,

Nameprep has not, as far as I know, been necessary to implement UTF-8 in 
(say) an operating system, an MUA, a web browser, etc.

>nobody implements

I would think rewriting (say) the operating system, the MUA, the web 
browser, etc. to understand DUDE at the same level at current UTF-8 support 
would imply a difference in deployment.

Don't you worry about replicating the Coloured Book insanity?

Rgds,
-drc

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Jul 2001 20:26:45 -0400

> timeframe the market wants.

Absolutely.  But this also favors an ACE solution.

True, you can provide a UTF-8 solution sooner to the market - but only if 
you pretend that the various things out there that already use UTF-8
do so in an acceptable manner.  For instance, you have to ignore the 
need for IDNs lookups to work even after the IDN has been transcribed 
by a human or transcoded into a different character set encoding.

> (say) an operating system, an MUA, a web browser, etc.

No, you don't need nameprep to implement UTF-8 lookups.  You just need
it before UTF-8 lookups (or any kind of lookup using the 10646 repertoire) 
will work well.

> would imply a difference in deployment.

I would think that also if it were relevant.  But in fact there's no need to 
rewrite an operating system, web browser, etc.  Applications  that already
know how to input and display 10646 characters (regardless of encoding) 
need only small changes to use an ACE for IDN lookups, and they will need
those changes in any case due to (a) the need to do nameprep and (b)
the need to interoperate with other legacy applications.  

And there's absolutely nothing that prevents a vendor from shipping an app 
that uses DUDE on legacy operating systems.

> Don't you worry about replicating the Coloured Book insanity?

There have been so many different sets of coloured books (with different
kinds of insanity) that I don't know which one you mean.

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 17 Jul 2001 22:21:47 -0400

> Sorry.  Showing my age.  UK Coloured Book naming system that reversed the
> labels (at least according to 1034/1035 specification).  Despite programs
> and gateways designed to reverse the UK names, they leaked and resulted in
> lots of annoyance and confusion.

ah yes.  (oddly enough, that particular kind of colored book hadn't occured
to me, even though I'd written code to deal with it in a mailing list manager)

no matter what solution is chosen, we'll have leakage (in both directions) 
between legacy environments (that in general can only deal with ASCII names)
and environments that are explicitly written to deal with IDNs.   the major 
difference between the proposal camps is in how often the names leak, and 
what breaks when the names leak.  

> I am worried that a similar thing will happen with folks using ACE --
> people will (say) cut and paste a (utf-8 representation of a) name from an
> application into something that is supposed to do lookups (based on ACE)
> and those lookups will fail because a conversion from UTF-8 to ACE didn't
> happen.  To the user, this won't make sense since could look identical.

we can't avoid that case no matter what solution.  most legacy apps won't
be able to deal correctly with IDNs that are input in native format, period.

Keith

---

From: "David R. Conrad" <[email-masked]>
Date: Tue, 17 Jul 2001 19:01:00 -0700

Rgds,
-drc

P.S. To be clear, I really am not taking sides.  I find all the options 
offensive (:-)).

---

From: "D. J. Bernstein" <[email-masked]>
Date: 18 Jul 2001 06:46:56 -0000

The ACE myth is that we can safely start using ACE IDNs before all the
browsers, mail clients, etc. have been modified.

Here's a counterexample, elaborating on David Conrad's messages. This is
an ACE failure involving a hypothetical ACE IDN, a hypothetical ACE
browser, a real mail client under a real UTF-8 xterm, and a real MTA.

I see a mailto URL in my browser. I don't like the browser's built-in
mailer, and changing the mailto handler is inconvenient, so I simply
copy-and-paste the address into my mail client. The address looks fine.
I type the message and send it. (This is a very common situation.)

The message bounces. In contrast, it would have gone through if the
domain name owner hadn't tried to use IDNs. ``Don't use IDNs!''

What went wrong is that the browser displayed the address as Unicode
characters. The address then passed through the (CTEXT UTF-8, although
UTF8_STRING would be better) copy-and-paste mechanism, the UTF-8 xterm,
the mail client, and the 8-bit-clean MTA, none of which converted back
to ACE. The resulting UTF-8 DNS lookup produced NXDOMAIN.

You might try to help the mail go through by also providing UTF-8 IDNs.
But we've been told again and again that this doesn't work. ``What about
mail clients that reject 8-bit bytes? What about the default behavior of
UNIX gethostbyname()?'' If we do the software upgrades to make this work
then we can simply use UTF-8 without ACE.

These failures can't occur when there are no ACE applications. They also
can't occur when there are no non-ACE applications. But they can and
will occur if we deploy ACE when there are _some_ ACE applications.

As I've commented before, moving to ACE (even without the subsequent
switch to UTF-8) appears to be much more expensive than moving directly
to UTF-8. See http://cr.yp.to/proto/idn.html for details. The ACE myth
has been the only counterargument, and it simply isn't true.

---Dan

---

From: Keith Moore <[email-masked]>
Date: Wed, 18 Jul 2001 11:28:24 -0400

> browsers, mail clients, etc. have been modified.

I don't think anyone has claimed that it's perfectly safe to do so -
only that there's less risk of leakage of native IDNs into environments
than can only deal with ASCII if we don't *deliberately* introduce 
native IDNs into protocols that were specified to deal with ASCII names.

> browser, a real mail client under a real UTF-8 xterm, and a real MTA.

Your example assumes that the browser will decode the ACE in the
mailto: URL when copying the URL to the paste buffer, when in fact it's 
much safer to leave it as ACE.  I won't claim that the browser vendor
will always do the right thing (there are plenty of counterexamples
to that idea)  but at least it is possible for them to do something 
sensible.

Keith

---

From: "Adam M. Costello" <[email-masked]>
Date: Thu, 19 Jul 2001 00:00:53 +0000

"D. J. Bernstein" <[email-masked]> wrote:

> This is an ACE failure...

In your scenario you paste a UTF-8 string into an IDN-unaware
application and it doesn't work.  I don't see how this is an ACE
failure.  It has nothing to do with ACE.

> These failures can't occur when there are no ACE applications.

False.  If the application is IDN-unaware but accepts 8-bit domain
names, then it will be able to look up the UTF-8 name if and only if the
old resolver API accepts UTF-8 queries.  We can debate whether it should
or not, but that issue is orthogonal to the issue of whether ACE exists
or not.

I think we all understand that ACE can serve to bridge IDN-aware and
IDN-unaware applications only when the IDN-aware applications know
that they have to use ACE (either because they know they're talking
to an IDN-unaware peer, or because they're talking via an IDN-unaware
protocol).  When the domain names are transferred out-of-band (via
cut-and-paste, for example), then the ACE-encoding might not happen.

Users who transfer domain names out-of-band will have to learn that
feeding UTF-8 to old applications is asking for trouble, and that's
equally true whether ACE exists or not.

I think there has been enough discussion to establish who favors ACE and
who doesn't, and you have no chance of dissuading the ACE supporters.  I
suggest that your time would be better spent working on UTF-8 standards
than on trying to prevent ACE standards.

AMC

---

From: "D. J. Bernstein" <[email-masked]>
Date: 19 Jul 2001 01:17:09 -0000

Adam has been telling us that UTF-8 IDNs are bad because using them may
cause failures in the short term, i.e., before the software upgrade is
complete. He's been claiming that ACE is better in the short term:

   Suppose I am considering getting an IDN for my domain. With ACE, this
   will make [some displays better, some displays worse] but nothing
   will actually break (mail will get through, web pages will load, etc)

This example shows that he's wrong. Some mail _won't_ get through if he
switches to an ACE IDN before all the software supports ACE.

Adam M. Costello writes:
> I don't see how this is an ACE failure.

You don't see that it's a failure? Or you don't see that the procedure
succeeds if the ACE IDN is replaced by a normal domain name?

Pasting an address from a browser into a mail client is a very common
operation. I do it all the time. I'll be extremely unhappy if it breaks.

> you have no chance of dissuading the ACE supporters

We'll see. I think that the ACE myth has been a big part of the ACE
advertising campaign on this mailing list. I find it interesting that
you're trying to abort the discussion of this example.

---Dan

---

From: "Adam M. Costello" <[email-masked]>
Date: Thu, 19 Jul 2001 01:56:51 +0000

"D. J. Bernstein" <[email-masked]> wrote:

> You don't see that it's a failure?  Or you don't see that the
> procedure succeeds if the ACE IDN is replaced by a normal domain name?

Of course it's a failure, but it's not an ACE failure, it's an IDN
failure.  The exact same failure would happen for any IDN in this
scenario regardless of whether ACE existed or not.

But there are other scenarios (like replying to a message, which is
even more common than cutting and pasting addresses) where ACE will not
fail, and any other encoding might (often would, given the popularity of
sendmail and BIND).

It's undeniable that ACE is more backward-compatible, and fails in fewer
instances, than any 8-bit encoding.  The only disagreement is how much
that additional backward compatibility is worth.  Some think it's worth
very little.  Others think it's worth a lot.  Fine.  Let some people
work on standards for ACE, and some people work on standards for UTF-8,
and if you're right about ACE being worthless, then the ACE standards
will get ignored.

AMC

---

From: "D. J. Bernstein" <[email-masked]>
Date: 19 Jul 2001 03:33:41 -0000

Adam M. Costello writes:
> The exact same failure would happen for any IDN in this scenario

Right! That's the whole point. IDNs, including ACE IDNs, aren't safe to
deploy until the software upgrade is complete.

The ACE myth is that ACE IDNs are safe. In fact, people who try using
ACE IDNs in the short term will encounter failures. You keep complaining
about this aspect of UTF-8 IDNs; but it's true for ACE IDNs too.

(By the way, all the programs I have in mind already handle UTF-8 IDNs.
But I agree that there are other programs that need upgrades.)

> instances, than any 8-bit encoding.

Actually, that claim requires a careful analysis.

What you're neglecting to take into account is that ACE is substantially
more expensive to deploy than UTF-8. See http://cr.yp.to/proto/idn.html.
ACE requires many more programs to be changed.

I agree that short-term deployment of ACE IDNs would break _fewer_
existing procedures than short-term deployment of UTF-8 IDNs. But those
procedures will be broken for _longer_ with ACE.

You were claiming before that ACE wouldn't cause any failures. ``Nothing
will actually break,'' you said. So you could ignore the question of how
long the ACE upgrade would take; zero times anything is zero. However,
without the ACE myth, the analysis is substantially more difficult.

---Dan

---

From: "Adam M. Costello" <[email-masked]>
Date: Thu, 19 Jul 2001 04:31:48 +0000

"D. J. Bernstein" <[email-masked]> wrote:

> What you're neglecting to take into account is that ACE is
> substantially more expensive to deploy than UTF-8.

If you're right, then it won't be deployed, so don't worry about it.

> You were claiming before that ACE wouldn't cause any
> failures. ``Nothing will actually break,'' you said.

I overstated it.  I was wrong.  Sorry.  But you agree that ACE will
break fewer things in the beginning than an 8-bit encoding.  I think
that's very important for the successful rollout of IDNs, so I'm going
to work on ACE.  You are free to believe that I'm wasting my time.

AMC

---

From: David Hopwood <[email-masked]>
Date: Thu, 19 Jul 2001 06:45:58 +0100

-----BEGIN PGP SIGNED MESSAGE-----

> scenario regardless of whether ACE existed or not.

No - it works for just-send-UTF8 (which I assume is Dan's point [*]),
and it also works for ACE + UTF-8 proposals in which either an ACE or a
UTF-8 name can be passed to the resolver (including IDN-8). IDN-8 pays
great attention to fixing some other types of cut and paste failure as
well; I consider any proposal where cut and paste doesn't work properly
to be seriously deficient.

[*] I don't agree with Dan that just-send-UTF-8 is an acceptable solution,
    but I do certainly agree that ACE-only proposals are not.

> fail, and any other encoding might (often would, given the popularity of

IDN-8 will work with sendmail and BIND. (I'll describe the details in
another post; please bear with me on that.)

> instances, than any 8-bit encoding.

Encodings don't succeed or fail; proposals do. So, which ACE proposal?

It isn't true in general that proposals using ACE as the primary encoding
fail in fewer instances than proposals where UTF-8 is preferred. IDNA, for
example, fails in some cases where IDN-8 works, including the example
given by Dan, and including any other situation where a UTF-8 name leaks
to an application using ACE. (Note that UTF-8 will be a very commonly used
character encoding - probably *the* most common - regardless of which
proposal is adopted, so it is impossible to completely prevent such leakage.)

If you count displaying an ACE name to a human user as a failure (which
I do, although it is less serious than a lookup failure), then ACE-only
proposals fail in many more cases.

- --

---

From: "D. J. Bernstein" <[email-masked]>
Date: 19 Jul 2001 06:07:08 -0000

The following diagram may be helpful:

           2002       2003       2004       2005       2006
          +----------+----------+----------+----------+----------+
   ACE   A|**********|**********|********* |********  |******    |
         B|          |          |          |          |          |
         C|XXXXXXX   |XXXXXXXXXX|XXXXXXXX  |XXXXXX    |XXXX      |
          +----------+----------+----------+----------+----------+
   UTF-8 A|******    |***       |*         |          |          |
         B|XXX       |XX        |          |          |          |
         C|XXXX      |XXX       |X         |          |          |
          +----------+----------+----------+----------+----------+

The X's represent mail reply failures: row B for built-in replies, row C
for copy-and-paste. The *'s in row A represent annoying IDN displays.
All the numbers are guesses; we need further analysis.

The ACE myth is that there are no X's in the ACE boxes. What's true is
that there are no X's in row B in the ACE boxes.

Adam M. Costello writes:
> But you agree that ACE will
> break fewer things in the beginning than an 8-bit encoding

Fewer procedures, yes---fewer lines in the above picture---but this
doesn't necessarily mean that the _number_ of failures will be smaller,
even if you look only at the 2002 box. Count the X's!

What you're neglecting to take into account this time is that UTF-8
support is already far past ``the beginning.'' That's why the rows of
X's for UTF-8 are relatively short.

Many programs can send mail to UTF-8 addresses, for example. They'll
work fine in the copy-and-paste situation I described, if we use UTF-8
IDNs rather than ACE IDNs.

---Dan

---

From: "Adam M. Costello" <[email-masked]>
Date: Thu, 19 Jul 2001 06:33:34 +0000

David Hopwood <[email-masked]> wrote:

> and it also works for ACE + UTF-8 proposals in which either an ACE or
> a UTF-8 name can be passed to the resolver (including IDN-8).

It works iff the old resolver API can resolve names encoded using the
8-bit encoding used by the clipboard when the mailto: URL was copied
from the browser.  Whether the old resolver API can or should do this is
a debatable question whether ACE exists or not.  The operation in this
scenario is going to succeed or fail regardless of whether this name is
represented using ACE or not.  These are what I meant when I said it's
not an ACE failure, but an IDN failure.

> > It's undeniable that ACE is more backward-compatible, and fails in
> > fewer instances, than any 8-bit encoding.
>

I wasn't talking about proposals, I was talking about domain names
encoded one way or another.  Take any situation where an ACE name fails
to work, replace the name with a corresponding 8-bit encoding, and it
still won't work.  But there are many situations where an ACE name
succeeds while an 8-bit name name would fail.  Therefore ACE is a useful
tool to have in your toolbox.  I think you agree, since you advocate a
system (IDN-8) that includes ACE.

> It isn't true in general that proposals using ACE as the primary
> encoding fail in fewer instances than proposals where UTF-8 is
> preferred.

I don't think I've ever advocated an ACE-only approach, or even argued
that ACE should be preferred.  I've merely argued that ACE is a good
thing to have as an option, for interoperating with old protocols and
old software.

> If you count displaying an ACE name to a human user as a failure
> (which I do, although it is less serious than a lookup failure), then
> ACE-only proposals fail in many more cases.

I'd count it as an annoyance or a glitch, but not a failure.  The
primary purpose of a domain name is to identify a network resource and
allow it to be accessed.  Being pretty/memorable/meaningful is very
desirable, but not essential.

AMC

---

From: Dave Crocker <[email-masked]>
Date: Thu, 19 Jul 2001 00:06:43 -0700

At 11:46 PM 7/17/2001, D. J. Bernstein wrote:
>...

The premise behind transition approaches that use special encoding, on top 
of an existing infrastructure that is unmodified, is that arbitrary pairs 
of participants can safely use the scheme, without waiting for 
infrastructure changes. The comparison, then, is against a scheme that 
requires deeper changes, to the infrastructure.

In this case, the question is about changes to the DNS server 
infrastructure.  UTF8 requires changes to them.  ACE does not.

Hence ACE can become usable far quicker than a raw binary approach.

This of course does not mean that ACE will work for everyone immediately. 
But, then, neither scheme will cause me to be able to type or read kanji.

On the other hand, if I DID understand kanji, then someone with a domain 
name that uses that character set would be accessible to me, subject only 
to my upgrading my own system.  Neither they nor I would have to wait for 
DNS server infrastructure upgrades.

With UTF-8, we would have to wait on the server operators to make the upgrade.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.273.6464

---

From: Martin Duerst <[email-masked]>
Date: Thu, 19 Jul 2001 16:47:27 +0900

At 01:56 01/07/19 +0000, Adam M. Costello wrote:
>Let some people

Adam - Some people working on UTF-8 standards (or trying to suggest
such work) have been told that this would not be possible to happen
in this WG. Nobody on the ACE side has been told thus. This is quite
asymmetric.

Another aspect where the situation is not symmetric is that
on the ACE side, there is disagreement about which ACE to take,
but everybody agrees with IDNA, because there is really not that
much to it. On the UTF-8 side, it's clear what UTF-8 is, but
there are a wide range of options of how to flag it (from
'just send' to 'new class'). I.e. the decision tree looks
quite different on each side. This makes it much more difficult
for any specific UTF-8 proposal to gather consensus. On the ACE
side, the fact that there is about the same disagreement as to
what ACE to take as there is as to what UTF-8 flagging is best
is just hidden behind IDNA.

Regards,   Martin.

---

From: Martin Duerst <[email-masked]>
Date: Thu, 19 Jul 2001 15:56:08 +0900

At 11:28 01/07/18 -0400, Keith Moore wrote:
>much safer to leave it as ACE.

It could as well assume that the mailto: uri was in UTF-8 from
the start, or was converted to that by a browser from an IRI.

Regards,   Martin.

---

From: Martin Duerst <[email-masked]>
Date: Thu, 19 Jul 2001 17:16:13 +0900

Dave - What is your guess as to how quickly servers will be
upgraded, as compared to how quickly applications will be
upgraded?

My idea is that servers would be upgraded quite quickly,
much more quickly than applications.
The number of DNS server software offerings is not that
large, and the people responsible for the software are
in one way or another following this WG.
Any serious isp would very quickly get requests to upgrade
the DNS servers they advertise in their DHCP, and would
upgrade. (maybe with the exception of some isp in the US
outback)
Local resolver software can also be upgraded easily,
because it's a single upgrade for a system, and because
most probably patches will be provided rather quickly by
the OS vendors or by third parties.
Of course DNS servers that authoritatively serve IDNs
can be assumed to be upgraded also.

For applications, it's much more difficult. In some cases,
you will be lucky, but in other cases, you won't. If you
depend on an application, you may have to wait forever.
Also, application upgrades may cost much more.

So upgrading the DNS side isn't that much on the critical
path as far as I can see.

Regards,   Martin.

---

From: Patrik Fältström <[email-masked]>
Date: Thu, 19 Jul 2001 11:52:04 +0200

> to ACE. The resulting UTF-8 DNS lookup produced NXDOMAIN.

In the IDNA proposal we have specified that an application which handle
IDNA (like you seem to say that the browser is) only support IDNA if also
the content of the clipboard is converted to ACE before the domainname is
passed to other applications (which doesn't handle IDNA/ACE solution).

  paf

---

From: "Brian W. Spolarich" <[email-masked]>
Date: Thu, 19 Jul 2001 08:04:24 -0400

| My idea is that servers would be upgraded quite quickly,
| much more quickly than applications.

  Martin, I suspect you are being extremely over-optimistic on this point.
:-)

  I've had discussions with a number of WALID clients who are interested in
deploying IDN resolution capabilities and have tried other, so-called
"server-side" approaches.  Those that in particular focus on requiring ISPs
to make changes have met with limited success in getting them to do so.

  There is a tremendous amount of inertia in the deployed nameserver
infrastructure.

  -bws

---

From: Eric Brunner-Williams in Portland Maine <[email-masked]>
Date: Thu, 19 Jul 2001 08:42:39 -0400

> There is a tremendous amount of inertia in the deployed nameserver

Yeah. Last I heard the ISPs and their users were going to be stuck with the
com/net/org troika under a single vendor _forever_  ;-)

Inertia isn't absolute, and our responsibility is to more than just the
present comfortable.

Any change in the IPR status?

Eric

---

From: "Brian W. Spolarich" <[email-masked]>
Date: Thu, 19 Jul 2001 08:52:18 -0400

| Inertia isn't absolute, and our responsibility is to more than just the
| present comfortable.

  Our responsibility is to engineer solutions to problems that may actually
be deployable.

| Any change in the IPR status?

  That's not my bailiwick.  You should contact our IPR person if you have
particular questions.

  -bws

---

From: Keith Moore <[email-masked]>
Date: Thu, 19 Jul 2001 09:02:59 -0400

thanks, that's a useful illustration.

I find it interesting that it assumes that MUAs will be upgraded more
quickly with UTF-8 than with ACE.  I would assume just the opposite - 
that UTF-8 is more immediately disruptive because the failures that 
it does cause are more numerous, and thus a larger barrier to adoption. 

Also, the X's in row C look larger than the spaces in row B - but 
built-in replies are surely far more common than copy-and-paste replies.

The graph also does not consider other kinds of failures due to UTF-8 -
e.g. failures of mail (including nondelivery reports) to be delivered
because one of the intermediate systems either couldn't deal with
the UTF-8 or because it munged the UTF-8 to the point that subsequent
systems couldn't make sense of it.

Precisely because of the copy-and-paste problem, I expect that
vendors will quickly upgrade MUAs to accept native IDN input 
regardless of whether ACE is used as the on-the-wire encoding.
Whether a large percentage of their customers upgrade quickly is
a different question-  but even if a few percent require that 
functionality, that's sufficient motiviation to fix the product.

Again, I don't think this is a either/or choice for this group-
most of us are convinced that we are going to need ACE for some
applications (at least during a transition period), and that we 
are going to use UTF-8 on the wire within other applications.  
Those choices will be made by other working groups.  

Keith

---

From: Keith Moore <[email-masked]>
Date: Thu, 19 Jul 2001 09:09:53 -0400

> much more quickly than applications.

I think server products will be upgraded quickly, but in many
parts of the world they will be deployed slowly.  It's not just
the ISPs that need to deploy new servers for this to work;
it's anyone that runs a DNS cache.

Keith

---

From: Keith Moore <[email-masked]>
Date: Thu, 19 Jul 2001 09:14:45 -0400

> much to it. 

yes, but I don't think that the choice of which ACE will be hugely
contentious.  It's just that we're still looking at various tradeoffs,
and people have been making productive suggestions that illuminate
those tradeoffs.  Most people who support IDNA would be willing to
accept any of the better ACEs.

Keith

---

From: "D. J. Bernstein" <[email-masked]>
Date: 19 Jul 2001 13:17:07 -0000

Dave Crocker writes:
> infrastructure.  UTF8 requires changes to them.

Wrong again. UTF-8 poses no problems for DNS servers. As Mark Andrews
says: ``UTF8 does not require a server upgrade.''

---Dan

---

From: Patrik Fältström <[email-masked]>
Date: Thu, 19 Jul 2001 15:54:47 +0200

> in this WG. Nobody on the ACE side has been told thus. This is quite

Reason for this is that ACE is backward compatible with the standards we
already have, such as RFC 2821 and RFC 2822. If we have a solution which
points at UTF-8 as the transport, then the RFC's specifying the application
layer protocols have to be updated -- and proper handshake (if needed) and
downgrade to ascii encoding (if needed) is defined.

I.e. the problem with using non-ascii in the labels in DNS, is that it is
really easy to get what I would call leakage into the application layer
protocols which can not handle other characters than ASCII.

How to handle Unicode and how to reach the goal of using UTF-8 encoding in
the future is something I rather see each wg which have knowledge about the
specific protocol. Regardless of how easy or hard it is.

  paf

---

From: "Edmon" <[email-masked]>
Date: Thu, 19 Jul 2001 11:01:12 -0400

> specific protocol. Regardless of how easy or hard it is.
But where do we start? Is not this a good opportunitiy and place to begin?
I am very much for a ACE+UTF solution and I think we should work on both
before we submerge ourselves into a possibly irreversible journey into ACE
forever.  Once a specifcation for eventually using UTF in DNS is set, it
will layout a framework for all the protocols that depend on DNS to start
upgrading.  The point I am trying to make is that DNS has to move first, it
has been from the begining, or else, why would the other protocols have
depended on DNS at all?

Edmon

---

From: <[email-masked]>
Date: Thu, 19 Jul 2001 11:22:36 -0400 (EDT)

Getting ISPs to upgrade to DNS that supports UTF-8 is simple.
Patch BIND now, wait for the next remote root exploit which is
followed by a worm.

-Bill

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 19 Jul 2001 11:11:06 -0500

> I.e. the problem with using non-ascii in the labels in DNS, is that
> it is really easy to get what I would call leakage into the
> application layer protocols which can not handle other characters
> than ASCII.

Will you *PLEASE* stop saying this! The use of separate APIs and message
types prevents this from happening.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 19 Jul 2001 11:07:58 -0500

> it's anyone that runs a DNS cache.

Where are these caches?

Half are on the back-end, half are at the client. Which half won't
upgrade? Which half doesn't want the technology they are trying to use?

---

From: John C Klensin <[email-masked]>
Date: Thu, 19 Jul 2001 11:55:04 -0400

> Patch BIND now, wait for the next remote root exploit which is

Sigh.

Folks, this is important.  Let's get real.

Unless recent consolidations have changed things dramatically --
and in directions other than the one my experience would
predict-- there are basically two kinds of ISPs: the ones who
are very careful and conservative about infrastructure software
updates and the ones who don't expect to be in business for much
longer (a few manage to fall into both categories, of course).

Bug-fixing patches to BIND (or anything else) that address a
security problem are installed quickly iff their content is
addressed specifically to that problem.  If they contain a lot
of irrelevant stuff, the software providers take a lot of flak.
Complete new versions that ISPs are expected to install on the
basis that it will be good for them, but without specifics, get
even more resistance.  (Those closer to BIND maintenance cycles
should confirm this or tell me I'm crazy.)

In general, anything that requires ISPs to make changes to
servers that impact all of their users or customers (and, if
they are in the hosting business, customers of their customers)
is going to be deployed when it is clearly in the ISP's business
interests, and/or after very careful and leisurely testing, or
never.

If you want fast deployment, select something that doesn't
require ISPs to make changes.  There may be fewer of them than
there are end users, but the latter can much more easily decide
to install software upgrades that meet their particular
perceived needs.  And don't depend on tricking the ISPs into
installing something -- the good ones are hard to trick and they
collectively represent a very large portion of the marketplace.

    john

---

From: Keith Moore <[email-masked]>
Date: Thu, 19 Jul 2001 12:16:17 -0400

> upgrade? Which half doesn't want the technology they are trying to use?

your network people must be more responsive than the ones I'm used to.

Keith

---

From: <[email-masked]>
Date: Thu, 19 Jul 2001 12:24:08 -0400 (EDT)

Actually, I should have been more clear.
There are currently three major versions of BIND
in common usage. 4.x, 8.x, and 9.1.x None of which
will be patched for new features. The 9.x series is still under
evaluation at several ISPs that I know of. If there are features that
need to go into the DNS server, I would think the best time to get them
in is now, before upgrades to 9.x have been completed. The security issue
will just push things forward, the real issue is 9.x is on the upgrade
schedule.

-Bill

---

From: John C Klensin <[email-masked]>
Date: Thu, 19 Jul 2001 12:42:46 -0400

--On Thursday, 19 July, 2001 11:28 -0500 "Eric A. Hall"
<[email-masked]> wrote:

> Yes let's.
> How many users of HTTP 1.1 didn't upgrade?
> 
> How many users of LDAP v3 didn't upgrade?
> 
> Give me a break. The people that want it will upgrade. Duh.

I think that was my point.  The people that want "it" will
upgrade, if they can.   The ISPs who run the servers may not see
"want" the same way.   So, once again, if you want rapid
deployment, pick a solution that leaves the upgrading in the
hands of those who need the functionality --the users in this
case-- rather than entertaining fantasies about what ISPs and
other server operators will do.

   john

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 19 Jul 2001 11:28:28 -0500

Yes let's.

---

From: Keith Moore <[email-masked]>
Date: Thu, 19 Jul 2001 12:50:49 -0400

> types prevents this from happening.

will you *PLEASE* stop making bogus assertions like this?

in most cases, it's simply not feasible to separate the use of IDNs from 
the use of ASCII DNS names and negotiate or convert at the boundaries.

for instance, in the particular case of email, we tried that with 
8-bit-SMTP and it failed miserably.

Keith

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 19 Jul 2001 12:01:27 -0500

> in most cases, it's simply not feasible to separate the use of IDNs
> from the use of ASCII DNS names and negotiate or convert at the
> boundaries.

Give me one example where it fails. Just one.

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 19 Jul 2001 11:57:59 -0500

> So, once again, if you want rapid

I am getting very frustrated over these specious claims. Sorry if it comes
out harsh. Let's think this through though, please.

1) We have the owners and operators of IDNS. They will want to
   upgrade their servers to get the full benefits of an UTF-8
   IDN approach. The only operators who are not likely to
   upgrade immediately are speculators; they won't be getting
   much traffic anyway.

2) We have the corporate user community. They will want to
   upgrade sooner or later, as business dictates. For those
   orgs who maintain an international presence, this will be
   sooner rather than later. For those orgs who are purely
   local, and who never ever do anything in any international
   market, later will be fine (ACE will work for the 1% of
   domain names which are IDN). Not a problem.

3) We have the *international* end-user community and their
   ISPs. What ISPs in any foreign market is going to insist
   that their end-users must continue using ASCII??????????

4) We have domestic ISPs. See #2 above. Some of these will
   be upgrading sooner than others. For the lazy xenophobes
   who think the rest of the world should just learn English,
   they may never upgrade. Chances are they aren't upgrading
   much of anything else, either. They will continue using
   ACE, until they have no customers left.

If we use absurd numbers and give each of those camps 25% share, then we
are looking at 75% cache upgrades. More importantly, we are looking at
100% penetration in the markets which are affected by the technology.

Who gives a fsck if the lazy xenophobes in the other 25% don't upgrade?
They aren't involved with international markets anyway.

Also keep in mind that once the application protocols start to embrace
UTF-8 IDNs, that the bar will raise further.

It would also be a good thing for the Internet for such an upgrade to be
required. Note that it isn't really required -- lazy xenophobes can
continue to use ACE -- but that it is required for maximum efficiency and
also likely to be required for future protocols.

Upgrading the servers is not a preventative issue. Please, let's drop 
this FUD and move on.

---

From: John C Klensin <[email-masked]>
Date: Thu, 19 Jul 2001 13:07:51 -0400

--On Thursday, 19 July, 2001 11:57 -0500 "Eric A. Hall"
<[email-masked]> wrote:

> I am getting very frustrated over these specious claims. Sorry
> if it comes out harsh. Let's think this through though, please.
> 

Eric,

As in our earlier (offline) conversation about use of TCP
options to negotiate the use of alternate character sets, it is
clear that our assumptions and experience are enough different
that we are not communicating well and, more important, unlikely
to convince each other of anything.   I see the landscape
differently than you do.  I have a certain amount of experience,
with and in major ISPs, that conditions my vision.   You can
reasonably disagree with either the value of my experience or my
conclusions (as I disagree with your conclusions).  But even
categorizing my experience and/or conclusions as "specious" is,
IMO, probably a little much.

regards,
    john

---

From: *click* <[email-masked]>
Date: Thu, 19 Jul 2001 13:29:55 -0400 (EDT)

> other server operators will do.

I am in operations, I have spoken with the people who decide when
DNS servers get upgraded at a few regional ISPs, this is not a fantasy.

-Bill

---

From: Keith Moore <[email-masked]>
Date: Thu, 19 Jul 2001 13:08:21 -0400

> Give me one example where it fails. Just one.

I'll give two.

1. an email message gets delivered to a flat file which is read by 
a user agent.  the process that delivers the message has no idea
whether the user agent can support native IDNs or not, and this
is subject to change at any time. 

2. an HTML document is saved in a flat file on a disk.  the process
that saves the file has no idea whether every browser that will ever
be asked to read that file can properly support IDNs.

and you're dreaming if you think that folks are going to add
explicit IDN negotiation even into those protocols that have
the ability to negotiate options.  option negotiation 
tends to add complexity and additional failure modes that 
make diagnosing and fixing bugs more difficult (because they
increase the complexity of the case analysis). the more
options you add to a protocol, the worse it gets.
on-the-fly format conversion adds it own set of problems.

Ketih

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 19 Jul 2001 12:50:07 -0500

> > Give me one example where it fails. Just one.

You gave me two application failures, not protocol failures.

Try again.

> is subject to change at any time.

You're assuming that the headers won't have to signify content, and that
the SMTP folk won't consider this in protocol design. Considering the
store-and-forward nature of mail, this is absurd.

> be asked to read that file can properly support IDNs.

Compliant HTML has a version tag.

> on-the-fly format conversion adds it own set of problems.

Such as?

---

From: Keith Moore <[email-masked]>
Date: Thu, 19 Jul 2001 14:09:43 -0400

> You gave me two application failures, not protocol failures.

delivering to a file is a protocol.  not all protocols involve
direct interaction between the producer and the consumer.
 
> store-and-forward nature of mail, this is absurd.

Considering the store-and-forward nature of mail, it's absurd to think
that you can use message headers for interactive feature negotiation.

> Compliant HTML has a version tag.

Version tags don't help if the browser doesn't understand the version.
(we realized that MIME-Version was essentially useless only after
RFC 1341 made it mandatory.  by the time we realized this, it was
too late to deprecate it - too many UAs refused to recognize MIME without it.)

> > on-the-fly format conversion adds it own set of problems.

It introduces errors that are difficult to trace and to fix.
It adds complexity (for example into MTAs) that shouldn't be
modifying their payloads.  It forces intermediaries to be aware
of their payloads' formats when those formats should be transparent.

Keith

---

From: Keith Moore <[email-masked]>
Date: Thu, 19 Jul 2001 14:12:01 -0400

> How many users of LDAP v3 didn't upgrade?

lots of folks are still using HTTP 1.0 and LDAP v2.
HTTP at least took great pains to allow interoperation between the two.

and changing either HTTP or LDAP doesn't break most of the other
protocols on the Internet.

Keith

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 19 Jul 2001 13:23:59 -0500

> lots of folks are still using HTTP 1.0 and LDAP v2.

Because they didn't find the upgrade necessary/compelling/whatever. That
is exactly the point.

> protocols on the Internet.

Neither does the dual-mode approach

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 19 Jul 2001 13:22:51 -0500

> direct interaction between the producer and the consumer.

If there is no direct interaction, there has to be an out-of-band
negotiation. However you do this is entirely up to you. Until you come up
with a mechanism that works reliably however, I would suggest that SMTP
should not be upgraded.

The dual-mode model helps you deliver on whatever you decide you can|wan't
to do. It won't break anything on it's own.

The example you cited is a broken SMTP implementation, and has absolutely
nothing to do with the DNS model.

> too late to deprecate it - too many UAs refused to recognize MIME

Thanks for proving my point; the only time that there will be bleed-over
is when the applications fail. You can use your MIME experience to prevent
this from happening again.

The dual-mode protocol won't break anything. Applications that implement
it poorly will.

> It introduces errors that are difficult to trace and to fix.

ACE even moreso.

> of their payloads' formats when those formats should be transparent.

If you want to insist that MTAs should track message formats on a
per-destination basis, that is your call.

---

From: Keith Moore <[email-masked]>
Date: Thu, 19 Jul 2001 15:00:07 -0400

> negotiation. 

no there doesn't.

> However you do this is entirely up to you. Until you come up

and this is one of the reasons why introduction of IDNs into email
will not be dependent on an SMTP option.

> to do. It won't break anything on it's own.

it will break things merely because it requires additional software changes.
changes invariably introduce new bugs.

> nothing to do with the DNS model.

precisely! it's an example of why dual mode models cause problems - 
because implementations will fail to implement them correctly.

> this from happening again.

nothing in the MIME experience helps us to keep applications from being
buggy.

> The dual-mode protocol won't break anything. Applications that implement

poorly-implemented applications are a given, I'm afraid.
and the more complex we require them to be, the more likely they are to
be buggy.

Keith

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 19 Jul 2001 14:18:15 -0500

> > The dual-mode model helps you deliver on whatever you decide you
> > can|wan't to do. It won't break anything on it's own.
> 
> it will break things merely because it requires additional software
> changes. changes invariably introduce new bugs.

> and the more complex we require them to be, the more likely they are to

These are all arguments against ACE transliteration (and IPv6 and just
about everything else for that matter).

The only way that UTF-8 IDNs will get into the legacy path is if a
particular implementation goes out of its way to use them where it
shouldn't. It will have to explicitly call the new API and then explicitly
pass data that is not supported by the application protocol. This scenario
is completely unrealistic.

You are essentially arguing that if an application makes two unrealistic
errors then UTF-8 IDNs will break the application.

Your initial comment was:

 | in most cases, it's simply not feasible to separate the use of IDNs
 | from the use of ASCII DNS names and negotiate or convert at the
 | boundaries.

You have yet to prove this point. On the contrary, I have shown that it
requires explicit effort for problems to occur, and even then the problems
are not due to anything inherent with the protocol itself.

Please, let's drop this FUD and move on.

---

From: Patrik Fältström <[email-masked]>
Date: Thu, 19 Jul 2001 20:46:01 +0200

>    upgrade immediately are speculators; they won't be getting

Eric, it is sad to use the same arguments on your mail as you are using on
mine.

As evidence that you are completely wrong I take Hotmail as an example.
This site is one of the largest in the world regarding free email to people
from all over the world. It is not only something which is targeting the
US. One can even choose when registering "Hotmail på svenska" which means
"Hotmail in Swedish" in English.

Now, do you think Hotmail can handle other characters than US-ASCII in
headers (header encoding of for example my last name "Fältström")? Do you
think Hotmail can handle other characters than US-ASCII in even the body of
a message?

Let me tell you the answers: No, and No.

Is Hotmail one site which "won't be getting much traffic anyway"?

Definitly not.

Hotmail is one of the largest problems we have on the Internet at the
moment just because they (a) don't follow the standards and (b) still get
lots of usage from users.

Now, why is this an argument for my discussion about "leakage"?

Just because Hotmail which have not even implemented non-US-ASCII
characters in MIME correctly, which have been around for quite some time,
what is the chance that it will implement suddenly non-US-ASCII characters
for IDN and addresses correctly?

The reason is that software companies which mainly is based in the US don't
have the experience (the programmers don't) of using non-US-ASCII
characters and because of that the implementations are "good enough" for
the largest portion of the market of the Internet -- which is the US.

Yes, I have tried to buy software from several US companies to Sweden, and
it is hard, very hard, because there is just no understanding of what
internationalization means.

And, I in Sweden still uses almost the same script as people in the english
speaking world. We only have three more characters which are "weird". It is
MUCH worse for people which do not even use the same kind of characters.

   paf

---

From: David Hopwood <[email-masked]>
Date: Thu, 19 Jul 2001 22:32:42 +0100

-----BEGIN PGP SIGNED MESSAGE-----

> passed to other applications (which doesn't handle IDNA/ACE solution).

That is practical in *some* situations: when the application uses a widget
that is specifically designed to display a domain name (or email address
or URI). In fact I've partially implemented a name entry widget in Java that
does exactly that. It is *not* practical for domain names appearing in free
text. Therefore, in order for cut and paste to work properly, ACE names must
be restricted as far as possible to a minimal set of well-defined contexts -
which is what IDN-8 does.

- --

---

From: Keith Moore <[email-masked]>
Date: Thu, 19 Jul 2001 17:44:48 -0400

> is completely unrealistic.

This is complete and utter nonsense, and I'm not going to dignify it
further by continuing the discussion.

---

From: David Hopwood <[email-masked]>
Date: Thu, 19 Jul 2001 23:13:47 +0100

-----BEGIN PGP SIGNED MESSAGE-----

>   There is a tremendous amount of inertia in the deployed nameserver

If, for example, a proposal required upgrading of the authoritative
servers *for zones that include IDNs* (but not necessarily any other
servers or proxies), I don't see any reason why that couldn't be done
quickly. Such zones have sufficient incentive to upgrade, if necessary.

- --

---

From: C C Magnus Gustavsson <[email-masked]>
Date: Fri, 20 Jul 2001 00:36:36 +0200 (MET DST)

> As evidence that you are completely wrong I take Hotmail as an example.

I agree. Whatever experience I have tells me that a lot of companies won't
bother with internationalization. They will upgrade for security reasons
but as we know, recently discovered vulnerabilities usually only affect
newer versions. (E g, recent exploits have been for BIND 8 while a lot of
people still run BIND 4.)

I'd just like to point out that scenarios like this is one of the 
arguments a number of people have tried to present in support of UTF-8. 
Software vendors won't test their products enough or prioritize fixing
problems which is why we'll most likely get problems with "leakage" of
ACE. And that's precisely the reason why we need to move toward the Single
Encoding in all protocols...

/Magnus

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Thu, 19 Jul 2001 16:45:28 -0700

>quickly. Such zones have sufficient incentive to upgrade, if necessary.

That's not what most of the proposals at hand require. They require 
that any name server that might do IDN queries be upgraded. This 
means the DNS caching servers run at every large company and every 
ISP. This probably also includes every DNS server that gets hit a 
recursive query (even if that server has no IDNs itself), such as the 
root servers and all the TLD servers.

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: "D. J. Bernstein" <[email-masked]>
Date: 20 Jul 2001 00:14:19 -0000

Patrik writes:
> headers (header encoding of for example my last name

More details, please.

What exactly shows up on the Hotmail mail-reading web page if the
message contains your name in UTF-8, without further encoding?

What exactly shows up on the Hotmail mail-reading web page if the
message contains your name in Moore's =?...?= MIME encoding?

What about replying to messages with 8-bit bytes? Is there any problem?
What about 8-bit bytes in domain names?

Of course, Hotmail has central control over their software and can
rapidly deploy a new version, so this is all much less important than
what traditional mail clients do, but it's still an interesting example.

---Dan

---

From: "Adam M. Costello" <[email-masked]>
Date: Fri, 20 Jul 2001 00:18:40 +0000

Edmon <[email-masked]> wrote:

> The point I am trying to make is that DNS has to move first, it has
> been from the begining, or else, why would the other protocols have
> depended on DNS at all?

DNS does not have to move first.

Assertion:  Very few applications/protocols interact with DNS directly;
the vast majority go through a resolver.

Consequence:  Most applications/protocols, while they might be
influenced by the presence or absence of a resolver API that supports
UTF-8, have no way to know (and hence no reason to care) whether the
resolver and DNS server communicate using ACE or EDNS/UTF-8.

Do you doubt the assertion?  Do you doubt that the consequence follows
from the assertion?

If you want to work on extending DNS, then do so.  You don't need to
sell us on the urgency or importance.

AMC

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 19 Jul 2001 20:17:44 -0500

> that any name server that might do IDN queries be upgraded.

This is true only if the operators of that domain want to provide UTF-8
IDNs to their users. If they are happy with the level of service provided
by ACE, they can stick with it.

NB: the choice is ACE or ACE-with-UTF-8. The operators do not have to
upgrade if they don't want to. No upgrades are *required* (the word you
used in your comment) by either the operators or end-users.

I'm not sure I understand where the confusion on this point comes from,
nor why it is proving to be so hard to dispel.

> This probably also includes every DNS server that gets hit a

This is accurate. Every 3LD has the authority to create their own IDN
labels, so every TLD must be able to process the resulting queries.

However, it is important to point out that most of these servers are
upgraded frequently, so the threat of lag is not very high. The only
question would be if the TLD operators want to support UTF-8 IDNs or not
(very probable, IMO). We can certainly ask them if they would refuse to
implement such a technology if it were the standard.

---

From: David Hopwood <[email-masked]>
Date: Fri, 20 Jul 2001 04:38:59 +0100

-----BEGIN PGP SIGNED MESSAGE-----

> that any name server that might do IDN queries be upgraded.

Well, UDNS does. In general it depends on why the servers are being
upgraded. To support labels longer than 63 octets (or names longer than
255 octets), yes, it's necessary to negotiate use of a new DNS version,
which requires updates to all servers in the query path. AFAICS it should
not be necessary to require updates to all servers in the path for any
other reason, though (for example, it is not necessary in order to change
the algorithm by which names as encoded on the wire are canonicalised for
matching against names that come from zone files).

So, I think pretty much any reasonable design could be tweaked to have
the property that names that fit the octet limits in their wire-encoded
form, are resolvable with at most changes to authoritative servers for
zones with IDNs. What, if anything, am I missing?

- --

---

From: Patrik Fältström <[email-masked]>
Date: Fri, 20 Jul 2001 09:31:20 +0200

>> headers (header encoding of for example my last name

(a) I registered at Hotmail, using a Microsoft Internet Explorer Browser
just to be sure I used a Microsoft product talking with a Microsoft service
(b) It asked about my name, and I entered "Fältström" as my last name
(c) I composed an email, and entered Swedish characters in both the subject
and the body of the email
(d) I sent the mail to a host where I could look at exactly what arrives
over SMTP from the Hotmail system

Result:

(1) The From: header look like this (well, not really, because I now use a
Mac, and there is a translation of ISO 8859-1 to the local Mac charset
which makes things not exactly what they should, but anyway):

From: "Patrik F?ltstr?m" <[email-masked]>

I.e. my last name include 8bit characters, and it is the ISO 8859-1 ones
which are sent directly on the wire.

No header encoding of the name.

(2) The Subject: have the same problem as the From: header.

Subject: Nu s?nder jag en r?ksm?rgÂs

ISO 8859-1 without any header encoding at all.

(3) The MIME-type is not correct. It is the following:

Content-Type: text/plain; format=flowed

No charset parameter.

The SMTP server I have do handle the ESMTP extension 8bitmime so I didn't
expect ant content-transfer-encoding.

I.e. my conclusion of working with charset issues in Internet protocols now
for 10 years is that all people should try to change their native language
to something which is not english for a couple of months and see how bad
implemented internationalization and handling of non-US-ASCII is in
applications. This regardless of whether the transport is 8bit clean or not.

     Patrik

---

From: Martin Duerst <[email-masked]>
Date: Fri, 20 Jul 2001 14:31:47 +0900

At 16:45 01/07/19 -0700, Paul Hoffman / IMC wrote:
>That's not what most of the proposals at hand require. They require that 
>any name server that might do IDN queries be upgraded. This means the DNS 
>caching servers run at every large company and every ISP. This probably 
>also includes every DNS server that gets hit a recursive query (even if 
>that server has no IDNs itself), such as the root servers and all the TLD 
>servers.

Is that 'all the TLD servers' or 'all the TLD servers that allow IDNs'?
Would it be possible to have it so that e.g. if Switzerland (.ch) would
decide they don't need IDNs anywhere, they could just prohibit it?

Regards,   Martin.

---

From: Martin Duerst <[email-masked]>
Date: Fri, 20 Jul 2001 14:37:25 +0900

Hello Brian - I acknowledge the amount of inertia. And I would be
very inert myself as an ISP at this point in the game. But if the IETF
work is done, and e.g. here in Japan, JPNIC officially announces things,
and the trade press runs articles and surveys (they do tons of
statistics here in Japan, there is a magazine that comes out
every month telling you at what time of the day which access
points of which provider are how buisy), how many ISPs would
resist how long?

Regards,   Martin.

---

From: Martin Duerst <[email-masked]>
Date: Fri, 20 Jul 2001 14:22:05 +0900

>passed to other applications (which doesn't handle IDNA/ACE solution).

The clipboard doesn't have any protocol to distinguish whether the
target will be IDNA-enabled or not. [Most clipboards now have mechanisms
to distinguish the encoding, but ACE won't work with that. It's also
possible to create new clipboard categories, but again old applications
won't pick that up.]

Of course, IDNA could require that the clipboard is always in ACE,
but then imagine the following: User A sees a cool website, and
copies the address (mostly a domain name) into a mail to user B.
The paste operation will suddenly show some random garbage rather
than the actual address. The user will retry the cut/paste a few
more times, with the same result. It will then decide that the
browser or the mailer, or both, are buggy, and will type in the
address by hand.

Also, assume that the clipboard is tagged and that the receiving
mailer actually gets the ace and understands it. Should that then
go into the mail body as characters, as the user expects? Or would
idna prescribe that it goes there as ace? Do we want idna to require
behavior that users will clearly see as buggy and not accept, and
that therefore reasonable implementers won't implement? And if
we take the napkin as another kind of clipboard, is there a way
to get the user to somehow change the things she sees to ACE?
Is this what we are working on this effort for? In my eyes,
clearly not.

By the way, what if user B receives the web address in characters
(rather than ace)? Well, they'll put it into their browser, and
it will either work or not. If it doesn't work, they'll blame the
browser. They may find out that they have to upgrade if they want
this functionality. To most, it should not be too difficult to
understand that to make browsers work with non-ASCII addresses
where they have up to now only worked with ascii addresses needs
some change somewhere. It's definitely much easier to understand
than the clipbord bug, and it's something that users will have
to learn anyway, *whatever approach we take*.

So my proposal would be: Don't try to be too intelligent with
the clipboard.

Regards,   Martin.

---

From: "Eric A. Hall" <[email-masked]>
Date: Fri, 20 Jul 2001 12:24:42 -0500

> decide they don't need IDNs anywhere, they could just prohibit it?

The root and TLD servers would only need to understand the extended label
type for UTF-8 IDN referrals (for downstream 4LD labels) to work.

If there is a need for a particular (root|TLD) zone to hold UTF-8 IDN
entries in particular (essentially, does it need to hold UTF-8 NS records
for delegation purposes), then the servers for that TLD would need to
support the ACE-plus-UTF8 dual-mode protocol.

It is feasible for each TLD to incrementally support the dual-mode IDN
according to their own i18n schedule. They would have to support the
extended label type at a minimim, but they would not need to run
ACE-plus-UTF8 immediately (COM can run without it forever, for example).

---

From: Patrik Fältström <[email-masked]>
Date: Sat, 21 Jul 2001 11:30:39 +0200

--On 01-07-20 14.46 +0900 Martin Duerst <[email-masked]> wrote:

> As Dan and others have shown, that's really easy also for ACE.
> Even if we get all software doing it as planned, users will leak.

But if ACE leak to a non-IDN aware application the user will be able to use
the domainname.

  paf

---

From: Patrik Fältström <[email-masked]>
Date: Sat, 21 Jul 2001 11:31:18 +0200

--On 01-07-20 14.22 +0900 Martin Duerst <[email-masked]> wrote:

> target will be IDNA-enabled or not.

It depends on what operating system you are using. Saying that the protocol
doesn't exist is wrong.

  paf

---

From: Martin Duerst <[email-masked]>
Date: Sat, 21 Jul 2001 21:50:11 +0900

Hello Patrick,

Sorry, I wrote:

The text in [] speaks exactly about what you said above.
Maybe I'll try to reword:

The DNS protocol has lot's of bells and whistles that could be
used to distinguish utf-8 and ace. Clipboard protocols do have
some bells and whistles that could be used to distinguish utf-8
and ace.

The DNS doesn't currently have a protocol to distinguish utf-8
and ace. Clipboards currently don't have a protocol to distinguish
utf-8 and ace.

Could you agree with that?

Regards,   Martin.

---

From: Martin Duerst <[email-masked]>
Date: Sat, 21 Jul 2001 21:41:02 +0900

>But if ACE leak to a non-IDN aware application the user will be able to use

Sorry Patrick, I was most probably too short. What I meant is:

Even with ACE, it's possible that real IDNs (i.e. non-ACEd) leak
to non-IDN-aware applications, via the user, some billboard or
napkin,... I.e. we cannot guarantee that all non-IDN-aware
applications only get ACE input.

Regards,   Martin.

---

From: Keith Moore <[email-masked]>
Date: Sat, 21 Jul 2001 11:01:02 -0400

> and ace. Clipboards currently don't have a protocol to distinguish

Either of these could be true; both could be misleading.
The devil is in the details.  

Can you make a DNS extension that distinguishes between utf-8 and 
ace which provides consistent results between utf-8 and ace, doesn't 
significantly delay either type of query, and doesn't require 
upgrades to intermediaries before it works?  Maybe, but we're still 
waiting for an existence proof.

For some particular OS or GUI environment, can you define a clipboard
interface such that non-IDN-aware applications  get the ACE form
of a domain name and IDN-aware applications get a native form of that
domain name, for arbitrary text that is copied into the clipboard
(not just domain names by themselves, but email addresses, URIs, 
and anything else that uses domain names)   Maybe, but I have my
doubts. 

(Do we really want a clipboard interface that treats almost any string 
with dots in it as a domain name?)

Keith