Subject: [idn] I fear I cannot use IDN in the next 10 years

From: Dan Oscarsson <[email-masked]>
Date: Thu, 4 Oct 2001 09:00:37 +0200 (MEST)

As the IDN list have been only focused on ACE and are now removing the
proposals that support IDN nativly in the DNS protocol, I fear that
I will not be able to use non-ascii domain names for the next 10 years.

To understand why, you have to look at how domain names are used on your
computer systems.
When I look at mine, I enter or handle domain/host names in many places,
for example: host-files, logs, web pages, text documents and databases.
In all these places the names are using my systems local native character set.
All programs handle that.
To use non-ascii names, the only acceptable way to do it is to enter them
natively in all places. Natively does not mean ACEencoded names, it means
using the native character set. My web pages will never containg ACE in
URLs, they will containg ISO 8859-1 characters just like the rest of
my web pages. My host files may not contain ACE, the must use ISO 8859-1.
To use ACE-encoded host names can never be an application specific modification.
The only viable way to do it is to convert all names when they leav the
DNS protocol into local character set. Under Unix this could be done
in the resolver libraries so that applications never need to know the
protocol used over DNS.
To allow mixed usage of ACE-encoded and native names will result in a
terrible mess. The only usable way is for all names to be nativly encoded
within a system. 

The current groups wanting ACE-encoded names do not take this into
consideration. At least I have seen no discussions about it.

As all ACE-encoded names need to be decoded into native names when
entering a system, why not use UTF-8 over the wire to support simpler
handling of names?
There are two places where ACE-encoded names are needed:
1) When talking to ancient systems.
2) When not being able to represent an UCS name using the local native
   character set.
But in all other cases UTF-8 will do and will result for systems
using UTF-8 natively that they can use dns names without any
convertion. This will never happen with ACE, no system will ever
use ACE-encoded text as native character set.

So forget about quick use of non-ASCII names because of ACE. As long as my
applications cannot accept a native name or an UTF-8 name, non-ASCII
names cannot be used. I am not going to expose my users to the
mess and confusion that ACE-encoded names will result in.

When I can register a domain name using non-ASCII characters and can
query and get it returned in UTF-8 or ISO 8859-1, and my system stop
rejecting names because they conatin non-ASCII characters, then I can
start using IDNs.

    Dan

---

From: Dave Crocker <[email-masked]>
Date: Thu, 04 Oct 2001 00:16:35 -0700

>All programs handle that.

You seem to be confused about the difference between exchange standards and 
user interface behavior.  They are entirely different things, and operate 
quite independently.

Also, as has been pointed out repeatedly, UTF8 is no more "native" than is ACE.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.273.6464

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Thu, 4 Oct 2001 10:41:49 -0700

>entering a system, why not use UTF-8 over the wire to support simpler

For the same reasons people have been repeating on this list for the 
past year. UTF-8 is the "native" encoding for a very small number of 
people. Further, even for that very small number of people, UTF-8 is 
almost never the only encoding they have. Because of that, we need to 
assume that essentially all software that takes in or displays IDNs 
will have to do some conversion before putting the name parts on the 
wire. If we are going to require conversion, it is better to do that 
conversion to something that is already compatible with the current 
DNS protocol.

Updating all of the DNS servers in the world is an option, but not 
one that makes much sense.

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 4 Oct 2001 23:26:35 -0500

From: "Paul Hoffman / IMC" <[email-masked]>
To: "Dan Oscarsson" <[email-masked]>; <[email-masked]>
Sent: Thursday, October 04, 2001 12:41 PM
Subject: Re: [idn] I fear I cannot use IDN in the next 10 years

> people.

Looking out the rear-view mirror, this is true. Looking out the
front-windshield, this is not true.

> UTF-8 is  almost never the only encoding they have.

It is the best option for getting to an "only encoding". Resistance in this
particular WG actively frustrates progress for everybody.

> Because of that, we need to

First of all, support for an UTF-8 encoding means that conversion is NOT
required for apps that generate UTF-8 natively. Secondarily, the DNS protocol
is NOT restricted to US-ASCII.

> one that makes much sense.

"all of the DNS servers" do not need to be upgraded for UDNS and similar
protocols to function properly. The only servers which need to be upgraded are
those servers where the network admin WANTS to serve IDNs as UTF-8 strings. If
an admin does not want to use UDNS, they can use ACE. We have been through
this many times.

--
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

---

From: Dave Crocker <[email-masked]>
Date: Thu, 04 Oct 2001 21:48:17 -0700

>front-windshield, this is not true.

no doubt your base of experience permits you to speak with such 
certitude.  those of us who have worked on Internet technology for less 
time, say only 20 or 30 years, find that such predictions are usually 
wrong.  sometimes right, but usually wrong.

However, in this case, it does not matter.

The physics of upgrading a large, infrastructure-critical service dictate a 
text-based encoding scheme.  Even better is that the performance (and 
storage) are not affected by the choice between UTF-8 or ACE.

It simply has no effect on significant system performance issues, other 
than transparency to the existing system... if an ACE is used.  UTF-8, on 
the other hand, will nicely break the existing system.

d/

ps.  please remember that "the system" is quite a bit more than the DNS 
servers.

----------

---

From: "Eric A. Hall" <[email-masked]>
Date: Fri, 05 Oct 2001 01:45:34 -0500

> wrong.  sometimes right, but usually wrong.

My "base of experience" gives me an ability to follow current activities
well enough to see that there is more active development for UTF-8 than
any other charset. Do you dispute this? I can infer nothing else from your
comments above.

> The physics of upgrading a large, infrastructure-critical service
> dictate a text-based encoding scheme.

Debating this point requires more context. Certainly IP, BGP and many of
the other "large, infrastructure-critical services" don't rely on
text-based encodings. DNS itself is 8-bit clean, and only uses a
text-based encoding for payload data, and that's only to support the
legacy HOSTS.TXT requirements.

What particular aspect of UTF-8 isn't "text-based" anyway? DNS itself
doesn't even use a text-based encoding (supporting 0x00-0x1F and 0x80-0xFF
natively) by the yardstick you seem to be applying.

> than transparency to the existing system... if an ACE is used. 

Transparency to users, admins and developers is also important. Think
about ACE in terms of everyday usage scenarios, particularly in markets
and development environments where people are trying to internationalize
their networks and services. The educational and implementation burden of
an ACE-only approach is massive. Worse, it is perpetually massive.

Allowing the users and developers to lower their implementation costs in a
non-disruptive form should be considered as our duty. Preventing it
without reasonable cause is pointless.

> UTF-8, on the other hand, will nicely break the existing system.

Dan J. is not arguing for UTF-8 *ONLY*. That's a non-starter. The UDNS
proposal allows for a dual-mode structure, one of which is ACE for
legacy-systems compatibility, the other of which is a UTF-8 for future
application support. They are COMPLEMENTARY.

---

From: Dave Crocker <[email-masked]>
Date: Fri, 05 Oct 2001 00:04:06 -0700

>My "base of experience" gives me an ability to follow current activities

It is "following" future events that is difficult.

>text-based encodings. DNS itself is 8-bit clean

The topic is the DNS and, apparently, you did not fully appreciate the 
postscript I placed at the end of my previous message.

>What particular aspect of UTF-8 isn't "text-based" anyway?

There is a difference between "octet" and "text".  John K's recent note 
elaborated this point.

>Transparency to users, admins and developers is also important.

See my previous comment about confusing user interface issues with protocol 
issues.

>proposal allows for a dual-mode structure

When uDNS moves from proposal to specification, the real feasibility and 
tradeoffs associated with it can be debated.  Until then, it is closer to a 
Rorschach test than a basis for technical debate.

d/

----------

---

From: "Eric A. Hall" <[email-masked]>
Date: Fri, 05 Oct 2001 02:23:05 -0500

> It is "following" future events that is difficult.

Clearly the world wants a UCS. While it is true that nobody can guarantee
that this will be 10646/Unicode, we can all be certain that it absolutely
WON'T be ASCII. What we can also be sure of is that UTF-8 will be the
favored encoding for the future that we can see.

> When uDNS moves from proposal to specification, the real feasibility
> and tradeoffs associated with it can be debated.  Until then, it is
> closer to a Rorschach test than a basis for technical debate.

Hopefully there will be some meritorious debate at that time.

---

From: Dave Crocker <[email-masked]>
Date: Fri, 05 Oct 2001 15:43:39 -0700

At 12:23 AM 10/5/2001, Eric A. Hall wrote:
>that this will be 10646/Unicode,

the world wants world peace.

such motivation is useful, but not sufficient, for achieving the desired goal.

>we can all be certain that it absolutely

what you seem to be missing is an appreciation of how often that phenomenon 
has occurred, yet proved wrong.

>Hopefully there will be some meritorious debate at that time.

that would be nice.  the current, meretricious debate is wearing thin.

d/

----------

---

From: "Eric A. Hall" <[email-masked]>
Date: Fri, 05 Oct 2001 18:47:08 -0500

Dave Crocker wrote:

> what you seem to be missing is an appreciation of how often that
> phenomenon has occurred, yet proved wrong.

My statement is supported. What I am seeing is policy direction such as
RFC2130, the protocols and formatting specifications under development
which are implementing UTF-8 as the preferred or only encoding, and the
number of platforms which are implementing UTF-8. Items 2 and 3 in
particular illustrate that UTF-8 is and will be (for the "foreseeable
future" anyway) the favored encoding for new Internet protocols (and even
for new implementations, to the extent where other charsets are only
provided as subsets of 10646/Unicode).

I'm certainly willing to change my position if you can provide evidence
that all of the above is just a bad expirement, and that NEW protocols are
in fact favoring some other encoding instead of UTF-8. Got any?

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Fri, 5 Oct 2001 17:32:57 -0700

At 6:47 PM -0500 10/5/01, Eric A. Hall wrote:
>number of platforms which are implementing UTF-8.

Every specification with a "preferred" encoding has one or more 
non-preferred encodings. Saying that UTF-8 is preferred is correct 
and irrelevant. As long as there are many popular protocols with lots 
of encoding choices, picking one of those choices for the wire format 
doesn't help any protocol except those that use that one encoding. 
Look at the popular protocols: they are almost all multi-encoding. 
Picking UTF-8 won't help any of them.

Further, remember that UDNS requires that the UTF-8 sent to the DNS 
be marked specially. The cost of adding code for a new encoding for 
DNS (ACE) and adding code for a new way of doing DNS (UDNS) is 
similar. The coder needs to think about one or the other, and both 
will be a small chunk of code.

>  Items 2 and 3 in

True, and irrelevant.

>in fact favoring some other encoding instead of UTF-8.

Then you won't change your position, even when faced with other 
better arguments about why UDNS won't work. Let's try those again:

- UDNS causes some strings that are legal in one encoding to be 
illegal in the other, and vice versa, meaning that some host names 
will be illegal part of the time, unpredictably

- UDNS requires more work to be done by authoritative DNS servers

- UDNS UTF-8 queries that fail will cause more load on the DNS

- UDNS is not compatible with DNS security

- UDNS requires that applications have the logic to send a new DNS query format

- Applications that implement the UTF-8 part of UDNS have to handle 
the inevitable errors that come from queries that bounce, have to 
recast those queries in ACE (assuming that the name is even legal in 
ACE, which some won't be), and then have to emit those queries again, 
causing more DNS traffic

- The errors caused by UTF-8 queries are the same as other legitimate 
DNS errors, meaning that applications will have to have their own 
(probably-nonstandard) logic for differentiating between expected 
errors and real errors

UDNS gives us all of these problems for the limited benefit that some 
applications don't have to implement one additional encoding for 
sending host names on the wire. Does that seem like a good balance?

--Paul Hoffman, Director

---

From: "Eric A. Hall" <[email-masked]>
Date: Mon, 08 Oct 2001 20:38:48 -0500

Paul,

First of all, I'd like to thank you for taking the time to enumerate your
many perceived technical issues with the UDNS model. These points deserve
to be fully debated and discussed.

However, I think that your comments also represent a larger issue which
needs to be resolved first. We need to come to a group consensus on the
question of whether or not a UTF-8 namespace is necessary, desirable, or
neither of those, and this needs to happen before the technical points can
be debated in appropriate context. Without consensus on this underlying
point, debates over relative costs will be debates over half-full vs
half-empty, and will get us nowhere.

> sending host names on the wire. Does that seem like a good balance?

The above is a comparison of UDNS' perceived cost relative to a benefit,
but we haven't fully discussed the benefits as of yet. Clearly, ACE has
many costs, some of which are quite high AND ongoing, although most of us
agree that some form of ACE is necessary, and is therefore worth whatever
cost may be required. What we do not agree on is the benefit that UDNS (or
a similar mechanism) would provide. Making a decision based on cost alone
is misrepresentative of the true value of UDNS, just as a decision on ACE
which was based solely on cost would have misrepresented the true value of
its benefits (being necessary, it can have almost any cost).

Personally, I believe that UDNS is also necessary, and is therefore worth
almost any cost (although as I will show, I think the cost is somewhat
lower than you do). I base this on a few key arguments:

 * BCP18's is "Official Internet Policy" which requires support for
   UTF-8 in all new protocols, and in modifications to all existing
   protocols: "lack of an ability to use UTF-8 is a violation of this
   policy; such a violation would need a variance procedure". Simply
   put, policy requires this WG to devise some kind of support for
   UTF-8, unless it can be proven unreasonable. Nobody has proven it
   to be unreasonable.

 * Without a UTF-8 DNS interface, no new protocols or applications
   can be developed that are UTF-8 clean. Instead, they will be
   UTF-8 for everything EXCEPT domain names, and in some cases this
   will be fatal. One example we have already discussed for this is
   mapping between LDAPv3 distinguished names and DNS domain names
   (mapping dc= RDNs to DNS). Failure to support UTF-8 is a heavy
   blow to such efforts, requiring tremendous amounts of development
   effort and infrastructure oversight, likely hindering the use
   and development of LDAP in the Internet.

 * Global public networking is only in its infancy; there will
   likely be many thousands of new protocols and applications which
   are developed over the next few decades (some in the IETF WGs,
   most in business, educational or personal networks, and most will
   be developed outside of the US). If those applications follow the
   Official Internet Policy, they will be UTF-8 only or use it as the
   preferred encoding. However, those applications will also be
   saddled with ACE conversion wherever they have to interact with
   domain names, meaning they cannot be UTF-8 clean. This will be
   extremely acute in non-US development environments.

 * Towards the above point, UDNS provides an optional, user-driven
   transition path from ASCII to UTF-8. UDNS allows applications to
   be written so that they only function in UTF-8 environments.
   While this may not seem a reasonable objective of this WG, this
   should be the long-term vision of what we are trying to achieve.
   We should be enabling the development of a truly international
   Internet infrastructure which is UTF-8 clean throughout, and
   UDNS provides this transitional path. Conversely, ACE does not
   provide this transition. In fact, the goal of seamless backwards
   compatibility actively hinders migration.

   Let me restate this point, since it is somewhat complex. Although
   UDNS supports a dual-mode model, once UDNS (or something similar)
   were approved, developers could begin to work on applications
   which ONLY used UDNS, without any code for ASCII or ACE. At first
   this would be small private apps, but over the course of a couple
   of decades, it would likely be most of the new apps. Without UDNS
   (or something similar), it would still be ASCII at the end of that
   timeframe. In short, we need UDNS if we are ever to drop ASCII.

 * The modernization and internationalization of legacy protocols,
   applications and formats will almost certainly require a UTF-8
   DNS eventually. It will not be possible to build and deploy
   UTF-8 extensions to SMTP without a UTF-8 DNS infrastructure. As
   with the above points, once a UTF-8 approach has gained some
   critical mass, these extensions become feasible. Without the
   infrastructure, SMTP will continue to be bound to ASCII.
  
 * UDNS is optional and transitional, on a per-domain basis. This
   allows a transition to occur as users see fit, without requiring
   a replacement of the existing DNS infrastructure with an
   alternate naming service. While this is technically not a
   "requirement" per se, the seamless user-driven transitional
   aspects should be requirements. Something very much like UDNS
   will be required for this. A "new naming service" is unlikely to
   reach critical mass without some kind of backwards compatibility,
   which UDNS provides.

   Furthermore, the cost of UDNS is incremental to that of ACE,
   since they share many common features and functions. If a
   developer is going to add ACE support to an application (which
   they must do in the short term), it is incremental to add UDNS
   support at the same time. On the other hand, adding ACE now and
   then going back to glue on something else is a second development
   effort, with greater costs.

 * UTF-8 is infinitely more manageable and serviceable than ACE.
   By being able to use UTF-8 tools and services, the Internet can
   be kept running much easier than users having to transliterate
   ACE operations. We should not be contributing to frailty. When
   a network is already broken, ACE obfuscates the problem by
   displaying aq--gobbledygook in traceroute, netstat, tcpdump and
   all of the other tools. ACE libs can certainly address this part
   of the issue, the extent of the support for tasks like importing
   trace data into a spreadsheet or viewing it in an editor is not
   as compelling. The massive number of UTF-8 tools are extremely
   compelling in terms of general manageability and serviceability
   of the global DNS.

 * Finally, there are some problems that ACE cannot solve, which
   UDNS can. The clipboard problem practically goes away, if not
   directly then indirectly, simply because we can facilitate the
   use of UTF-8 everywhere, rather than having to transliterate
   between application- and protocol-specific encodings. EG, if we
   ALLOW for the consistent use of UTF-8, then it is more likely
   to happen than if we actively PROHIBIT it by mandating yet
   another encoding which MUST be accounted for in every operation.

So that's the "benefit" side of the argument. If we can agree that these
are important considerations, and that some kind of support for UTF-8 is a
requirement (as per item #1), then we can debate costs and features as
they compare to that benefit.

Here is my evaluation of your perceived costs.

> will be illegal part of the time, unpredictably

This issue would have to be resolved if we were ever to move beyond ACE.
It is beneficial to design for such limits up front, rather than revoking
names later which will be incompatible. This is only a cost if ACE and
some other service are developed at different times, and is NOT a cost if
they are developed together. This is, in fact, a motivating factor for
beginning work on UDNS *NOW*.

> - UDNS requires more work to be done by authoritative DNS servers

Yes, UDNS requires authoritative servers to maintain name mapping
information. However, I see this as being a transitional cost. In the
beginning, few clients will support UDNS, but over the course of several
years, most clients will support UDNS. At some point (two decades out?
shorter?) it should be possible to switch entirely to UDNS, meaning that
there will no longer be a need for servers to store both versions. The
longer we delay beginning such a transition, the longer it will be before
we can complete that transition. If we never start it, we will never
complete it.

> - UDNS UTF-8 queries that fail will cause more load on the DNS

Yes, it will cause one additional lookup. This is equitable with requiring
one extra delegation server in the path. This is not a great cost to begin
with. Furthermore, seeing as how it is also transitional, it will only be
a cost for the short-term, and will not be a cost after a few years.

> - UDNS is not compatible with DNS security

I'm not sure I understand this point. Also, DNSSEC has many issues, and
appears to be destined for a redesign at this point.

> - UDNS requires that applications have the logic to send a new DNS query

Yes, an alternative API is required to prevent clashes with legacy
applications (this was proven by DJB's pi test). However, this is not a
cost burden which is solely related to UDNS. For example, when hostnames
are extended beyond the RFC972/1123 rules, an alternative wide() API will
be required for several functions. Applications which transliterate ACE at
relatively deep levels (such as for DHCP) will also require this. For
these reasons, I consider this a cost of internationalization rather than
a cost for UDNS.

> ACE, which some won't be), and then have to emit those queries again,

As with the other issues, this is a transitional cost which dissipates as
deployment goes up. It is a cost in the beginning, but it is not a
significant cost after a few years.

> (probably-nonstandard) logic for differentiating between expected

Because UDNS is an applicaton of EDNS, it uses the EDNS error codes. My
experience is that these codes are farily simple to work with.

From my perspective, you have raised four transitional costs (one of which
is nearly permanent but which does have an end), no perpetual costs, and
one cost I don't grok. That is not very expensive considering the benefits
and requirements, in my opinion. Conversely, ACE-only does have at least
one ongoing cost which is quite high (client-side name mutations), and
several perpetual costs in terms of usability and internationalization of
the global Internet.

---

From: Dave Crocker <[email-masked]>
Date: Mon, 08 Oct 2001 19:07:21 -0700

At 06:38 PM 10/8/2001, Eric A. Hall wrote:
>  We need to come to a group consensus on the
>neither

Eric,

It is neither.

To repeat the conclusions from my presentation at the last IETF:  uDNS has 
no significant technical benefit over an ACE, but it does have a serious 
problem. It impedes transition, since it requires changes to the 
infrastructure, whereas an ACE does not.  A copy of the slides, from the 
presentation, is at:

         <http://www.brandenburg.com/presentations/iDN-choices-comments.ppt>

>The above is a comparison of UDNS' perceived cost relative to a benefit,

A more immediate problem is that uDNS is an idea, not a complete 
specification.  The current document is substantially incomplete.

>many costs, some of which are quite high AND ongoing

if you mean the encode/decode cost, please be serious.  remember how short 
the string is.  if you mean something else, please enumerate.

and please remember that UTF-8 is also an encoding and, therefore, carries 
an encode/decode cost.

>    UTF-8 in all new protocols,

Please re-read section 2:

>    This document does not mandate a policy on name internationalization,
>    but requires that all protocols describe whether names are
>    internationalized or US-ASCII.

You are also neglecting to observe the fact that the DNS protocol already 
exists.  BCP18 is not mandating a change to existing protocols.

>    will be fatal.

"Fatal" is a very strong word.  Also an incorrect one.  Feel free to 
provide empirical data that supports your certitude.

>One example we have already discussed for this is
>    blow to such efforts

Why would an encoding form affect the ability to translate between two 
different naming environment?  Encoding is for transport, rather than for 
"native" form.

A more serious limitation is the core difference between LDAP and DNS 
naming rules.

And this raises the major problem that protocol efforts, like the current 
one, must not be used to try to fix larger and unrelated problems, such as 
mapping between two, incompatible name spaces.

>  * UTF-8 is infinitely more manageable and serviceable than ACE.

What is the empiricial basis for this assertion?

>    UDNS can. The clipboard problem practically goes away,

1.  You are talking about representation inside a computer, rather than 
transport across a network.  Internet protocols pertain to transport, not 
storage.

2.  You are also wrong about this particular problem.  This choice does not 
affect the ability to clipboard a name.

d/

----------

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Mon, 8 Oct 2001 20:44:43 -0700

>to be fully debated and discussed.

They have been; I was repeating what had already been said.

>be debated in appropriate context.

First, what the heck is a "UTF-8 namespace"? We're talking about 
Internet protcols, not namespaces. Second, the debate of "is there an 
advantage to carrying UTF-8 in the DNS protocol" has already been 
debated extensively. It was decided that debating in the abstract was 
not likely to produce a tangible result (namely a protocol); 
discussing specific proposals might lead to a good result. To date, 
there have been no UTF8-based proposals without major technical flaws.

>  Without consensus on this underlying

We weren't debating them; again, I was restating what had already be 
said over and over. If you really feel that there may be a way to do 
UTF-8 in the DNS protocol, write an Internet Draft and submit it. 
"I'm too busy to write a proposal, but I have lots of time to argue 
about the basis for it" won't sway many people.

>but we haven't fully discussed the benefits as of yet.

Some people would disagree with your definition of "fully".

>  Clearly, ACE has
>cost may be required.

Could you be specific about which parts of IDNA have "high" costs? 
The conversion from whatever local encoding to ACE is not much higher 
than converting to UTF-8. That's a pretty minor cost for all 
protocols that allow anything other than UTF-8.

>  What we do not agree on is the benefit that UDNS (or
>a similar mechanism) would provide.

That's one of the things. You skipped over the parts of my list that 
would make UDNS impossible to deploy for end users, namely the fact 
that some names would be illegal or legal in different parts of the 
Internet. That's a show-stopper.

>  Making a decision based on cost alone

This has been gone over many times; it is a red herring. Read the archives.

>    and development of LDAP in the Internet.

It is interesting that you are the only one who feels this way; none 
of the people active on LDAP protocol development or deployment have 
expressed the same concerns.

>    extremely acute in non-US development environments.

Again, this is based on a misreading of BCP18. Read the archives, 
particularly the messages from the author of BCP18. This is a dead 
horse, but feel free to beat it some more.

>    transition path from ASCII to UTF-8.

"User-driven"? UDNS requires changes to name servers. How is that 
"user-driven"?

>  UDNS allows applications to

Not true at all. Every application would have to have an ACE fallback mode.

>    UDNS provides this transitional path.

Either you are assuming that all current protocols that allow user 
choice of charsets will disappear, or they will will require only 
UTF-8 in the future. Both of those do a severe disservice to the 
users.

>  Conversely, ACE does not

Hogwash. It has nothing to do with migration to UTF-8.

>    which ONLY used UDNS, without any code for ASCII or ACE.

You are not talking about the UDNS that is before the WG. That 
protocol *requires* an ACE fallback.

>  At first

That's too bad, because UDNS it fatally flawed, as described before.

>    infrastructure, SMTP will continue to be bound to ASCII.

Some people reading that paragraph might assume that you are, or have 
been, active in the development of the protocols you are writing 
about. Of course, that isn't true. What is true is that most of the 
people who have been active in developing those protocols are pushing 
IDNA (or are at least rejecting UDNS as fatally flawed).

>    alternate naming service.

Again, this is simply wrong. Users don't control name servers.

>  While this is technically not a
>    they must do in the short term),

So, if you admit that here, why did you contradict it above?

>  it is incremental to add UDNS

You haven't shown any need to go back and glue.

>    ACE operations.

Users don't have to do that. Red herring.

>  We should not be contributing to frailty. When
>    all of the other tools.

So, you are saying that those tools today correctly display UTF-8? Yeah, right.

>  ACE libs can certainly address this part

"Massive", eh? Could you catalog those?

>    another encoding which MUST be accounted for in every operation.

This again takes the view that all current protocols will eventually 
go to UTF-8 only. That is insulting to the people throughout the 
world who have a strong preference to their own character sets.

>they compare to that benefit.

So far, no agreement.

>beginning work on UDNS *NOW*.

Sorry, but I missed the part where you explain how to fix the 
problem. Without a fix, UDNS is dead in the water.

>information. However, I see this as being a transitional cost.

And you demand that all of the other people running name servers will 
agree with you.

>  In the

In other words, you don't think that making the com/net/org servers 
and the root servers do normalization and mapping is a problem. I 
take it you don't run any of those.

>a cost for the short-term, and will not be a cost after a few years.

You blithely double the load on the DNS during the "transition". Nice.

>appears to be destined for a redesign at this point.

Yeah, that's a popular argument.

>these reasons, I consider this a cost of internationalization rather than

IDNA internationalizes just as well as UDNS, at lower cost.

>deployment goes up.

Wrong. It is with us as long as there are current applications running.

>  It is a cost in the beginning, but it is not a

First you said "twenty", now it's down to "a few". Some of us think 
there is a difference.

>experience is that these codes are farily simple to work with.

You have misread UDNS, then. The query errors from un-upgraded 
servers will not be EDNS errors: they will be format errors from the 
current DNS.

--Paul Hoffman, Director

---

From: "D. J. Bernstein" <[email-masked]>
Date: 9 Oct 2001 04:48:38 -0000

Paul Hoffman / IMC writes:
> To date, 

One exception is the straightforward set of Unicode/UTF-8 IDN operations
described in http://pi.cr.yp.to.

Yes, Unicode and UTF-8 require many software upgrades, but (1) those are
upgrades that we want to do anyway, and (2) all the other IDN proposals
require even more software upgrades.

---Dan

---

From: "Eric A. Hall" <[email-masked]>
Date: Tue, 09 Oct 2001 00:04:31 -0500

Dave Crocker wrote:

> uDNS has no significant technical benefit over an ACE

It advances the state of the Internet to a point beyond ASCII hackology,
which directly translates into a significant technical benefit for EVERY
OTHER PROTOCOL. Whereas there may not be any significant technical
difference between the two encodings from a high-level perspective, the
difference they make on the ground -- inside the applications which use
the encodings, particularly those applications and formats which already
use UTF-8 -- is tremendous.

> but it does have a serious problem. It impedes transition, since it
> requires changes to the infrastructure, whereas an ACE does not.

I disagree with both of those assertions:

 1) The changes which are required are optional. Admins that don't
    want to use IDNs will never have to change anything. Admins
    that want to use IDNs can deploy UDNS at their leisure. The
    root servers are the only servers that MUST be upgraded. It is
    entirely optional for every other zone, including the gTLDs,
    ccTLDs and end-user domains. It does not "require changes to
    the infrastructure" in the context you imply (unless your
    implication of "the infrastructure" was "the root servers").
    It can be rolled out at the whim of each and every other zone.

 2) ACE does require every application to implement transliteration
    for the applications to be usable as intended. ACE requires a
    fork-lift upgrade of the entire Internet, and we're left with
    ASCII hacks when that's over. UDNS is an incremental cost to
    this upgrade process, and leaves us with an infrastructure
    which is completely internationalized.

> >but we haven't fully discussed the benefits as of yet. Clearly, ACE
> >has many costs, some of which are quite high AND ongoing
> 
> if you mean the encode/decode cost, please be serious.  remember how
> short the string is.  if you mean something else, please enumerate.

As was illustrated, the principle ongoing cost is that every application
which is compliant with BCP18 will have to maintain separate mapping
services for ACE. Many protocols are intimately tied to domain names, and
all of them will have to implement a secondary mapping service.

> and please remember that UTF-8 is also an encoding and, therefore,
> carries an encode/decode cost.

If the OS and/or development environment is already UTF-8 clean, there is
minimal cost for the UTF-8 mode, but only for the ACE mode.

> >    internationalized or US-ASCII.

It is hard to imagine how any application can comply with the objectives
given in BCP18 and not use UTF-8 for protocol data such as embedded domain
name components. That section 2 disclaims "names" in the context of
protocol labels is no surprise. That it would be interpreted as reasonable
cause to enforce yet another mandatory encoding on every protocol not yet
invented is questionable, IMO. That is the result of ACE-only.

Is there some unspoken intention of promoting ACE to the status of
~Internet-standard-encoding for new applications?

> You are also neglecting to observe the fact that the DNS protocol
> already exists.  BCP18 is not mandating a change to existing protocols.

It appears that I misread section 3.2. I concede this point.

> provide empirical data that supports your certitude.

"Extreme hindrance to the point of not bothering" is more accurate,
although that translates to "fatal". One example is cited below.

> different naming environment?  Encoding is for transport, rather than
> for "native" form.

So what's the solution? Should LDAP DNs store dc components as ACE-encoded
strings, even though the DN is UTF-8 aware already? Or should LDAP clients
and servers transliterate every DN they encounter for comparison?

Why force this choice of lesser evils, when we do not have to, and when we
arguable should not do so in the first place?

> A more serious limitation is the core difference between LDAP and DNS

No, the dc= syntax already implies these rules. The problem is reflective
of applications which embrace UTF-8 and which also use DNS names as
protocol data. Every such application invented from here on out will face
this dilemma.

> And this raises the major problem that protocol efforts, like the
> current one, must not be used to try to fix larger and unrelated
> problems, such as mapping between two, incompatible name spaces.

The incompatibilities between dc= and DNS are exaggerated by ACE.

> What is the empiricial basis for this assertion?

 | ACE libs can certainly address [] part
 | of the issue, the extent of the support for tasks like importing
 | trace data into a spreadsheet or viewing it in an editor is not
 | as compelling. The massive number of UTF-8 tools are extremely
 | compelling in terms of general manageability and serviceability
 | of the global DNS.

For example, there are plenty of large network admins who still maintain
address assignment spreadsheets. International users manipulating this
data in ACE form are faced with much greater tasks than would be required
if they could use UTF-8. At the least, UTF-8 is likely to be a supported
encoding in the OS, if not the spreadsheet app, while ACE is likely to be
supported in neither. These things add up to affect serviceability; where
did I transliterate that name wrong?

> transport across a network.  Internet protocols pertain to transport,
> not storage.

We sure spent a lot of time talking about it.

> 2.  You are also wrong about this particular problem.  This choice does
> not affect the ability to clipboard a name.

It does affect transliteration requirements.

> specification.  The current document is substantially incomplete.

Yes, it needs some work, but the underlying principles are sound (IMO).

---

From: "Eric A. Hall" <[email-masked]>
Date: Tue, 09 Oct 2001 00:08:44 -0500

Paul Hoffman wrote:

[overlapping comments and non-responses eliminated]

> protocol *requires* an ACE fallback.

The fallback mode also serves transitional purposes. After the majority of
queries start succeeding, only a minority of queries will fallback. After
a few more years, they all succeed, and there are no fallbacks. Dual-mode
facilitates transition.

> So, if you admit that here, why did you contradict it above?

There's no contradiction with UDNS providing ACE+UTF8 in the short term
and eventually turning into UTF-8 only.

> >This issue would have to be resolved if we were ever to move beyond
> >ACE. It is beneficial to design for such limits up front, rather than
> >revoking names later which will be incompatible. This is only a cost
> >if ACE and some other service are developed at different times, and is
> >NOT a cost if they are developed together. This is, in fact, a
> >motivating factor for beginning work on UDNS *NOW*.
> 

The "fix" that would be required is to define common maximum lengths. This
will be required in any event, and doing it up front makes most sense.

> And you demand that all of the other people running name servers will

Versus the ACE-only demand that every application support a secondary
encoding that isn't used anywhere else in the world. Forever.

> take it you don't run any of those.

The com/net/org servers don't HAVE to do it. The question of whether or
not sufficient computing resources are available on the market to service
those zones is a question for those admins, not for you or me. I wouldn't
be shocked if it wasn't rolled out on those zones right away.

> You blithely double the load on the DNS during the "transition". Nice.

It's only doubled if every lookup fails. Where would this load come from?
Are there going to be URLs with non-existent IDNs floating around that
users will always try to follow with their non-existing apps? Please.

Most of the early delegations will result in failed queries. Once
compliant apps were released and servers were deployed, less queries would
fail. Eventually the load will return to normal. At some point it shifts
from fallback-to-ACE to don't-fallback-it-worked, and that shouldn't be
impossibly far away.

The sooner it is started, the sooner we will finish, anyway. Better to do
it now than to do it 10 years from now when the loads are higher.

> Wrong. It is with us as long as there are current applications running.

And they will always be with us in their current form if we go with
ACE-only. Do we really want to be designing for ARPANET forever? At what
point do we start thinking about laying the necessary groundwork for
migration to eight-bit technology?

> First you said "twenty", now it's down to "a few". Some of us think

What I said was that ACE mode could be disabled in twenty years if not
sooner, when the transition is complete. Processing a "significant number"
of failovers should only be an issue for a few years, until the common
case is UTF-8 success. These are different scenarios and they have
different timetables.

> servers will not be EDNS errors: they will be format errors from the

No, they aren't different. The same errors that EDNS is intended to
trigger should be the errors which UDNS triggers. If this doesn't work
with UDNS, then it won't work with EDNS either.