Subject: RE: [idn] iDNS transition: end-system vs. infrastructure?

From: "Russ Rolfe" <[email-masked]>
Date: Fri, 11 May 2001 15:32:40 -0700

Dave Crocker wrote: 7 May 2001

         Hence, the primary concern for "just do pure UTF-8" is
> the effect of sending UTF-8 to an aDNS server.  However the nature of 
> DNS is such that a DNS field with a UTF-8 string only gets sent to a 
> server that is supposed to be able to resolve the that field.

Since we are looking at alternatives depending on the Walid proposal and
"just do pure UTF-8" is an alternative, could someone kindly post a
simple list of the reasons they feel "Just do pure UTF-8" is *not* a
valid alternative.  I am still not convinced that we should not just use
pure UTF-8.

Regards, Russ Rolfe
Microsoft

---

From: Keith Moore <[email-masked]>
Date: Fri, 11 May 2001 18:51:50 -0400

> could someone kindly post a

we cannot use pure UTF-8 because non-IDN-aware applications will still 
need to be able to lookup ACE-encoded IDNs.

so while we could have a pure UTF-8 query interface to look up IDNs,
we would still need the ACE query interface also.  and we'd want the
two to produce consistent results.

but we should be careful about using the patent as justification for
adding another query interface.  as far as I can tell IDNA doesn't 
infringe on the patent anyway.

Keith

---

From: "Adam M. Costello" <[email-masked]>
Date: Fri, 11 May 2001 23:02:10 +0000

Russ Rolfe <[email-masked]> wrote:

> could someone kindly post a simple list of the reasons they feel "Just
> do pure UTF-8" is *not* a valid alternative.

Domain names appearing in message headers, SMTP commands, URIs, etc. are
required by various standards to contain only ASCII letters, digits,
hyphens, and dots.  Therefore there is no telling what existing software
would do if faced with domain names containing other characters.  Some
of it would surely break.  DNS is not the problem.

AMC

---

From: John C Klensin <[email-masked]>
Date: Sat, 12 May 2001 07:54:21 -0400

--On Friday, 11 May, 2001 18:51 -0400 Keith Moore
<[email-masked]> wrote:

>> simple list of the reasons they feel "Just do pure UTF-8" is
>> *not* a valid alternative.  I am still not convinced that we
>> should not just use pure UTF-8.
> 
> we cannot use pure UTF-8 because non-IDN-aware applications
> will still  need to be able to lookup ACE-encoded IDNs.

Keith,

While I agree -- for the reasons mentioned by Adam and that I'm
sure will be repeated and expanded upon by others-- that "just
send UTF-8" is a bad idea, I'm not sure the above follows.

We have approaches --the "use a different DNS Class" idea is
certainly one example-- which would prevent a non-IDN-aware
application from ever encountering an IDN in a context in which
being able to resolve it is a reasonable expectation.  And
having an application that is completely non-IDN-aware handle
ACE-type IDNs correctly (with IDNA or otherwise), is likely, but
not certain, to bypass unanticipated problems.

So I would contend that the community actually has to make a
choice here, a choice among:

	(i) A family of solutions, including "just send UTF-8"
	which expose non-IDN-aware applications to things they
	almost certainly are not expecting and which, given our
	experience, are likely to cause application problems.
	Showing otherwise would require analysis of all
	implementations of all applications, which is roughly
	equivalent to proving a universal negative.  And, with
	the understanding that this is an observation, not a
	slam, I would assume that anyone with experience with
	applications and operating systems that respond to
	unexpected conditions and data formats with blue screens
	would be particularly sensitive to this sort of problem.
	
	(ii) A family of solutions, including IDNA and
	presumably most of the other ACE-based ideas, that
	attempts to encode the IDN data so that it will slip by
	non-IDN-aware applications, databases, and systems.
	These solutions are, I think clearly, less likely to
	exhibit serious problems than those in the first group,
	but there are still risks.   And some of the human
	interface, and text-imbedded-in-document, issues seem to
	me quite daunting.  The community could conclude that
	the advantages (including a form of quick deployment
	that does not change the DNS and leaves applications on
	their own) are a good compromise with the risks, but the
	approaches are still compromises with risk.
	
	(iii) A family of solutions that are "no risk", because
	their designs prevent non-IDN-aware applications from
	ever seeing an IDN in a DNS context.  If a putative URL
	appears in running text, the application can either cope
	with it or not, but, if it tries to do something with it
	and violates its own rules about what it can handle and
	pass to the DNS, that is clearly a problem with that
	application (not just failure to upgrade).  And, with
	these solutions, DNS queries and responses from
	non-upgraded applications just will not involve IDNs.  

The solutions in the third group, which might be characterized
as "not in Class=IN" or "not in the DNS", or "need to fully
negotiate first" (as far is I know, we don't have any plausible
ones of the latter on the table, but they are logically
possible) have the clear disadvantage that they require serious
deployment to work for the people/ language groups/ countries
that need IDNs.  So they are not quick fixes.  But some of them
provide functionality, possibly critical functionally (such as
imprecise search), that neither of the other two groups do.  So,
again, it is a tradeoff.

But let's avoid assuming that the second group of approaches are
the only alternative to the first and, hence, that applications
will need to handle ACE names no matter what.

   john

---

From: "Adam M. Costello" <[email-masked]>
Date: Sat, 12 May 2001 19:41:05 +0000

John C Klensin <[email-masked]> wrote:

>  (iii) A family of solutions that are "no risk", because their designs
>  prevent non-IDN-aware applications from ever seeing an IDN in a DNS
>  context.
>
> But let's avoid assuming...that applications will need to handle ACE
> names no matter what.

Could you elaborate on this, with examples?  What sorts of things
would occupy the domain name position in a message header field, or
an SMTP command, or a URI?  Does family (iii) entail revising every
protocol/format that embeds domain names?

What about the case where I send an email message to two people, and one
of them has an IDN address and the other is using a mail program that
predates IDN.  Can the latter reply to all of us?

I might not be understanding what you have in mind, but (iii) looks
difficult to deploy without a lot of effort/pain.

AMC

---

From: John C Klensin <[email-masked]>
Date: Sat, 12 May 2001 16:34:01 -0400

--On Saturday, 12 May, 2001 19:41 +0000 "Adam M. Costello"
<[email-masked]> wrote:

>>  their designs prevent non-IDN-aware applications from ever
>>  seeing an IDN in a DNS context.
>> 
>> But let's avoid assuming...that applications will need to
>> handle ACE names no matter what.
> 
> Could you elaborate on this, with examples?  What sorts of
> things would occupy the domain name position in a message
> header field, or an SMTP command, or a URI?

Depends on which member of family (iii) one picks.  

The notion of using a different DNS class (see
draft-klensin-i18n-newclass-... for some discussion; I gather a
more detailled document is coming) is in this family.  If that
were chosen, the answer in all the cases you list would be "DNS
name", but DNS name in a class that would not be used by
non-IDN-aware applications.   If those names were passed to a
non-IDN-aware app, they would presumably be rejected as improper
format if they were in UTF-8 format and passed through with a
"Class=IN" query in ACE-ish formats (or if UTF-8 slipped
through), after which they would get NODOMAIN responses.

At least some of the "directory" / "search system" / "keyword"
approaches also fall into family (iii).  See
draft-klensin-dns-role-... for some discussion of them and watch
for a new and more extensive document/ discussion RSN.  Those
assume that we would gradually migrate all applications that
reference objects using "words" or "names" or "service marks"
--rather than protocol identifiers for network objects-- to
utilize an intermediate layer (or two) whose behavior more
closely matches the expectations of normal users.  I.e.,
searching, imprecise matches, and, generally, a "when in doubt,
ask the user" strategy would be possible and normal.   That, of
course, requires changing not only the code of all of those
applications, but their user interface design.  Large deployment
problem, but possibly the right solution in the long term...
indeed, possibly that only one that will be seen as "working" in
the long term.

  Does family (iii)
> entail revising every protocol/format that embeds domain names?

And worse in some cases.  See above.

> What about the case where I send an email message to two
> people, and one of them has an IDN address and the other is
> using a mail program that predates IDN.  Can the latter reply
> to all of us?

Most likely not, at least without going to some effort.  On the
other hand, this is not necessarily bad news either.  It
encourages those who need to communicate to upgrade, just as the
relative invisibility of new hosts being added to the Internet
was one of the things that finally got people moving on DNS
deployment.  Specifically, if I sent mail to two people, one of
whom had a DNS-only address (possibly with MX records) and the
other one of whom was still using host tables, the latter could
not reply to me and the other correspondent without going
through a more or less complicated drill to translate the DNS
name into an address.  So we have been there before, with what I
would contend was a very similar situation, and we survived it.

> I might not be understanding what you have in mind, but (iii)
> looks difficult to deploy without a lot of effort/pain.

I have carefully not claimed otherwise.  The difficulties with
(i) and (ii) is that they

* Risk considerable damage and subtle interoperability problems
that users can't work around (note that the Hosttable-DNS
transition example above is a case where we have worked examples
of a transition situation the user can work around)

* Arguably don't solve the problem.  As long as there is a
single case in which the result produced by nameprep may be
debatably wrong from the standpoint of a user (characters that
match across scripts and different treatments of case matching
are likely candidate problem areas here, but there are others),
or in which mapping rules are required that may be complex, not
one-one, or that may have heuristic properties, we are sooner or
later going to need something with "search" or "fuzzy match",
not just "lookup" properties.   

The DNS can't do that, and that makes the question one of "if we
have to deploy an 'other layer' solution eventually --including
all those application changes-- do we really help ourselves by
putting complex and risky solutions into the DNS?".  I have
opinions about the answer to that question, but wouldn't claim
to have any certainty about the correct answer.

    john

---

From: Keith Moore <[email-masked]>
Date: Mon, 14 May 2001 16:28:17 -0400

> being able to resolve it is a reasonable expectation.  

people have suggested such approaches.  I personally do not believe
that they are viable.  in particular, we cannot implement such an
approach without adding explicit negotiation for IDN-compatibility
to every protocol which currently carries a DNS name - including
protocols such as internet mail (rfc2822/mime) and html which are
exchanged via many different bearer protocols.  it's simply not
feasible to add IDN-capability negotiation to each of these.  Nor,
absent conversion to some sort of ascii equivalent,  do there appear 
to be any acceptable modes of operation for the case when the 
negotiation reveals that the destination is not IDN-capable. 

> And

I think we've established there there will be some problems no
matter which approach is chosen.  We can still hope to minimize
them.

So in summary, I agree with your categorization of the design 
choices that this group has to make, but for me, both:
(i) just send native IDNs to non-aware applications , and
(iii) preventing non-aware applications from seeing native IDNs
are non-starters, at least for a general approach to retrofitting
existing applications.   (some new applications could perhaps
reasonably employ a type iii solution if they never exchanged
IDNs with other applications)

I disagree with the categorization of type iii as "no risk",
because to me it appears that there is considerable risk 
associated with adding IDN capability negotiation to the 
large number of existing applications that use DNS names.
This risk might not be so much that existing applications
are exposed to IDNs, but instead that updating a large
number of applications to explicitly support IDN negotiation
will introduce new failure modes that are not easily understood 
by users, and require changes to software that will sometimes 
require major restructuring and thereby introduce bugs that
will cause operational errors.

Keith

---

From: [email-masked] (David C Lawrence)
Date: Tue, 15 May 2001 01:38:11 -0400 (EDT)

Russ Rolfe writes:
> could someone kindly post a simple list of the reasons they feel
> "Just do pure UTF-8" is *not* a valid alternative.

Besides the other reasons already mentioned, there is the issues that
the security risk is greatly increased that someone can spoof domain
names.  Just craft a string in some character set or encoding other
than UTF-8 that can be interpreted as valid UTF-8, then trick people
with non-aware clients into sending the binary name as a DNS request.

Take for example a bad guy who sends a KOI-8 email to victim that has
a link within it to {TSE}{IO}.ru, perhaps the web site of a well-known
company.  Victim's legacy browser requests {TSE}{IO}.ru, but it is
interpreted as UTF-8, so he gets back the address for
{LATIN SMALL LETTER O WITH ACUTE}.ru -- the bad guy's sham site.

This admittedly takes some special circumstances (I am not suggesting
that every IDN would be spoofable this way) but with all of the
permutations of non-UTF-8 and millions of IDNs, there will surely be
plenty of opportunity for either accidental or intentional
misdirection to happen.

This does not rule out UTF-8, it only rules out the "Just Send" part.
There must be some way for a program to indicate that it is aware of
the rules surrounding the use of binary goop as a DNS name.