Subject: Re: [idn] Re: 7 bits forever!

From: "John Stracke" <[email-masked]>
Date: Tue, 2 Apr 2002 09:54:32 -0500

D. J. Bernstein wrote:

>It is not reasonable to demand that this
>7-bit garbage be allowed to remain on the network forever, at the
>expense of all future implementors and users.

OK, you're obviously in a better position than I am to appreciate the cost 
to implementors.  But users? Suppose you could streamline qmail by 
dropping support for talking to 7-bit servers.  How do you convince a 
postmaster to install it? The bandwidth cost of having to use 7-bit 
encoding is real, but small; the cost of losing connectivity to sites is 
potentially high.  Worse, it's almost impossible to put an upper bound on 
the cost of lost connectivity.  For a company, it could mean losing 
messages (and sales) from important customers; businesses have failed over 
such things.

For commercial implementors, it's a no-brainer: dropping 7-bit support 
will lose customers, so don't do it.  For open-source implementors such as 
yourself, naturally, things are less clear-cut.

/==============================================================\
|John Stracke                    |Principal Engineer           |
[email-masked]   |Incentive Systems, Inc.      |
|http://www.incentivesystems.com |My opinions are my own.      |
|==============================================================|
|"This [OLE] isn't an API, this is a shaggy dog story in C!" --|
|Unknown                                                       |
\==============================================================/

---

From: "Edmon Chung" <[email-masked]>
Date: Tue, 2 Apr 2002 09:56:44 -0500

I think the issue here is how we can plan a better transition than what we
have done for SMTP.
That is, if we start thinking about "8+ everywhere" instead of "ACE
everywhere" scenario, then how we can get there from here. That is the
question for the IDN group I believe.
Edmon

---

From: "Edmon Chung" <[email-masked]>
Date: Tue, 2 Apr 2002 10:29:00 -0500

I am not saying its going to be non-backwards compatible.
Ok, I have mentioned a possible migration path before, let me try it again
(btw, it is funny that everytime I put this out, no one really debate the
merits of it... but then a spurt of discussion on UTF8 breaksout)

1. IDNA clients
    - the client does ACE and display it properly for the user
    - servers (including DNS and mail and http, etc.) does NOT upgrade for
ACE.  Even administration side is kept as is, allowing leakage
    - IDNA clients MUST also be able to decode IDN(UTF8)/EDNS responses

2. IDN(UTF8)/EDNS DNS Server upgrade
    - DNS servers will now accept both ACE (as usual) and IDN(UTF8)/EDNS
requests
    - ACE requests will yield Both an ACE response + an IDN(UTF8)/EDNS
response
    - IDNA clients continue to send in ACE but will start to see UDNA
responses

3. IDN(UTF8)/EDNS for clients & everywhere
    - IDNA clients begin to obsolete, new versions of application software
comes with IDN(UTF8)/EDNS built in.
    - In case DNS servers are not upgraded yet, clients will continue to use
the IDNA client, thereby also creating an incentive for server side to
upgrade
    - Other applications/servers upgrade to IDN(UTF8)/EDNS to accept UTF8
domains, administrators might continue to see ACE leakage if the client
still uses IDNA to send requests, but UDNA requests will be managed in UTF8

4. IDN(UTF8)/EDNS
    - Slowly but certainly, we will move towards full UTF8 because
administrators will likely want to deal with UTF8 than ACE, especially for
say those admins in China who are dealing with IDNs frequently.  Added to
the urge from the user community to upgrade the server as they upgrade their
clients to IDN(UTF8)/EDNS.

Thoughts anyone?

Edmon

----- Original Message -----
From: "John Stracke" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, April 02, 2002 10:08 AM
Subject: Re: [idn] Re: 7 bits forever!

> >everywhere" scenario, then how we can get there from here.
>
> Same question: if you have a non-backwards-compatible server, how do you
> get people to use it?
>
> /========================================================\
> [email-masked]   |Incentive Systems, Inc.|
> |http://www.incentivesystems.com |My opinions are my own.|
> |========================================================|
> |Speak softly and carry an Illudium Q-32 Explosive Space |
> |Disintegrator.                                          |
> \========================================================/
>
>

---

From: Keith Moore <[email-masked]>
Date: Tue, 02 Apr 2002 11:16:09 -0500

> question for the IDN group I believe.

I cannot imagine that the IDN group is competent to decide how email
(or any other specific application) should migrate.  

Keith

---

From: "Edmon Chung" <[email-masked]>
Date: Tue, 2 Apr 2002 12:43:22 -0500

Hi Keld,

----- Original Message -----
From: "Keld Jørn Simonsen" <[email-masked]>
> I was amongst the most vocal proponents af a general 8-bit email
> exchange protocol, the one that emerged as ESMTP. I remember being
> at IETF in Santa Fé 1992-ish, where we discussed this, and I was the
> only European around when it was asked if somebody, especially from
> Europe, really wanted a general 8-bit exchange protocol. And I raised
> my voice and insisted. So it was not really questioned anymore
> and ESMTP was produced. But mostly as a courtesy to the Europeans.
> Sometimes you can make a difference:-)

Whole heartedly agree.  This time we have yourself and the Asian community
standing up saying we want IDN(UTF8), hopefully, this is unquestioned as
well.

> I think getting to >8bit is a non-starter, as per IESG policies
> that promotes utf-8 as the primary encoding in protocols.
> And I really do not see why we should open up to a myriad
> of charsets (being the author of rfc 1345 I need to stress that:-)
> We have one, we can deal with two, not many.
>
> What we have now is a restricted ASCII for DNS, and we want to
> go to a restricted utf-8. And nothing else. And we want the
> backwards compatibility and operation between ascii and utf-8
> so we can ensure a happy migration and coexistance.
>
> Best regards
> Keld
>

Makes good sense.

Edmon

---

From: "James Seng" <[email-masked]>
Date: Wed, 3 Apr 2002 02:25:46 +0800

> standing up saying we want IDN(UTF8), hopefully, this is unquestioned as

I am aware that some chinese members in this group have stood up to ask for
TC/SC in IDN. I am also aware that Lee have asked for reordering.

But I am not aware of any "Asian community standing up saying we want
IDN(UTF8)". I honestly hope I havent missed anything important.

Or is this just another fragment of your imagation?

It is frustrating to hear you repeating "I want UTF-8, I want UTF-8 now!"
again and again. Lets just say "yes, we hear you say that and lets move on".

By moving on, I mean you write a draft to cover:
a) what is the advantage of IDN(UTF8) over the existing IDNA
b) how IDN(UTF8) is going to work
c) how you intend to get there

-James Seng

---

From: "Edmon Chung" <[email-masked]>
Date: Tue, 2 Apr 2002 14:24:43 -0500

James,

I think you missed the point completely.  The point, as Keld have mentioned
is that there are people, who believes that it is beneficial to have UTF8
option, I being one of them and Keld being one of them, and it is a fact
that cannot be ignored and should not be ignored.  From my interpretation of
your previous words, you also believe that it is beneficial to go beyond
ASCII.  That, is my point and please dont try to confuse the message with
the means!  Human language is not machine language and you have to
understand the context before you can get the message.  If you cant
understand this simple depth and intricacy of human interaction and
communication, I cant help you.

The exercise is to establish a meaningful support before I waste my time
working on a draft if no one wants to talk about it.
I have a draft of a draft inplace for UDNA, and I am updating it as we
continue to discuss some of the elements of it on the list.  I think it
makes more sense this way, than to throw something in here fresh.  Also I
think it would be beneficial to have a draft on IDNX (IDN Transition).  So
please stop distrupting the discussion of a good discussion here.

Edmon

----- Original Message -----
From: "James Seng" <[email-masked]>
To: "Edmon Chung" <[email-masked]>; "Keld Jørn Simonsen" <[email-masked]>
Cc: <[email-masked]>; "John Stracke" <[email-masked]>;
<[email-masked]>; <[email-masked]>; <[email-masked]>
Sent: Tuesday, April 02, 2002 1:25 PM
Subject: Re: [idn] Re: 7 bits forever!

---

From: Keld Jørn Simonsen <[email-masked]>
Date: Tue, 2 Apr 2002 19:10:32 +0200

> question for the IDN group I believe.

Well, I think SMTP was very successful in getting to 8-bits everywhere.
Or almost everywhere. Today ESMTP is the predominant protocol for email
exchange. And this was really not in the cards when we designed the
MIME stuff. The predominant policy at those times was  "use 7-bit
generally, and 8-bit locally", the so-called "enclave" model.
I was amongst the most vocal proponents af a general 8-bit email
exchange protocol, the one that emerged as ESMTP. I remember being
at IETF in Santa Fé 1992-ish, where we discussed this, and I was the
only European around when it was asked if somebody, especially from
Europe, really wanted a general 8-bit exchange protocol. And I raised
my voice and insisted. So it was not really questioned anymore
and ESMTP was produced. But mostly as a courtesy to the Europeans.
Sometimes you can make a difference:-)

Best regards
Keld

---

From: Keld Jørn Simonsen <[email-masked]>
Date: Tue, 2 Apr 2002 22:30:43 +0200

> communication, I cant help you.

Well, I have to state that I am not so sure we need an UTF-8 upgrade.
Maybe ACE is enough. Things can get complicated if eg the primary DNS
handles utf-8 while one of the secondaries do not.
DNS is different from SMTP as all DNS clients need to *understand*
all DNS servers, each character needs to be understood. In SMTP the
MTAs are only exchanging messages between consenting adults, and
it is up to the sender and receiver to mutually understand eachother.

Also the situation back in 1991 with SMTP was different from
todays DNS issues. Then we had a lot of email that was not
ASCII, but a myriad of other encodings, incl national 7-bit variants,
japanese, and iso-8859-?. Today we almost entirely still have
only (restricted) ascii in the DNS.

I frankly think that for interoperation, all DNS data needs to be
stored in ACE encoded ISO 10646, and on the wire it should be ACE
encoded queries and responses. It would then be up to the clients
to do the normalization and conversion from/to ACE.
That would mean no changes to existing DNS protocols!

Then you could have tools for maintenance of the zone data in
one or more charsets for convenience of the DNS zone maintainers.

A weak point of mine is that *iff* we go beyond ASCII, UTF-8 is
the only additional encoding we should consider for the wire,
and *iff* seemless interoperability can be proven with existing
operations and some gains can be shown, I would not be against
a new UTF-8 protocol for DNS.

Best regards
Keld

---

From: "James Seng" <[email-masked]>
Date: Wed, 3 Apr 2002 08:41:44 +0800

And you miss the point completely too.

IETF move forward with draft. Repeatively say "i want utf-8, i want it now"
after the first time are just noise.

Write your draft. It is make sense, then support will come.

-James Seng