Subject: [EAI] Add requirement for punycode conversions in downgrade

From: Ernie Dainow <[email-masked]>
Date: Mon, 17 Mar 2008 09:55:42 -0400

When sending email from an address containing an ASCII local name and a 
non-ASCII domain to an all ASCII recipient on a non-UTF8SMTP system, a 
downgrade should occur. Following the current EAI drafts, downgrade 
fails if there is no Alternate Address provided for the sender. In this 
case, an Alternate Address is not really required, as the sender's 
address can be represented in ASCII by doing a punycode conversion on 
the domain part of the address. There is also a similar case when the 
recipient address is of the form [email-masked].

There are several ways to address these cases:
1. No change to the current drafts. It is the responsibility of the 
sender to provide an Alternate Address if they want Downgrade to occur. 
They can provide the punycode equivalent address as an Alternate Address.
2. The MUA can automatically do punycode conversions for these cases 
when Alternate Addresses are not provided by the user.
3. Downgrade should do a punycode conversion instead of failing because 
of no Alternate Address.

Since attempting delivery is almost always better than failing a 
delivery, I would support adding a requirement to do the punycode 
conversion for these cases in the Downgrade specification.

Ernie Dainow
_______________________________________________
IMA mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ima

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 18 Mar 2008 02:34:12 +0100

> conversion for these cases in the Downgrade specification.
For the case of non-ASCII localparts, this has been discussed 
extensively, and version 3 has been consistently rejected by the WG.

I haven't seen a discussion that focused on the ASCII@non-ASCII situation.

               Harald

---

From: John C Klensin <[email-masked]>
Date: Tue, 18 Mar 2008 12:25:05 -0400

--On Tuesday, 18 March, 2008 02:34 +0100 Harald Tveit Alvestrand
<[email-masked]> wrote:

>> 3. Downgrade should do a punycode conversion instead of
>> failing because  of no Alternate Address.
>> 
>> Since attempting delivery is almost always better than
>> failing a  delivery, I would support adding a requirement to
>> do the punycode  conversion for these cases in the Downgrade
>> specification.

It seems to me that we have a tradeoff.  As Yao points out, no
presentation of punycode to a user is user-friendly.  Punycode
encoding can also be used to disguise various types of attacks
that would be obvious if the string were presented in "native"
form -- an analogy to the use of IP addresses, rather than
domain names, in URLs might apply here.     One can say "Since
attempting delivery is almost always better than failing...",
but perhaps this is one of the exceptions in which it is better
to get the failure message back to the sender and sending MUA in
the hope of getting the punycode change made there (or some
other decision being made).

In any event, it seems to me that the case for _requiring_
punycode conversion as a downgrade method is weaker than some of
this discussion would indicate.

    john

---

From: Ernie Dainow <[email-masked]>
Date: Tue, 18 Mar 2008 17:03:26 -0400

> this discussion would indicate.
The user interface at the recipient can and probably should translate 
punycode domains back to the original UTF8. Web browsers do this (at 
least Firefox and Internet Explorer). If you enter a punycode domain in 
the url field, the browser will translate it to UTF8 if you have the 
appropriate fonts installed. Email clients should provide a similar 
translation for email addresses containing punycode domains that are to 
be displayed to the user. I can add this to the Email Client guidelines 
document.

If Downgrade fails when sending to a recipient of the form 
ascii@non-ascii , what is the recourse for the user? A very few IDNA 
literate users might figure out how to do a punycode conversion and use 
it as an Alternate Address. Most users would have to revert to using a 
traditional email client.

One can also argue that this case will not be very common, as an 
organization that sets up email accounts using an Internationalized 
domain name would surely use EAI email software. However, during the 
transition period that we are in (internationalized domains are being 
registered), this case may occur often enough that Downgrade should 
handle it.

Ernie

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 18 Mar 2008 22:23:33 +0100

Ernie Dainow skrev:
> literate users might figure out how to do a punycode conversion and 
> use it as an Alternate Address. Most users would have to revert to 
> using a traditional email client. 
Well, there are some possibilities for why we get that situation:

- User (or admin) has forgotten to set the Alternate Address 
(misconfiguration). Is it better for a mid-net relay to pass the message 
or to report to the sender that there's a misconfiguratoin?

- User (or admin) has decided that he doesn't care whether or not 
messages get to non-UTF8SMTP users. Is it better for a mid-net relay to 
pass the message or to report to the sender that his wishes have been 
"honored"?

- User (or admin) expected this message to go to only UTF8SMTP users. Is 
it better for a mid-net relay to pass the message or to report to the 
sender that his expectation was unrealistic?

Note: I don't know whether the answers are one or the other. But I can 
see arguments on both sides of the issue - "pass the message" is not 
necessarily always the right choice.

              Harald

---

From: John C Klensin <[email-masked]>
Date: Tue, 18 Mar 2008 18:47:32 -0400

--On Tuesday, 18 March, 2008 22:23 +0100 Harald Tveit Alvestrand
<[email-masked]> wrote:

>> ascii@non-ascii , what is the recourse for the user? A very
>> few IDNA  literate users might figure out how to do a
>> punycode conversion and  use it as an Alternate Address. Most
>> users would have to revert to  using a traditional email
>> client. 
> (misconfiguration). Is it better for a mid-net relay to pass
> the message or to report to the sender that there's a
> misconfiguratoin?
> 
> - User (or admin) has decided that he doesn't care whether or
> not messages get to non-UTF8SMTP users. Is it better for a
> mid-net relay to pass the message or to report to the sender
> that his wishes have been "honored"?
> 
> - User (or admin) expected this message to go to only UTF8SMTP
> users. Is it better for a mid-net relay to pass the message or
> to report to the sender that his expectation was unrealistic?
> 
> Note: I don't know whether the answers are one or the other.
> But I can see arguments on both sides of the issue - "pass the
> message" is not necessarily always the right choice.

I think we are on the same page.  I don't know what the right
answer is (either), but "convert and pass" is not as obvious to
me as it apparently is to Ernie.

Two other observations:

	* I haven't seen an analysis of the potential "leakage"
	cases, and whether or not they are a problem, yet.
	
	* Unless we get IDNA200X through, and get to an end
	result with no input-side mapping, Ernie's comment "The
	user interface at the recipient can and probably should
	translate punycode domains back to the original UTF8" is
	basically not possible to satisfy.  The interface can
	translate back to some approximation to the "original"
	UTF8, but, since IDNA2003 ToASCII provides a many
	(perhaps one should say "many-many") to one mapping, the
	receiving MUA will not have sufficient information to
	reconstruct the original UTF8, only an IDNA-equivalent
	version of it.

--john

---

From: "YAO Jiankang" <[email-masked]>
Date: Wed, 19 Mar 2008 10:37:49 +0800

----- Original Message ----- 
From: "Ernie Dainow" <[email-masked]>
To: "John C Klensin" <[email-masked]>
Cc: <[email-masked]>
Sent: Wednesday, March 19, 2008 5:03 AM
Subject: Re: [EAI] Add requirement for punycode conversions in downgrade

> be displayed to the user.

my concern is that if the reciever smtp server is not EAI-compatible, the recipient is more unlikely to use an EAI-compatible email client.
so the form presented to the user has to be the form of unfriendly punycode version if the punycode form is produced in the email client .

Yao Jiankang

> I can add this to the Email Client guidelines 
> document.
>

---

From: Ernie Dainow <[email-masked]>
Date: Wed, 19 Mar 2008 11:10:58 -0400

> me as it apparently is to Ernie.
Yes these are all possible scenarios, but whether the receiver is 
UTF8SMTP compliant or not is of interest to a relatively few number of 
technically knowledgeable email users compared to the millions of 
average email users who basically just want to see their email get 
delivered. After all, wasn't Downgrade included in the specification for 
exactly this reason -- to keep email flowing during an interim period 
when there is a mix of traditional and UTF8SMTP email systems?

The failure situation here is a lack of inter-operability between EAI 
and "IDNA" email. By IDNA email I mean a traditional email system in 
which the email client has been modified in accordance with RFC3490, so 
that users are able to enter email addresses with UTF8 domains (but the 
local part is restricted to all ASCII). The email client does punycode 
conversions on the domain part so that the messages sent are traditional 
email. I know there is one implementation that does this; I don't know 
how much it is used at this point. As the time stretches out to 2009 to 
get a standards track EAI completed, these "IDNA" email systems might be 
more widely used. So we have a situation where EAI supports traditional 
email via Downgrade but possibly not "IDNA" email.

As a concrete example of this, ICANN has an email test page for testing 
International top level domains at http://idn.icann.org/E-mail_test. 
Their second test is to copy an email address of the form 
ascii@non-ascii into an email client. Our current EAI implementation 
fails this test, following a strict adherence to the current EAI specs. 
I don't see any reason an implementation can't decide it will do the 
punycode conversion in Downgrade. It is supported by section 2.2 in 
draft-ietf-eai-smtpext: "An SMTP Client . . . MAY transmit the domain 
parts of mailbox names within SMTP commands or the message header in 
either the form of ACE labels as specified in IDNA [RFC3490] or as UTF-8 
strings." If we leave this decision up to the implementation, then we 
have a situation where EAI may or may not inter-operate with IDNA email. 
I think it would be better to have a clear statement on this, either EAI 
supports inter-operability with IDNA email or it does not.

Ernie

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 19 Mar 2008 20:10:32 +0100

Ernie Dainow skrev:
> delivered. After all, wasn't Downgrade included in the specification 
> for exactly this reason -- to keep email flowing during an interim 
> period when there is a mix of traditional and UTF8SMTP email systems?
>
> which the email client has been modified in accordance with RFC3490, 
> so that users are able to enter email addresses with UTF8 domains (but 
> the local part is restricted to all ASCII). The email client does 
> punycode conversions on the domain part so that the messages sent are 
> traditional email. I know there is one implementation that does this; 
> I don't know how much it is used at this point. As the time stretches 
> out to 2009 to get a standards track EAI completed, these "IDNA" email 
> systems might be more widely used. So we have a situation where EAI 
> supports traditional email via Downgrade but possibly not "IDNA" email.
>
> As a concrete example of this, ICANN has an email test page for 
> testing International top level domains at 
> http://idn.icann.org/E-mail_test. Their second test is to copy an 
> email address of the form ascii@non-ascii into an email client.
Into the "from" or the "to" fields?
My opinion is slightly different for the two cases....
> Our current EAI implementation fails this test, following a strict 
> adherence to the current EAI specs. I don't see any reason an 
> implementation can't decide it will do the punycode conversion in 
> Downgrade. It is supported by section 2.2 in draft-ietf-eai-smtpext: 
> "An SMTP Client . . . MAY transmit the domain parts of mailbox names 
> within SMTP commands or the message header in either the form of ACE 
> labels as specified in IDNA [RFC3490] or as UTF-8 strings." If we 
> leave this decision up to the implementation, then we have a situation 
> where EAI may or may not inter-operate with IDNA email. I think it 
> would be better to have a clear statement on this, either EAI supports 
> inter-operability with IDNA email or it does not.
>
> Ernie
>
>
>
>
>
>

---

From: Ernie Dainow <[email-masked]>
Date: Thu, 20 Mar 2008 11:37:14 -0400

Harald Tveit Alvestrand wrote:
>> UTF8SMTP compliant or not is of interest to a relatively few number 
>> of technically knowledgeable email users compared to the millions of 
>> so that users are able to enter email addresses with UTF8 domains 
>> (but the local part is restricted to all ASCII). The email client 
>> does punycode conversions on the domain part so that the messages 
>> sent are traditional email. I know there is one implementation that 
>> does this; I don't know how much it is used at this point. As the 
>> time stretches out to 2009 to get a standards track EAI completed, 
>> these "IDNA" email systems might be more widely used. So we have a 
>> situation where EAI supports traditional email via Downgrade but 
>> possibly not "IDNA" email.
>>
This particular case was copied into the To field.
More completely, the two failure cases are
      UTF8SMTP                   "IDNA" Server
1)   From: ascii@ascii.         To: ascii@non-ascii  
2)   From: ascii@non-ascii   To: ascii@ascii         
No Alternate Addresses for any.
Note that if the user replaces the non-ascii domain with its punycode 
translation, the email is sent.

>> leave this decision up to the implementation, then we have a 
>> situation where EAI may or may not inter-operate with IDNA email. I 
>> think it would be better to have a clear statement on this, either 
>> EAI supports inter-operability with IDNA email or it does not.
>>
>> Ernie
>>
>>
>>
>>
>>
>>
>