Subject: [EAI] [IETF] Internationalized Email Internet Draft

From: "Harish Chowdhary" <[email-masked]>
Date: 6 Oct 2016 05:54:47 -0000

Dear All,Greetings!We have just posted an INTERNET DRAFT&nbsp; on deployment Issues of International Emails on IETF.https://datatracker.ietf.org/doc/draft-elkchow-iea-deploy/?include_text=1Here is a brief description of what we want to discuss during IETF 97 (Seoul,South Korea)International Email Addresses (IEA) are far from the global reality. The current de-facto language of the Internet is English.&nbsp; Even today, many of the users of the Internet do not speak English as their primary language.&nbsp; The next billion users of the Internet are likely to be even less familiar with English. IEA is probably the first application needed in a truly internationalized Internet.&nbsp; The Email Address Internationalization (EAI) Working Group defined the RFCs to support internationalized email.&nbsp; The time may now finally have come to develop best practices and to discuss the deployment challenges for IEA.Your feedback and suggestions are most welcome to make above cited draft inclusi
 ve of all the matters related to &quot;Deployment of International Emails.We may further extend it to take up all the UA related issues.Hoping for your support.Thanks,Harish Chowdhary

-------------------------------------------------------------------------------------------------------------------------------
[NIXI is on Social-Media too. Kindly follow us at:
Facebook: https://www.facebook.com/nixiindia & Twitter: @inregistry ]
This e-mail is for the sole use of the intended recipient(s) and may
contain confidential and privileged information. If you are not the
intended recipient, please contact the sender by reply e-mail and destroy
all copies and the original message. Any unauthorized review, use,
disclosure, dissemination, forwarding, printing or copying of this email
is strictly prohibited and appropriate legal action will be taken.
-------------------------------------------------------------------------------------------------

---

From: "HANSEN, TONY L" <[email-masked]>
Date: Thu, 6 Oct 2016 16:39:36 +0000

I think getting deployment feedback from EAI is important, and this draft is an excellent start.

I’m not convinced that section 1.2 describes a real problem. People do this all the time today with various combinations of languages. Why is the combination of Russian and Chinese any different? If you think it is, then please expand on the aspect that does make it more difficult.

I forwarded a number of nits to the authors.

                Tony Hansen

From: IMA <[email-masked]> on behalf of Harish Chowdhary <[email-masked]>
Reply-To: "[email-masked]" <[email-masked]>
Date: Thursday, October 6, 2016 at 1:54 AM
To: "[email-masked]" <[email-masked]>
Subject: [EAI] [IETF] Internationalized Email Internet Draft

Dear All,
Greetings!

We have just posted an INTERNET DRAFT  on deployment Issues of International Emails on IETF.

https://datatracker.ietf.org/doc/draft-elkchow-iea-deploy/?include_text=1

Here is a brief description of what we want to discuss during IETF 97 (Seoul,South Korea)

International Email Addresses (IEA) are far from the global reality. The current de-facto language of the Internet is English.  Even today, many of the users of the Internet do not speak English as their primary language.  The next billion users of the Internet are likely to be even less familiar with English. IEA is probably the first application needed in a truly internationalized Internet.  The Email Address Internationalization (EAI) Working Group defined the RFCs to support internationalized email.  The time may now finally have come to develop best practices and to discuss the deployment challenges for IEA.

Your feedback and suggestions are most welcome to make above cited draft inclusive of all the matters related to "Deployment of International Emails.

We may further extend it to take up all the UA related issues.

Hoping for your support.

Thanks,
Harish Chowdhary

---

From: John C Klensin <[email-masked]>
Date: Sun, 09 Oct 2016 22:57:48 -0400

--On Thursday, October 06, 2016 4:39 PM +0000 "HANSEN, TONY L"
<[email-masked]> wrote:

> I'm not convinced that section 1.2 describes a real problem.
> People do this all the time today with various combinations of
> languages. Why is the combination of Russian and Chinese any
> different? If you think it is, then please expand on the
> aspect that does make it more difficult.
> 

Hi.  I was going to hold off until some later and more mature
version of this draft, but since Tony has commented, while I
believe the issues with EAI deployment are important, I see
several problems with this draft, some of which were actually
discussed in the WG but appear to be ignored here.  Perhaps more
important, it is seriously incomplete relative to issues that
have been discussed at great length in the EAI WG, at the APEC
meeting on internationalized email in Beijing in October 2014,
the May 2015 workshop in Thailand, and elsewhere.  I strongly
suggest that, if there is going to be a discussion in Seoul,
this document is in need of a great deal of work first.  Some of
those issues are:

(1) The so-called EAI standards, as listed in the Introduction,
are about email envelope and header information presented
directly (e.g., in UTF-8) as non-ASCII characters.  A good deal
of the document appears to address mail content information such
as textual message bodies, in other scripts.  With the possible
exception of language selection when a message is sent with the
same basic text in several languages (multipart/alternative was
designed with that case in mind but have been used in other
ways), we thought we solved that content problem with MIME in
1992.  If MIME is inadequate, the authors or others should
produce a document explaining the issues and not confuse them
with EAI / SMTPUTF8.  If it is adequate, then, like Tony
although perhaps for different reasons, I don't see what Section
1.2 is doing here, what the relevance of Section 3.2 is, and
several other statements should be examined carefully to be sure
they are talking about addresses and/or headers and not content.

(2) Within an address, there is, as the I-D points out and
consistent with RFC 5321, a local part and a domain part.  RFCs
6530 and 6531 make it quite clear (at least we thought they did)
that they are handled differently.  For the domain part, the
rules are laid out in the IDNA2008 specs (RFC 5890ff).  Issues
about look-alike characters have been extensively discussed and
written about (even though some of us have questioned the
quality of some of that work).  It does not seem useful to me to
revisit those issues here, especially without reference to the
prior work and discussions or if some of the discussion here is
wrong or contains obvious omissions.  As an example from the
first paragraph of Section 6.1, Latin "c" (U+0063) and Cyrillic
"c" (U+0441) are typically written with identical graphemes, but
are not on the list.    More important, while the "paypal"
example with U+0430 substituted for "a" (U+0061) has been used
repeatedly, including in a careful study in an article that is
not cited in this draft, it is possible to write "раура1"
with the first five characters in Cyrillic and the last one a
digit (which is script independent)
(\u'0440'\u'0430'\u'0443'\u'0440'\u'040'\u'0031' [1]), therefore
not even violating conventions prohibiting mixed-script labels.
There is, of course, no ambiguity in the A-label form, although
the authors quite properly point out that it is not
user-friendly.

By contrast, Section 1.1 talks about display of email addresses,
including the local part ("in Punycode" [2]).  While a mail
delivery server is free to create whatever aliases for a mailbox
local part it likes, including "xn-t2bmh3a" or "123456",
"george" or "example", in general converting a local part using
the Punycode algorithm and displaying the result is prohibited
by the EAI standards (and, incidentally, RFC5321).  More
important, it will often lose information and is potentially
very dangerous.

(3) Arabic should not be confused with a strictly right-to-left
writing system.  I am not aware of any such systems in wide use
for contemporary languages today.  The problem is that numerals,
whether written in European digits, Arabic or Arabic-Indic
digits, Chinese (Han) digits, or many others, have been written
left to right since that type of positional notation was
invented and became widely used.  As a result, the scripts are
referred to (in Unicode-speak) as "bidirectional" or "bidi" [3].
Their implications for domain names and IDNA are the subject of
RFC 5893.

(4) Multiple addresses for one user (and Section 4).  Keeping in
mind that many people maintain a number of identities, and even
multiple email addresses, for different purposes, I don't
understand what point you are trying to make with this section.
Many of us believe that users who have mailboxes whose names
involve non-ASCII local parts and who engage in communications
outside their primary language group will find it necessary to
maintain either separate all-ASCII mailboxes or all-ASCII
aliases to their primary mailboxes and to do so for a very long
time.  That issue has been extensively analyzed and discussed
but this document avoids that work, which is both a problem and
an opportunity.

(5) Section 2.1 asserts that email servers), implying all of
them, store data (messages?) in relational databases.  That is
simply false.  Some do; others don't.   Even for those that do,
there may be a difference between Unicode-capable data storage
and Unicode-capable keys or indexes.   There is also absolutely
no requirement that any such system store Unicode strings
encoded in UTF-8; many do not.  

(6) There is a necessary difficulty with SMTPUTF8, which is that
one cannot transmit a message with non-ASCII characters in
addresses or headers to a system that does not support them.
Final delivery systems should probably not accept messages
unless they have reason to predict that the mail store will
handle them _and_ that the user associated with the target
mailbox will be able to retrieve them.  Since a user with an
all-ASCII mailbox name might still receive a message with, e.g.,
a non-ASCII backward-pointing address in the envelope or
headers, making that decision is not straightforward.  That
leads to a strong case that, if one wants broad deployment of
SMTPUTF8, the place to start is with the MUAs (including the
Webmail systems) and associated POP and IMAP servers and
clients.   The "to various extents" list in the first part of
Section 3 is not particularly helpful in that regard.

(7) Finally, this is an internationalization (i18n) problem as
much as it is an email problem.  Terminology (and, where
characters or code points are referred to, their precise
identification) is very important because the alternative is
typically a good deal of user confusion about what you are
talking about and other impediments to making progress.  Saying
"English" were you mean "Basic Latin Script" or "ASCII" is not
helpful, especially given that 5321 local parts can include any
ASCII character and that ASCII is not sufficient to write
English.  Conversely, it appears that there are a few places
where, correctly or incorrectly, you really do mean "English"
when you say that.   Similarly, talking about one particular
encoding when you mean "Unicode" is confusing and may be
misleading.  RFC 6365 may give you a start on some of the issues.

regards,
    john

  -------------
[1] I recommend the authors have a look at RFC 5137.

[2] Punycode is an encoding method, not a display format.  See
RFC 5890, Section  2.3.4. 

[3] http://unicode.org/reports/tr9/

---

From: <[email-masked]>
Date: Mon, 10 Oct 2016 14:25:40 +0000 (UTC)

John,

Thanks so much for taking so much time to think about this issue and how to set a solid foundation.

Please bear with us as we are trying to catch up with the work that has been done by the WG and others.

Let us think over your comments and then I think we will start a separate thread for each comment.  Let me read and think quietly over what you have said before I reply to the comments.
 
Nalini Elkins
Inside Products, Inc.
www.insidethestack.com
(831) 659-8360

----- Original Message -----
From: John C Klensin <[email-masked]>
To: "HANSEN, TONY L" <[email-masked]>; [email-masked]
Sent: Sunday, October 9, 2016 7:57 PM
Subject: Re: [EAI] [IETF] Internationalized Email Internet Draft

regards,
    john

  -------------

_______________________________________________
IMA mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ima

---

From: <[email-masked]>
Date: Fri, 14 Oct 2016 13:30:36 +0000 (UTC)

John / Tony,
I am going to split your comments into separate threads so that I can keep track of each.   The first is about co-mingling content vs. headers.
>(1) The so-called EAI standards, as listed in the Introduction, are about email envelope and header information presented directly (e.g., in UTF-8) as non-ASCII characters.  A good deal of the document appears to address mail >content information such as textual message bodies, in other scripts.  With the possible exception of language selection when a message is sent with the same basic text in several languages (multipart/alternative was designed with >that case in mind but have been used in other ways), we thought we solved that content problem with MIME in 1992.  If MIME is inadequate, the authors or others should produce a document explaining the issues and not confuse >them with EAI / SMTPUTF8.  If it is adequate, then, like Tony although perhaps for different reasons, I don't see what Section 1.2 is doing here, what the relevance of Section 3.2 is, and several other statements should be examined >carefully to be sure they are talking about addresses and/or headers and not content.

Yes.  I see your point.   Let me say first the basic thing that we are trying to do is to discuss the holistic user experience of internationalized emails from an operational point of view.   In so doing, the co-mingling happened.  We could do a second draft for content issues or change the abstract of this one to better state what our real goal is.
Secondly, as you guys know well, there are lots of other issues with IDN, browser support, etc.   What we were actually hoping is that we could have a forum (perhaps like DNSOps or v6Ops) where we could come together to define and discuss such problems, move towards best practices (or work arounds! Not that I like that, but it happens.)   Because we have not even started on problems that we see such as search algorithm ranking of IDNs and so on.   We were hoping that others would step up to author such other drafts.
 Thanks,
Nalini ElkinsInside Products, Inc.www.insidethestack.com(831) 659-8360

regards,
    john

  -------------

---

From: <[email-masked]>
Date: Fri, 14 Oct 2016 13:50:30 +0000 (UTC)

John,

I am splitting your comment #2 into two parts: Homographic attacks and display of email addresses.

First, homographic attacks:

>(2) Within an address, there is, as the I-D points out and consistent with RFC 5321, a local part and a domain part.  RFCs 6530 and 6531 make it quite clear (at least we thought they >did) that they are handled differently.  For the domain part, the rules are laid out in the IDNA2008 specs (RFC 5890ff).  Issues about look-alike characters have been extensively >discussed and written about (even though some of us have questioned the quality of some of that work).  It does not seem useful to me to revisit those issues here, especially without >reference to the prior work and discussions or if some of the discussion here is wrong or contains obvious omissions. 

We can certainly refer to the prior work.  I will contact you offline for your kind assistance.

But, I think that the area of homographic attacks is definitely worth talking about.  If I might use the example of IPv6, I think at the IETF people think thought that IPv6 was very well defined but operationally, in the "real" world, many people had (and still have!) absolutely no idea how it works.

Many people have NO idea of how homographic attacks work and as we go forth into a truly internationalized Internet then this is very definitely a topic that needs to be discussed - and often!  

>As an example from the first paragraph of Section 6.1, Latin "c" (U+0063) and Cyrillic "c" (U+0441) are typically written with identical graphemes, butare not on the list.    More >important, while the "paypal" example with U+0430 substituted for "a" (U+0061) has been used repeatedly, including in a careful study in an article that is not cited in this draft,

Will contact you for reference

>it is possible to write "раура1" with the first five characters in Cyrillic and the last one a digit (which is script independent) (\u'0440'\u'0430'\u'0443'\u'0440'\u'040'\u'0031' [1]), therefore>not even violating conventions prohibiting mixed-script labels. There is, of course, no ambiguity in the A-label form, although the authors quite properly point out that it is not
>user-friendly.

Indeed.  I think this topic merits much discussion including the area of what the ICANN rules REALLY are in this area and whether registrars are complying.   As we get "new blood" into this area, then some of the topics that are so familiar to you, Tony, and many of the long time members of this WG will need to be revisited as the rest of us try to catch up to you. 

But, I hope it will be worth it for the new energy that new people bring.  Not to mention that as concepts proceed into operations, issues that no one had thought of surface.
 
Thanks,

Nalini Elkins
Inside Products, Inc.
www.insidethestack.com
(831) 659-8360

________________________________

regards,
    john

  -------------

---

From: <[email-masked]>
Date: Fri, 14 Oct 2016 14:49:12 +0000 (UTC)

John / Tony,

Continuing my splitting of topics!   Hope this makes some kind of sense to others.
>By contrast, Section 1.1 talks about display of email addresses, including the local part ("in Punycode" [2]).  While a mail delivery server is free to create whatever aliases for a ?>mailbox local part it likes, including "xn-t2bmh3a" or "123456", "george" or "example", in general converting a local part using the Punycode algorithm and displaying the result is >prohibited by the EAI standards (and, incidentally, RFC5321).  More important, it will often lose information and is potentially very dangerous.

This is a very interesting problem.   We are hoping to do some kind of spreadsheet or other visual where we can show what happens with a number of mail servers.  For example, what does Yahoo mail do, what does gmail do, why some clients fail, etc.  
I am in the process of setting up a demo system for all this.  Let me tell you, I have learned quite a bit.   Including about DNS queries which don't resolve properly.  Sigh.  That is still ANOTHER topic.   As I say, I want to get organized and have a good way to show this.   Not quite there yet!
 
Thanks,

Nalini Elkins
Inside Products, Inc.
www.insidethestack.com
(831) 659-8360

regards,
    john

  -------------

---

From: <[email-masked]>
Date: Fri, 14 Oct 2016 14:54:26 +0000 (UTC)

John / Tony,

>(3) Arabic should not be confused with a strictly right-to-left writing system.  I am not aware of any such systems in wide use for contemporary languages today.  The problem is that> numerals, whether written in European digits, Arabic or Arabic-Indic digits, Chinese (Han) digits, or many others, have been written left to right since that type of positional notation was> invented and became widely used.  As a result, the scripts are referred to (in Unicode-speak) as "bidirectional" or "bidi" [3].
Will change to say "bidi".  
I am very curious as to the "on the ground" implications of this.   I have heard that there is some activity in setting up a test bed for EAI for Arabic languages.   Is there anyone who can comment on real life problems and issues?

>Their implications for domain names and IDNA are the subject of RFC 5893.
Will cite.
 
Thanks,

Nalini Elkins
Inside Products, Inc.
www.insidethestack.com
(831) 659-8360

regards,
    john

  -------------

---

From: <[email-masked]>
Date: Fri, 14 Oct 2016 15:07:17 +0000 (UTC)

John / Tony,

>(4) Multiple addresses for one user (and Section 4).  Keeping in mind that many people maintain a number of identities, and even multiple email addresses, for different purposes, I >don't understand what point you are trying to make with this section. 
Sure.  People have multiple email addresses for different reasons.   The question was actually with aliases.  Maybe it is not possible to have all email consolidated to one box.
For example, if I want to have two mailboxes:
[email-masked] and नलिनी@mymailserver.com (or नलिनी@[myidnserver].[myinternationaltld]) all come to one mailbox is that possible?   Should it be possible?
I *think* people want that.   I think we will know as this all takes off.

>Many of us believe that users who have mailboxes whose names involve non-ASCII local parts and who engage in communications outside their primary language group will find it >necessary to maintain either separate all-ASCII mailboxes or all-ASCII aliases to their primary mailboxes and to do so for a very long time.  That issue has been extensively analyzed and >discussed but this document avoids that work, which is both a problem and an opportunity.
Can you point me to some of that work?  At the very least, we can cite it & I can become familiar with it.   Then, let's see what real life usage tells us about possible requirements in this area.
 
Thanks,

Nalini Elkins
Inside Products, Inc.
www.insidethestack.com
(831) 659-8360

regards,
    john

  -------------

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 14 Oct 2016 11:08:50 -0400

On Fri, Oct 14, 2016 at 02:49:12PM +0000, [email-masked] wrote:
> This is a very interesting problem.   We are hoping to do some kind of spreadsheet or other visual where we can show what happens with a number of mail servers.

I think that effort has already been undertaken by the "Universal
Acceptance Steering Group" over in ICANN.

A

-- 
Andrew Sullivan
[email-masked]

---

From: <[email-masked]>
Date: Fri, 14 Oct 2016 15:17:30 +0000 (UTC)

John / Tony,
>(5) Section 2.1 asserts that email servers), implying all of them, store data (messages?) in relational databases.  That is simply false.  Some do; others don't.  Even for those that do, there may be a difference between Unicode-capable >data storage and Unicode-capable keys or indexes.  There is also absolutely no requirement that any such system store Unicode strings encoded in UTF-8; many do not.  
This actually came from my own operational experience in implementing an IEA capable email server.   Will definitely change the wording to not imply that ALL email servers store in relational databases or store in UTF-8.   I can see that is not a requirement.

But, the problem that I had was with storing the user name, domain name, matching, etc.   Not the messages.   It was totally a problem that I ran into that the back end (whatever knows about the mailbox) had a problem with storing and matching user names & doing the DNS queries.
Let me think about how to re-write this.   We may, in the end, want to do some Best Practices kind of things so that people do not struggle for quite as long as I did!
 Thanks,
Nalini ElkinsInside Products, Inc.www.insidethestack.com(831) 659-8360

regards,
    john

  -------------

---

From: <[email-masked]>
Date: Fri, 14 Oct 2016 15:19:44 +0000 (UTC)

Andrew,

>I think that effort has already been undertaken by the "Universal Acceptance Steering Group" over in ICANN.

Will reach out to them & see if we can get the results of their efforts.   Thanks for pointing that out.   I believe Harish has already had a number of discussions with that group & knows the people there.

Nalini

---

From: <[email-masked]>
Date: Fri, 14 Oct 2016 15:31:02 +0000 (UTC)

John / Tony,

>(6) There is a necessary difficulty with SMTPUTF8, which is that one cannot transmit a message with non-ASCII characters in addresses or headers to a system that does not support them.
>Final delivery systems should probably not accept messages unless they have reason to predict that the mail store will handle them _and_ that the user associated with the target
>mailbox will be able to retrieve them.  Since a user with an all-ASCII mailbox name might still receive a message with, e.g., a non-ASCII backward-pointing address in the envelope or
>headers, making that decision is not straightforward.  That leads to a strong case that, if one wants broad deployment ofSMTPUTF8, the place to start is with the MUAs (including the

>Webmail systems) and associated POP and IMAP servers and clients.  The "to various extents" list in the first part of Section 3 is not particularly helpful in that regard.

This is a huge problem.   Maybe the biggest problem of all.  It is actually shades of the IPv6 / IPv4 problem.   I think we need to have a discussion on migration strategies.

Remember v4 - v6 tunneling?   Hope we do not end up there!   We can certainly talk more about some of the migration issues here.

The reason for the "various extents" of support is an obfuscation because I have not reverse engineered the servers / clients listed.  (Nor do I plan to!)

But, let me just say this - without playing some "games" -- that is not actually supporting the RFCs on how to transmit email, I have a feeling that some of these servers / clients would not work.  But, I do not want to say more without more thorough testing.  And, maybe not even then. 
Thanks,

Nalini Elkins
Inside Products, Inc.
www.insidethestack.com
(831) 659-8360

regards,
    john

  -------------

---

From: <[email-masked]>
Date: Fri, 14 Oct 2016 15:41:45 +0000 (UTC)

John / Tony,

>(7) Finally, this is an internationalization (i18n) problem as much as it is an email problem.  Terminology (and, where characters or code points are referred to, their precise 
>identification) is very important because the alternative is typically a good deal of user confusion about what you are talking about and other impediments to making progress.  Saying 
>"English" were you mean "Basic Latin Script" or "ASCII" is not helpful, especially given that 5321 local parts can include any ASCII character and that ASCII is not sufficient to write 
>English.  Conversely, it appears that there are a few places where, correctly or incorrectly, you really do mean "English" when you say that.  Similarly, talking about one particular 
>encoding when you mean "Unicode" is confusing and may be misleading.  RFC 6365 may give you a start on some of the issues.

Thanks for pointing that out.  I will go through the draft and fix.

Thanks,

Nalini Elkins
Inside Products, Inc.
www.insidethestack.com
(831) 659-8360

----- Original Message -----
From: John C Klensin <[email-masked]>
To: "HANSEN, TONY L" <[email-masked]>; [email-masked]
Sent: Sunday, October 9, 2016 7:57 PM
Subject: Re: [EAI] [IETF] Internationalized Email Internet Draft

regards,
    john

  -------------

---

From: John C Klensin <[email-masked]>
Date: Fri, 14 Oct 2016 11:45:18 -0400

Nalini,

Given your decision to go for separate threads, which I think is
a good one, some general comments:

--On Friday, October 14, 2016 13:30 +0000
[email-masked] wrote:

> John / Tony,
> I am going to split your comments into separate threads so
> that I can keep track of each.   
>...
> Yes.  I see your point.   Let me say first the basic thing
> that we are trying to do is to discuss the holistic user
> experience of internationalized emails from an operational
> point of view.   In so doing, the co-mingling happened.  We
> could do a second draft for content issues or change the
> abstract of this one to better state what our real goal is.
> Secondly, as you guys know well, there are lots of other
> issues with IDN, browser support, etc.   What we were
> actually hoping is that we could have a forum (perhaps like
> DNSOps or v6Ops) where we could come together to define and
> discuss such problems, move towards best practices (or work
> arounds! Not that I like that, but it happens.)   Because we
> have not even started on problems that we see such as search
> algorithm ranking of IDNs and so on.   We were hoping that
> others would step up to author such other drafts.

A few very general comments about context.  This might
anticipate some future note from you, so I hope I can get it off
quickly...

(1) I trust you are aware of the ICANN=-sponsored "Universal
Acceptance" effort, which is covering some of this same ground.
Personally, I believe they are seriously out in the weeds, doing
a good deal of work on assumptions about the DNS and the EAI
work that are just not valid for a number of technical reasons,
but it would be unfortunate to waste whatever cycles are
associated with duplicative efforts.  As far as I know, Don
Holland is still lead; I think he is on the EAI list but am not
sure.

(2) I trust you are also aware that there is an IAB I18N
program.  It has been, again IMO, fairly unhealthy in the last
couple of years, largely because the available cycles of the
most active and expert people have disappeared into the IANA
Transition activity.  If you want a forum or workshop, they
might be a good place to start.  Ted Hardie is lead; I have no
idea whether he is on the EAI list.

(3) Despite the obvious interest and importance of this issue, I
have observed very little actual energy in the IETF for dealing
with it or i18n issues more generally.  By the time the EAI WG
got its documents out, there was little interest in specific
work beyond "we just want this to work".  By the time the PRECIS
WG got its RFCs out, there was little general interest in
careful review of documents (resulting in a need to start work
on revisions almost immediately thereafter).  AFAICT, the
strongest and most general sentiment in the WG was "just tell me
what to do so I don't need to understand or think about this".
That is understandable, but not a good foundation for getting
work done.  The community has been completely unable to engage
with the "non-decomposing character" issue that has paralyzed
some important IDNA applications for a couple of years now (IMO,
the LUCID BOF, supposedly addressed to that issue, did a good
job of illustrating the more fundamental problems with IETF
engagement, including a lengthy discussion of just how to spell
Zürich in English (or, for that matter, in various versions of
German).  

(4) Even among those who were active in the EAI work, there has
been, AFAICT, silence so far except from Tony and myself.  I
know that some are off chasing general equivalence of DNS names.
I believe that effort shows a fundamental lack of understanding
of the properties of the relevant data structures but it seems
to be getting agenda time in Seoul (see "DNSBUNDLED" on Jari's
"New Work in Seoul" BOF list).

(5) You should also be sure you understand that many of the
issues with non-ASCII identifiers, especially identifiers 

(5) If you want rapid deployment of fully-internationalized
email messages, it is important that you understand that there
were alternate hypotheses for how to get both the IDN and that
work done.   For IDNs, we could have changed the DNS and server
matching algorithms (which might have permitted approximate
matching and made a number of issues with comparison and
normalization much easier) or we could have adopted an "above
DNS" model which would have also helped with the matching and
search issues. However, there was a great deal of pressure, some
of it emotional and paralleling the comments about "English" in
your draft, for "fast" and "in the DNS", and that got us the
mappings, restrictions, and trick encoding of IDNA.   Every bit
of new evidence or pressure for synonymous labels and matching
that is predictable given local language culture is an argument
that we got it wrong and should have gone for one of the other
approaches, probably a member of the "Above DNS" family.   For
EAI, there was a similar choice.  We could have stuck with
all-ASCII address local parts and focused on better use of
encoded names (e.g., by making it clear that not copying a name
phrase into a reply or to or from an address book was a very bad
practice, encouraging address book lookups by the name phrase or
equivalent as well as the address, etc., thereby saving a good
many transition and interoperability problems.  I note that was
the solution adopted by the ITU, an organization with far more
representation and influence from countries and people who speak
English as a second language if at all, for a broad range of
I19n issues.   Instead, there was a strong commitment in the WG
and those who pushed for it to have all-non-ASCII mailbox
addresses and to phase out the encoded word approach.  At least
a significant fraction of the WG understood the likely (or
certain) costs of that decision including slow deployment and
interoperability issues that would likely limit EAI use to
"within community" communications.  If, at any point, you want
to argue for discarding SMTPUTF8 and returning to an encoded
word model, there are less difficult (but also less culturally
and aesthetically attractive) ways to move forward.

best,
   john

(BTW, as a further illustration of the "too few people with
spare cycles" problem, I'm writing this note rather than working
on the current round of PRECIS revision drafts (and they are, in
turn, holding up work in URNBIS) and am copying Peter, who I
don't think is on the EAI list either, so he knows what is
happening.)

---

From: <[email-masked]>
Date: Fri, 14 Oct 2016 15:56:55 +0000 (UTC)

Tony,

>I’m not convinced that section 1.2 describes a real problem. People do this all the time today with various combinations of languages. Why is the combination of Russian and Chinese any >different? If you think it is, then please expand on the aspect that does make it more difficult.

I think what we were thinking of is the actual difficulties involved in typing or user interface on a desktop.  For example, if you are speaking in a particular language and don't have the associated keyboard for it, then you might have to bring up a virtual keyboard to type in a particular language.  Then, you might have to set the language preference on your desktop.  
If you use a second language, then it becomes more complicated.   So, you have two virtual keyboards + an actual physical keyboard?  Hmm.
I guess I am also assuming that in real life, you would actually be speaking both languages and not trying to do something like a "google translate" to read the message from the second language.  Let me think how to rephrase this section.
Also, I think we have to distinguish between mobile and desktop.   On my cell phone, I easily go back and forth from, for example, my english keyboard, and emoji or bitmoji keyboard.  So, I think that it would be easy enough to go from english to hindi to french and then back to insert some emoji's.   Just thinking out loud about how this would actually work.  (This is making me think that maybe I text too much!  Why am I doing all this back and forth even now? )
 
>I forwarded a number of nits to the authors.
Thanks!
Nalini 
                Tony Hansen

Dear All,
Greetings!

International Email Addresses (IEA) are far from the global reality. The current de-facto language of the Internet is English.  Even today, many of the users of the Internet do not speak English as their primary language.  The next billion users of the Internet
 are likely to be even less familiar with English. IEA is probably the first application needed in a truly internationalized Internet.  The Email Address Internationalization (EAI) Working Group defined the RFCs to support internationalized email.  The time
 may now finally have come to develop best practices and to discuss the deployment challenges for IEA.

Hoping for your support.

Thanks,
Harish Chowdhary

---

From: <[email-masked]>
Date: Fri, 14 Oct 2016 16:28:56 +0000 (UTC)

John,

>Holland is still lead; I think he is on the EAI list but am not

Yes.  I am aware of this & want to get Don Holland involved.  I somehow had the idea he would be in Seoul but maybe I am wrong.  I think I will try to have a private conversation with him and Harish ahead of time.

>(2) I trust you are also aware that there is an IAB I18N>program.  It has been, again IMO, fairly unhealthy in the last
I had a very brief conversation with Ted Hardie over email.  I will contact him again.

>"New Work in Seoul" BOF list).

Sure.  I understand your frustration.  
Harish and I have been in touch with the community in Africa and Latin America as well, as of course, India.   If you will, since this is more of a problem for the non-English speaking world, then those of us who actually speak languages other than English should step up to the plate and be responsible for doing work.  One of the representatives for one of the communities has told me that they may fund someone to come to Seoul just to work on this project.
We have also started a conversation with some of the OS / browser manufacturers - the Mozillas and Microsofts of the world.
I want to get a core group of maybe 5 - 10 people from various communities and then people from the OS / browser world.  It will take some time to get critical mass but as you say, the issue is very critical.  That is definitely being seen at least in the non-Western parts of the world.  (Sorry, I don't know if I am being somehow politically incorrect in phrasing this!)
I hope that you and Tony can guide us.  It will take us some time to come up to speed.  It has definitely helped to do a proof of concept implementation.  Harish and I conference call every week sometimes for two hours to discuss these issues.   We have learned a lot in the past two months.  Mostly based on problems that I have had!
We appreciate very much your and Tony's efforts to get the WG and documents to this stage.  But, now we need to work on implementation and migration issues.   I hope you will be available to guide us.  I am not thinking that we will expect you to be responsible for doing all the work.
I was born in India.  Many, many of the people in India do not (and never will) speak English.   Two of the happiest years of my life were living at the edge of the bush in French speaking West Africa.  Again, English was not widely spoken. 
But, English is the de-facto language of the Internet.  The next billion people on the Internet need access in local language.   I think that at the IETF one thing that we can do is quietly and thoughtfully concentrate and work out the technical issues.   That is our strength.   It will take us some time to form a core group.   My hope is that by IETF99, we will have enough people to do something interesting.

>issues with non-ASCII identifiers, especially identifiers 

Yes.   Are there any RFCs or papers that you might point me to?

>and aesthetically attractive) ways to move forward.

This is very interesting.  Let me think it over and we may want to discuss in our core group.   We also need the people from DNS involved.  Let me talk to a few people over there & pick their brains.

best,
  john

>don't think is on the EAI list either, so he knows what is
John, thank you for everything you have done & for all your help with this.
Nalini

---

From: John C Klensin <[email-masked]>
Date: Fri, 14 Oct 2016 12:34:33 -0400

--On Friday, October 14, 2016 14:49 +0000
[email-masked] wrote:

>> By contrast, Section 1.1 talks about display of email
>> addresses, including the local part ("in Punycode" [2]). 
>> While a mail delivery server is free to create whatever
>> aliases for a ?>mailbox local part it likes, including
>> "xn-t2bmh3a" or "123456", "george" or "example", in general
>> converting a local part using the Punycode algorithm and
>> displaying the result is >prohibited by the EAI standards
>> (and, incidentally, RFC5321).  More important, it will
>> often lose information and is potentially very dangerous.

> This is a very interesting problem.   We are hoping to do
> some kind of spreadsheet or other visual where we can show
> what happens with a number of mail servers.  For example,
> what does Yahoo mail do, what does gmail do, why some clients
> fail, etc. 

I am glad to see another one, but note that this was done with
what was then available before the Beijing workshop, that
something similar has been done (or claimed to be done) in the
"Universal Acceptance" group (see below), and maybe elsewhere.

You also need to be very careful about the tests you run.  For
example, on delivery and when referring to its own (virtual)
mailbox names, gmail apparently discards some or all ASCII
delimiter characters in local parts, treating local parts
containing such characters as equivalent to ones with the
characters dropped.  I have no idea what they do with delimiter
characters from other scripts, but some other systems would
consider that delimiter-dropping behavior pathological or worse.
A different way of looking at that is that they are preserving
the local parts but dropping certain characters in mapping from
mailbox name to actual storage.  Either is fine with the
protocols as long as the computational aliasing is done only at
or beyond the delivery server.  

There are also a large number of issues at the boundaries
between email and HTML/HTTP that affect many implementations
that fail to account for the differences.  That problem, and to
some extent the gmail behavior above, are examples of another
situation we see quite often: the problems are there even in
all-ASCII strings and identifiers, i18n simply amplifies them
and makes them more obvious.

>  I am in the process of setting up a demo system
> for all this.  Let me tell you, I have learned quite a bit.

Sadly, I am not surprised.  I am also not surprised if a good
deal of what you have learned has been painful.  It certainly
has been for many of the rest of us.  I think it is also
important to keep in mind that most of the underlying issues are
ultimately the result of variations and evolution in human
language and writing systems.   Some of the decisions we have
made in the IETF, and that the Unicode Consortium has made, have
made some issues harder and some easier but, as long as we want
to deal with the full range of human languages and writing
systems, most of the problems are inherent and the choices end
up being about compromises (or, if you prefer, winners and
losers).  Some of the relationships are rather old.  For
example, rather explicit decisions were made many centuries ago
that Latin and Chinese characters should be easily
distinguishable by people with some familiarity with them but
without necessarily being literate.  Latin ended up with a lower
familiarity requirement by having very few characters; Chinese
ended up with many characters and the advantages of a single
writing system that could express meaning of many languages,
even ones that are mutually incomprehensible while Rome took the
approach that everyone should simply learn Latin.  That  makes
the two scripts special even though some evolution to both in
more recent centuries has muddied the distinguishability
somewhat.

>   Including about DNS queries which don't resolve properly.
>  Sigh.  That is still ANOTHER topic.   As I say, I want to
> get organized and have a good way to show this.   Not quite
> there yet!  

See Andrew's comment (and my earlier one) about the Universal
Acceptance effort.   But I suggest you need to go further than
reaching out to them.  The sad reality is that there are very
few people in the Internet (not just IETF) community with a good
understanding of email protocols and operations, DNS and IDNA
protocols and operations, writing systems and character coding
including the many writing system issues that have nothing to do
with character coding, and Unicode operations.  On many days,
I'm not even sure I'm part of that rather small group.   Most of
us are willing to try to explain issues to others, but only to
those who want to put the energy into listening and learning,
not just saying thing that amount to "as long as I think it
works for my language, everything else is fine" or "just tell me
what to do because I have no intention of learning or thinking
about the issues".  That expertise is spread sufficiently thin
that answering a question or reviewing a document in one places
causes something else to be delayed or not happen (e.g., as
mentioned earlier, responding to your notes, which I considered
important, has retarded progress on both PRECIS and URNBIS work
for several days).  So there are extremely pragmatic reasons for
you to try to either merge your work into the Universal
Acceptance effort (or vice versa) or to establish really clear
boundaries between the two.

And that is probably all the time I can put in on this today.

best,
    john

---

From: Franck Martin <[email-masked]>
Date: Fri, 14 Oct 2016 09:53:17 -0700

I tried to subscribe to this list using my email address [email-masked] but
mailman replied:

Your subscription is not allowed because the email address you gave is
insecure.

On Fri, Oct 14, 2016 at 6:30 AM, <[email-masked]> wrote:

> John / Tony,
>
> I am going to split your comments into separate threads so that I can keep
> track of each.   The first is about co-mingling content vs. headers.
>
> >(1) The so-called EAI standards, as listed in the Introduction, are
> about email envelope and header information presented directly (e.g., in
> UTF-8) as non-ASCII characters.  A good deal of the document appears to
> address mail >content information such as textual message bodies, in
> other scripts.  With the possible exception of language selection when a
> message is sent with the same basic text in several languages
> (multipart/alternative was designed with >that case in mind but have been
> used in other ways), we thought we solved that content problem with MIME
> in 1992.  If MIME is inadequate, the authors or others should produce a
> document explaining the issues and not confuse >them with EAI /
> SMTPUTF8.  If it is adequate, then, like Tony although perhaps for
> different reasons, I don't see what Section 1.2 is doing here, what the
> relevance of Section 3.2 is, and several other statements should be
> examined >carefully to be sure they are talking about addresses and/or
> headers and not content.
>
> Yes.  I see your point.   Let me say first the basic thing that we are
> trying to do is to discuss the holistic user experience of
> internationalized emails from an operational point of view.   In so doing,
> the co-mingling happened.  We could do a second draft for content issues or
> change the abstract of this one to better state what our real goal is.
>
> Secondly, as you guys know well, there are lots of other issues with IDN,
> browser support, etc.   What we were actually hoping is that we could have
> a forum (perhaps like DNSOps or v6Ops) where we could come together to
> define and discuss such problems, move towards best practices (or work
> arounds! Not that I like that, but it happens.)   Because we have not even
> started on problems that we see such as search algorithm ranking of IDNs
> and so on.   We were hoping that others would step up to author such other
> drafts.
>
>
> Thanks,
>
> Nalini Elkins
> Inside Products, Inc.
> www.insidethestack.com
> (831) 659-8360
>
>
> ------------------------------
> *From:* John C Klensin <[email-masked]>
> *To:* "HANSEN, TONY L" <[email-masked]>; [email-masked]
> *Sent:* Sunday, October 9, 2016 7:57 PM
> *Subject:* Re: [EAI] [IETF] Internationalized Email Internet Draft
>
>
>

---

From: John C Klensin <[email-masked]>
Date: Fri, 14 Oct 2016 20:21:23 -0400

--On Friday, October 14, 2016 09:53 -0700 Franck Martin
<[email-masked]> wrote:

> I tried to subscribe to this list using my email address
> [email-masked] but mailman replied:
> 
> Your subscription is not allowed because the email address you
> gave is insecure.

Franck,

First and most important, the IETF mail servers, or at least
ietfa.amsl.com, do not advertise SMTPUTF8 as a capability.  So,
given the issues discussed in RFC 6783 and elsewhere, it would
be really stupid for the IETF version of mailman to accept a
non-ASCII address.   

One can debate whether, under the "eat our own dogfood"
principle or something else, the IETF mail servers should be
SMTPUTF8-capable and advertising that extension.  I encourage
anyone who feels strongly that the servers should support
SMTPUTF8 at this time to take it up with the IESG and/or IAOC,
or, perhaps even better, to try some extended rants during the
Seoul plenary, perhaps delivering those rants in multiple
languages that  none of the IESG, IAOC, or IAB understand.  That
would help to make the point about how full SMTPUTF8 support
within IETF discussion lists would facilitate communication and
diversity.

Seriously, at least in the near term, I'd oppose letting anyone
post to an IETF list from a non-ASCII address.  This has been
discussed before in other contexts, but it is important that we,
as a standards body, be able to identify who is posting to our
lists and trying to influence outcomes.  The IETF possibly
doesn't go far enough in that direction -- again, a topic that
has been discussed before.  At least as long as we do our
business in English and don't have authoritative translations to
and from any other accepted languages of just about everything
available, we should stick to addresses that everyone who
participates here (in English) can read.

I'm belaboring this point only because the special issues
involving mailing lists, especially mailing lists whose
membership/recipients may involve people from more than one
language group, are yet another issue that has to be considered
carefully on the way to a world that is fully SMTPUTF8-enabled.

best,
   john

---

From: John Bucy <[email-masked]>
Date: Tue, 18 Oct 2016 17:27:06 -0700

On Fri, Oct 14, 2016 at 8:07 AM, <[email-masked]> wrote:

> John / Tony,
>
> >(4) Multiple addresses for one user (and Section 4).  Keeping in mind
> that many people maintain a number of identities, and even multiple email
> addresses, for different purposes, I >don't understand what point you are
> trying to make with this section.
>
> Sure.  People have multiple email addresses for different reasons.   The
> question was actually with aliases.  Maybe it is not possible to have all
> email consolidated to one box.
>
> [email-masked] and नलिनी@mymailserver.com (or
> नलिनी@[myidnserver].[myinternationaltld]) all come to one mailbox is that
> possible?   Should it be possible?
>
> >Many of us believe that users who have mailboxes whose names involve
> non-ASCII local parts and who engage in communications outside their
> primary language group will find it >necessary to maintain either separate
> all-ASCII mailboxes or all-ASCII aliases to their primary mailboxes and to
> do so for a very long time.
>

While we don't have concrete product plans at this time, it is my
expectation that Gmail will support adding on an eai alias to an existing
account long before we will support creating a Google Account with a
non-ascii primary username. This might look similar to the existing "custom
from" feature.

cheers
john

---

From: <[email-masked]>
Date: Wed, 19 Oct 2016 02:04:34 +0000 (UTC)

John,

On Fri, Oct 14, 2016 at 8:07 AM, <nalini.elkins@insidethestack. com> wrote:

John / Tony,

Sure.  People have multiple email addresses for different reasons.   The question was actually with aliases.  Maybe it is not possible to have all email consolidated to one box.
For example, if I want to have two mailboxes:
[email-masked] and नलिनी@mymailserver.com (or नलिनी@[myidnserver].[myint ernationaltld]) all come to one mailbox is that possible?   Should it be possible?
I *think* people want that.   I think we will know as this all takes off.

>Many of us believe that users who have mailboxes whose names involve non-ASCII local parts and who engage in communications outside their primary language group will find it >necessary to maintain either separate all-ASCII mailboxes or all-ASCII aliases to their primary mailboxes and to do so for a very long time.

>While we don't have concrete product plans at this time, it is my expectation that Gmail will support adding on an eai alias to an existing account long before we will support creating a >Google Account with a non-ascii primary username. This might look similar to the existing "custom from" feature.

Great!   Definitely a start!
If you would like to test out the alias feature, please let us know.   We will be happy to help.
Nalini

---

From: John C Klensin <[email-masked]>
Date: Wed, 19 Oct 2016 09:25:10 -0400

--On Friday, October 14, 2016 15:07 +0000
[email-masked] wrote:

>...
>> (4) Multiple addresses for one user (and Section 4). 
>> Keeping in mind that many people maintain a number of
>> identities, and even multiple email addresses, for different
>> purposes, I >don't understand what point you are trying to
>> make with this section. 

> Sure.  People have multiple email addresses for different
> reasons.   The question was actually with aliases.  Maybe it
> is not possible to have all email consolidated to one box. For
> example, if I want to have two mailboxes:
> [email-masked] and नलिनी@mymailserver.com
> (or नलिनी@[myidnserver].[myinternationaltld]) all
> come to one mailbox is that possible?   Should it be possible?

One more comment on this.  I'm encouraged by John Bucy's comment
that Gmail will support this and I hope the other large
providers will too.  However, to put your question in
perspective, ability to create aliases for mailbox names has
been common in Internet email systems (and supported by every
publicly-available one I have encountered) for a very long time.
>From the perspective of the SMTP specs, even having upper and
lower case ASCII match in the local-part is an aliasing issue,
not a fundamental requirement or transformation.   Expanding an
otherwise well-designed and "8 bit clean" mail delivery system
and server to treat an incoming non-ASCII local-part as an alias
for an all-ASCII one should be simply a matter of modifying the
server to accept non-ASCII local parts and then disabling any
internal syntax checks that prohibit such addresses.

The harder problems lie elsewhere. If the server for example.com
accepts [email-masked] and an alias for, and equivalent
to, [email-masked], whether Joe's MUA and submission
servers will allow sending from the former address form is a
matter for those servers, not the mailbox name or delivery
systems.  Some will, some won't.  Similarly, there are a few
different ways to handle aliases in terms of what, if anything,
is done to the message headers.  One can imagine a number of
interesting issues in IMAP / POP interfaces or in receiving MUAs
and that number increases when non-ASCII addresses are added in.

> Can you point me to some of that work? 

In (non-IETF) tutorials about Internet email and documents for
specific systems, yes.  Other than a comment here and there
(with case sensitively as a good example), the IETF standards
have deliberated avoided these issues because handling of
mailbox names and aliases are considered to be purely local
matters.

    best,
      john