Subject: [EAI] Conversations in other forums, wiki changes, question about local-parts

From: John C Klensin <[email-masked]>
Date: Thu, 04 Dec 2014 12:30:58 -0500

Hi.

As many of you know but some may not, not only was there a very
successful (at least IMO) meeting about the SMTPUTF8 extensions
and internationalized mail in Beijing at the end of October
(thanks to Jiankang Yao), but there has been a recent discussion
about address syntax in W3C forms and the W3C
internationalization group.  

A few issues have come up in the various discussions that
inspired me to make some changes to the WG Wiki.  You might want
to take a look at them and either edit or raise issues here if
you disagree.   If anything I've said seems controversial,
please do identify the issues here.

As a reminder, the wiki is at
https://tools.ietf.org/wg/eai/trac/wiki

In addition, the folks working on Forms, notably Steven
Pemberton (copied) have noticed that the mailbox local-part
syntax of RFC 6532 allows some truly nasty Unicode strings,
including strings that start with combining marks and that
therefore may be very hard to have as input.  My recollection is
that was an intentional decision and not a mistake -- as with
passwords, hard-to-type strings can sometimes be an advantage in
email addresses.   However, it seems to me that they are very
much like case-sensitive strings in RFC 5321: we insist that
those be treated as separate up to the point of final delivery,
we forbid any message originating systems from assuming case
equivalence, and forbid intermediate systems from performing
case folding.  Then we advise those creating mailboxes or
operating delivery servers that they probably should not take
advantage of case-independence except in special circumstances
because case sensitive addresses are almost certain to create
confusion and to be mishandled by badly-written implementations.

It seems to me that we should say much the same thing about
badly-formed Unicode strings: support for them is required but
delivery servers that allow creation of mailbox names containing
such strings had really better know what they are getting into
and, otherwise, probably shouldn't.

If there is general agreement about that principle (or no strong
objections) I'll patch a note into the Wiki so we can come back
to it and formally see if there is consensus when we review the
documents for Full Standard.

Of course, if my memory is wrong and allowing such strings was
just a mistake, someone should start a discussion on that
subject and we should figure out what to do about it.

     john

---

From: Franck Martin <[email-masked]>
Date: Thu, 4 Dec 2014 18:22:41 +0000

> documents for Full Standard.
I’m worried that the message here is “figuring out that 2 UTF8 strings are equivalent is left as a trivial exercise for the reader”

The gmail dot introduce another level of equivalence besides the capitalization equivalence for ASCII email addresses.

Another use case, you could have your email in a do not send list, but just adding +<string> to an email address would bypass most blocking lists….

It could have some serious implications on all the legal unsubscribing requirements…

In short, with EAI we need more than ever a standard to define LHS equivalence.

---

From: John C Klensin <[email-masked]>
Date: Thu, 04 Dec 2014 14:01:46 -0500

--On Thursday, December 04, 2014 18:22 +0000 Franck Martin
<[email-masked]> wrote:

>...
>> If there is general agreement about that principle (or no
>> strong objections) I'll patch a note into the Wiki so we can
>> come back to it and formally see if there is consensus when
>> we review the documents for Full Standard.

> I'm worried that the message here is "figuring out that 2
> UTF8 strings are equivalent is left as a trivial exercise for
> the reader"

No.  The message is much simpler and, again, comes directly out
of 5321 (and 821 and everything in between).   There simply are
no equivalence rules other than exact matching of bit strings.
To spell it out in detail, it is

 Unless you are the delivery server, you should not even try to
figure out whether two local-parts are equivalent (whether they
are all-ASCII or are in UTF-8 and contain characters outside the
ASCII repertoire).  You can safely assume that they are the same
if they are bit-string identical.  Otherwise, they are not
equivalent.  They might be aliases, but, again unless you are
the delivery server or have out of band information, you have no
more reason to assume "Joe" and "joe" are equivalent to you do
to assume that "Frank" and "Franck" or "Tom" and "Dick" are.
You can't even assume that normalized and unnormalized forms of
a string are equivalent.  If you are the delivery server, you
have to consult your own alias tables and any code you have that
simulates aliases.

> The gmail dot introduce another level of equivalence besides
> the capitalization equivalence for ASCII email addresses.

I don't know what you are talking about but, if it applies and
is relevant at any point other than on the delivery server, it
violates the standard -- not just the EAI standards but 5321.

> Another use case, you could have your email in a do not send
> list, but just adding +<string> to an email address would
> bypass most blocking lists….

Yep.   Just as the price of my using subaddresses extensively is
that, in principle, I have to discourage senders one subaddress
form at a time.  Life is hard, but, again, the EAI work doesn't
introduce anything new in that area.

> unsubscribing requirements…

See above, but I actually don't think so.
 
> In short, with EAI we need more than ever a standard to define

As with many other i18n things, the SMTPUTF8 extensions make
some issues a little more evident than they were before, but
this is really not an EAI problem.  Take it up on the SMTP list.
Given that we had this discussion in the Host Requirements work
that produced RFC 1123, in DRUMS (and hence 2821), and when 5321
was produced, and reached the same conclusion each time, good
luck getting traction there, but you can try.

best,
    john

---

From: Franck Martin <[email-masked]>
Date: Thu, 4 Dec 2014 22:26:06 +0000

> To spell it out in detail, it is

It may not be the job of EAI WG to do this standard, but it is a bit frustrating that while IETF is all about security, a protocol is designed and the “increased" consequences are left to others to deal with… 

but let’s talk about something more interesting. Any insight on the meeting in China you can share with this group?

---

From: "Martin J. Dürst" <[email-masked]>
Date: Fri, 05 Dec 2014 15:38:24 +0900

> documents for Full Standard.

I agree.

> subject and we should figure out what to do about it.

I think your memory is correct. We may want to point to some of the 
results of PRECIS (which hopefully soon will finish) to give some 
recommendations about what delivery servers might want to do.

Regards,   Martin.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 5 Dec 2014 09:17:04 +0100

On Thursday, December 4, 2014 8:01:46 PM CEST, John C Klensin wrote:
> ASCII repertoire).

Corollary 1: If you are an address book implementation, you should not even 
try to compare and sort addresses the way your users do.

It's a good thing that software can't talk back ;)

Arnt

---

From: John C Klensin <[email-masked]>
Date: Fri, 05 Dec 2014 13:43:52 -0500

--On Thursday, December 04, 2014 22:26 +0000 Franck Martin
<[email-masked]> wrote:

>> No.  The message is much simpler and, again, comes directly
>> out of 5321 (and 821 and everything in between).   There
>> simply are no equivalence rules other than exact matching of
>> bit strings. To spell it out in detail, it is

> It may not be the job of EAI WG to do this standard, but it is
> a bit frustrating that while IETF is all about security, a
> protocol is designed and the "increased" consequences are
> left to others to deal with… 

First, I think you would find many people who would disagree
with "while IETF is all about security" -- I might have said, at
most, that IETF is about protocol design for the Internet and
that makes the Internet better, and security is one of the
important aspects to be considered in those protocol designs".
I wouldn't claim consensus for that view either, but...

More important, it really isn't "increased consequences", it is
about a 30+ year old rule that examining some i18n issues might
make a little more obvious.  And, again, my personal experience
is that internationalizing almost any existing protocol has the
same effect, i.e., introducing very new new problems but making
existing problem cases more clear and sometimes promoting former
obscure edge cases into more of the mainstream.  FWIW, at least
at the applications layer that situation typically comes as more
of a surprise for protocols developed during the period when
Unix and Unix-inspired systems running ASCII (or ISO 8859-1) on
byte or 32-bit-word-oriented environments could be assumed to be
all that existed on the Internet than for protocols that were
either developed with Unicode in mind or developed at a time
when a variety of different operating systems, on different
platforms, and with a variety of repertoires, coded character
sets, and encoding systems were the norm.

> but let's talk about something more interesting. Any insight
> on the meeting in China you can share with this group?

Yes, but no time.  Several others on this list, including the
primary organizer, were there and should comment when they have
time.  I am still hoping for posted slides and/or papers and
notes, but assume that JianKang and his colleagues have been at
least as busy as I have.  

I think you probably know Don Hollander.  He circulated some
personal notes to some of the attendees and, I assume, to some
APTLD list(s).  You might ask him for a copy and/or permission
to post a link to the WG Wiki.

     john

---

From: John C Klensin <[email-masked]>
Date: Fri, 05 Dec 2014 14:30:21 -0500

--On Friday, December 05, 2014 15:38 +0900 "\"Martin J.
Dürst\"" <[email-masked]> wrote:

>...
> We may want to point to some of the results of PRECIS (which
> hopefully soon will finish) to give some recommendations about
> what delivery servers might want to do.

Ack.  I've hacked together a few paragraphs and, as suggested,
mentioned PRECIS.  Someone might try to remember to replace the
pointer to the WG with a list of RFCs when that is feasible and
appropriate.

There are now three questions for the WG that don't need
immediate answers:

(1) When are we really to start the work to move the technical
specs to full standard?  It has certainly been enough time, so
the question is equivalent to when someone is going to feel
enough confidence to put together an implementation report.

(2) My sense is that all, or almost all, of the specs need work,
at least to clarify assorted issues (many of them editorial),
but that most of the work will need to be the incorporation of
the types of guidance that are now in the Wiki into 6530bis.
When do people think we will be ready to start on that?  Or do
people disagree and believe the current documents can be moved
to full standard without being revised?

(3) The last months of the WG were not characterized by a lot of
enthusiasm and activity.  Whether a move to full standard is
attempted with individual document refined on the mailing list
or by restarting the WG, it is clear that me that some strong
evidence of people willing to do work is going to be needed.
People should start thinking about whether they are interested
enough to invest time and resources or if we should just let
this go.

best,
   john