Subject: [EAI] I-D Action: draft-ietf-eai-mailinglistbis-00.txt

From: [email-masked]
Date: Tue, 22 Nov 2011 08:44:44 -0800

A New Internet-Draft is available from the on-line Internet-Drafts directories. This draft is a work item of the Email Address Internationalization Working Group of the IETF.

	Title           : Mailing Lists and UTF-8 Addresses
	Author(s)       : John Levine
                          Randall Gellens
	Filename        : draft-ietf-eai-mailinglistbis-00.txt
	Pages           : 8
	Date            : 2011-11-21

   This document describes considerations for mailing lists with the
   introduction of internationalized email addresses.  It outlines some
   possible scenarios for handling lists with mixtures of
   internationalized and traditional addresses, but does not offer
   implementation or deployment advice.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-eai-mailinglistbis-00.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

This Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-drafts/draft-ietf-eai-mailinglistbis-00.txt

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 23 Nov 2011 17:24:53 +0100

>        Title           : Mailing Lists and UTF-8 Addresses

Hi, top down in I-D.ietf-eai-mailinglistbis-00:

|  For more information, see [listheaders].

Pointer apparently AWOL.

| the list software might want to modify the VERP bounce
| addresses to be ASCII.

Please add an example.  At the moment VERP is "specified" in
Wikipedia, maybe this draft could explain it for the IETF.

| Future work may extend  these header fields

IMO this draft can simply do this "while at it".  After the
decision to break Message-Id it is straight forward to also
break the rest of the zoo not limited to List-Id.

| Note that discussion on whether internationalized domain
| names should be percent-encoded or puny-coded, is ongoing;
| see [I-D.duerst-iri-bis].

Please strike or replace this IRIbis detail, for the purpose
of "legacy" (= existing) list tools only the "cannot punycode"
interpretation makes sense, other <iregname> constructs, i.e.,
stuff outside of LDH DNS, are not relevant for mailing lists.

| Downgrading list headers

Should that get a reference to the pop-imap-downgrade draft?

-Frank

---

From: "John Levine" <[email-masked]>
Date: 23 Nov 2011 16:32:05 -0000

>|  For more information, see [listheaders].

Bug in xml2rfc.  They know about it.

>Wikipedia, maybe this draft could explain it for the IETF.

It's per-recipient bounce addresses that embed the recipient's
address.  For this doc, I don't see any need to describe it in more
detail.

>stuff outside of LDH DNS, are not relevant for mailing lists.

But this isn't about legacy list tools.  It's about how list tools
might change to handle EAI mail.

>Should that get a reference to the pop-imap-downgrade draft?

Dunno, what do other people think.

R's,
John

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 23 Nov 2011 18:04:47 +0100

> Bug in xml2rfc.  They know about it.

Yes, I saw your messages on the xml2rfc list, but as user of
the online xml2rfc tools I can't tell how to fix your offline
installation.  For the online tool if I don't like the bibxml
output I just copy the bibxml snippet and then tweak it until
I'm satisfied. ;-)

> address.  For this doc, I don't see any need to describe it in
> more detail.

Mostly I have forgotten all VERP and BATV details, but there are
"local part length" dragons, IIRC.  For SPF "local part" simply
*is* the dragon, otherwise SPF would be EAI-ready, and all weird
security considerations for alternate addresses will be obsoleted
by EAIbis.

> But this isn't about legacy list tools.  It's about how list
> tools might change to handle EAI mail.

Where does using a percent-encoded instead of a punycoded <ihost>
help wrt mailing lists?  So far I think you can either keep UTF-8
"as is" or "get it right" (= 3987 + 5890).

-Frank

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 23 Nov 2011 18:34:56 +0100

On 23 November 2011 18:14, John R. Levine <[email-masked]> wrote:

> It's an internal reference to another section.  If they don't fix
> xml2rfc before this goes to the RFC Ed, I can do the xref by hand.

For the spf-eai draft I got oddities such as [havoc] instead of
(section x.y) with the experimental xml2rfc tool in -05, with the
release tool in -06 it was okay.  IOW, just use the online release
xml2rfc tool before wasting time with temporary manual workarounds:
<http://www.ietf.org/mail-archive/web/xml2rfc/current/msg03188.html>

>> "local part length" dragons, IIRC. ...

> So what?  For this document, all that matters is that it embeds the
> recipient's address, which will include UTF-8 characters if it's an
> EAI address.

Maybe it is only me, but figuring out the length limitations wrt IDNA
or here SMTP addresses is tricky.  If you don't want to talk about it
here maybe don't mention VERP.  The cases where it fits are "simple"
for concepts of "simple" = as in the "S" of SMTP.

-Frank

---

From: "John R. Levine" <[email-masked]>
Date: 23 Nov 2011 12:14:36 -0500

> output I just copy the bibxml snippet and then tweak it until

It's an internal reference to another section.  If they don't fix xml2rfc 
before this goes to the RFC Ed, I can do the xref by hand.

>> VERP is per-recipient bounce addresses that embed the recipient's

So what?  For this document, all that matters is that it embeds the 
recipient's address, which will include UTF-8 characters if it's an EAI 
address.

> "as is" or "get it right" (= 3987 + 5890).

I still prefer to let other people weigh in on this.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

---

From: John C Klensin <[email-masked]>
Date: Thu, 24 Nov 2011 07:42:21 -0500

Speaking strictly personally as an instance of "other people"...

--On Wednesday, November 23, 2011 12:14 -0500 "John R. Levine"
<[email-masked]> wrote:

>> Where does using a percent-encoded instead of a punycoded
>> <ihost> help wrt mailing lists?  So far I think you can
>> either keep UTF-8 "as is" or "get it right" (= 3987 + 5890).
> 

In part because [email-masked] has historically meant
something special in email (where "bar" could easily be, e.g.,
61 or 62 with [email-masked] as a routing) and in part
because we just wanted to avoid the risks and attack vectors
associated with multiple encodings, we have strongly discouraged
use of %-encodings and other tricks wrt email addresses.  The
"discussion" last week in the IRI WG that illustrated the
non-equivalence in different environments between a string in
U-label form, one in A-label form, and one that was %-encoded
further illustrates the desirability of a "just get it right"
approach.   Pop-imap-downgrade illustrates the possibility of
using the more mail-like approach of encoded words rather than
%-encoding, especially %-encoded UTF-8.

So my preferences is close to Frank's.

> Dunno, what do other people think.

Not unless you decide to use encoded words.  And maybe not then.
pop-imap-downgrade is designed to be applicable to communication
with remote mail reading clients by the mail store or delivery
MTA.

Again, just personal opinions at this point.

    john

---

From: "John R. Levine" <[email-masked]>
Date: 24 Nov 2011 11:09:18 -0500

>>> either keep UTF-8 "as is" or "get it right" (= 3987 + 5890).

You can't keep it as is in a downgraded message.  How about if I just say 
that you have to do something in downgraded messages, the obvious approach 
of leaving it in UTF-8 in EAI messages doesn't work there, and all of the 
currently proposed alternatives have problems.

Regards,

---

From: John C Klensin <[email-masked]>
Date: Thu, 24 Nov 2011 16:22:48 -0500

--On Thursday, November 24, 2011 11:09 -0500 "John R. Levine"
<[email-masked]> wrote:

> You can't keep it as is in a downgraded message.  How about if
> I just say that you have to do something in downgraded
> messages, the obvious approach of leaving it in UTF-8 in EAI
> messages doesn't work there, and all of the currently proposed
> alternatives have problems.

For the current round, ok.   In the slightly longer term, I
think we should draw from the experience of pop-imap-downgrade
(and the earlier downgrading work on which it is based) and
think of modifying the message to suppress the address and then
use encoded words for the relevant content (since we know that
"?=foo?Q?gibberi=FF" is a perfectly valid local-part).   

See draft-klensin-encoded-word-type-u (in posting queue) as the
basis for a discussion if we want to take a slightly different
direction with this.

   john

---

From: "John Levine" <[email-masked]>
Date: 26 Nov 2011 05:14:07 -0000

>> I still prefer to let other people weigh in on this.

I went back and looked at that section again.  (Excuse: it's from
Randy's old draft.)  The sentence in question says: 

  Note that discussion on whether internationalized domain names
  should be percent-encoded or puny-coded, is ongoing; see <xref
  target="I-D.duerst-iri-bis" />.

In relation to ASCII mail, it's probably worth pointing out that a few
characters, notably percent and exclamation point, have been used for
routing hacks, and it's still risky to use them in e-mail addresses.
Do we think those characters will still be risky in EAI mail?  If so,
is this the right place to mention it?

I'm thinking of rewriting the section on List-* headers just to say
that the specs for encoding UTF-8 into URLs are in flux, so even in
EAI mail, it may be a challenge to create non-ASCII List-* headers
that interoperate reliably.  How's that?

Although I think that John K's \U'abcd' looks reasonable, this draft
is currently limited to describing the problems rather than guessing
how to deal with them, so it doesn't seem appropriate to mention it.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 26 Nov 2011 05:20:20 -0000

>"?=foo?Q?gibberi=FF" is a perfectly valid local-part).   

Well, give or take that pesky 64 octet mailbox limit.

By the way, I don't see anything in 5336bis about length limits other
than adding the UTF8SMTPbis string to the MAIL command, so the maximum
mailbox length in EAI mail is still 64 octets, not 64 UTF-8 characters
or anything like that, right?

R's,
John

---

From: "Martin J. Dürst" <[email-masked]>
Date: Sun, 27 Nov 2011 09:22:41 +0900

On 2011/11/26 14:14, John Levine wrote:
>    should be percent-encoded or puny-coded, is ongoing; see<xref

Why not just point to RFC 3986, last two paragraphs of section 3.2.2 
(http://tools.ietf.org/html/rfc3986#section-3.2.2)?

That's an Internet Standard rather than an "ongoing discussion".

Regards,    Martin.

> _______________________________________________
> IMA mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ima
>

---

From: Frank Ellermann <[email-masked]>
Date: Sun, 27 Nov 2011 02:11:37 +0100

> That's an Internet Standard rather than an "ongoing discussion".

All those "<reg-name> outside of DNS" considerations in STD 66 are
IMO irrelevant for UTF8SMTPBIS addresses.  If that's correct we end
up with the last two sentences in RFC 3986 3.2.2:

| When a non-ASCII registered name represents an internationalized
| domain name intended for resolution via the DNS, the name must be
| transformed to the IDNA encoding [RFC3490] prior to name lookup.
| URI producers should provide these registered names in the IDNA
| encoding, rather than a percent-encoding, if they wish to maximize
| interoperability with legacy URI resolvers.

That is an ordinary MUST, and obviously the tool or party knowing
how it works cannot delegate "punycoding" to another tool or party
which might not know how it works.

-Frank

---

From: John C Klensin <[email-masked]>
Date: Sun, 27 Nov 2011 11:04:33 -0500

--On Sunday, November 27, 2011 02:11 +0100 Frank Ellermann
<[email-masked]> wrote:

>...
>>>   Note that discussion on whether internationalized domain
>>> names   should be percent-encoded or puny-coded, is
>>> ongoing; see<xref   target="I-D.duerst-iri-bis" />.
> 
> All those "<reg-name> outside of DNS" considerations in STD 66
> are IMO irrelevant for UTF8SMTPBIS addresses.

Largely concur, especially if the IRI efforts to retroactively
revise the URI spec are relevant.   From my personal point of
view, there are a relatively small number of possibilities here:

(1) RFC3986/STD66 is the controlling spec.  If so, it is
irrelevant to this WG: email addresses are not inherently URIs
and the WG's decision that encoded forms in other than UTF-8
--especially encoded forms that interact with the %-routing
convention-- makes RFC 3986's notion of
internationalization-by-%-encoding inappropriate.

(2) The IRI WG has determined that RFC 3986 has outlived its
usefulness (STD or not) and that IRIs are the new URIs -- a
direct-use (protocol element) form, not user-friendly
presentation of URIs.  Email addresses may _still_ be irrelevant
to IRIs and vice versa, but the other question then becomes
whether the right answer for URIng involves further tuning (or,
depending on one's point of view, piling onto) the IRI spec or
justifies some rethinking.  

If an email address is transmitted via an IRI form (e.g., in an
extended  MAILTO URL), the question of when the string stops
being an IRI (or IRI encoded in a URI) and starts being an email
address becomes pretty important.

>...
  If that's
> correct we end up with the last two sentences in RFC 3986
> 3.2.2:
> 
> | When a non-ASCII registered name represents an
> internationalized | domain name intended for resolution via
> the DNS, the name must be | transformed to the IDNA encoding
> [RFC3490] prior to name lookup. | URI producers should provide
> these registered names in the IDNA | encoding, rather than a
> percent-encoding, if they wish to maximize | interoperability
> with legacy URI resolvers.

> That is an ordinary MUST, and obviously the tool or party
> knowing how it works cannot delegate "punycoding" to another
> tool or party which might not know how it works.

Right.  However, the discussion in the recent IRI WG f2f about
the RFC 6055 implications of that statement take us back to the
question of whether 3986 is still relevant and, more important,
into a possible dilemma: Suppose one "tool or party" knows how
the IDNA encoding works but doesn't know whether the string will
be interpreted in a public DNS environment or so other
environment (one that does not use IDNA encoding).  So it can't
apply the IDNA encoding (especially the IDNA2003 one, which may
lose information) without risking considerable damage if the
string is interpreted and looked up in an non-public-DNS/IDNA
context.  Conversely, if the Unicode string is (as nearly as
possible) preserved until lookup time, the "tool or party" doing
the lookup might, given your implied hypothesis above, not know
how to apply the IDNA encoding.  RFC 6055 effectively argues for
preserving the Unicode form as long as possible, but that may
not always be the right answer (or "as long as possible" may
need careful interpretation) and, more important, %-encoded
UTF-8 is not "the Unicode form", especially since the email
sender or intermediary cannot know what the delivery server does
about normalization (once again, the "no one interprets an email
address (especially a local part) other than the delivery
server" rule comes into effect... and somewhat contradicts all
of the permitted and required transformations in the current and
proposed IRI specs).

For the purposes of this WG, I think we need to concentrate on
email addresses, not IRIs or URIs.  If mailto doesn't work in
the headers, let's figure out what, from an email point of view,
works and is needed and then work backwards to mailto (or
MAILTobis) and whatever _we_ need from URIbis or IRIbis, rather
than taking those specs as given or restrictive.

Just my personal opinion, of course.

    john

---

From: "Martin J. Dürst" <[email-masked]>
Date: Mon, 28 Nov 2011 16:10:41 +0900

> revise the URI spec are relevant.

The IRI WG currently doesn't have such a mandate. The IRI WG charter 
clearly says (see http://datatracker.ietf.org/wg/iri/charter/):

Changes to RFC 3986 ("Uniform Resource Identifier (URI):
Generic Syntax") are explicitly out of scope of this charter,
and may only be considered with a charter update.

> From my personal point of

Of course email addresses are not URIs. Domain names are not URIs. Path 
names or file names are not URIs. Query parameters in some requests are 
not URIs. Almost none of the parts of an URI, or the things from which 
URIs are constructed, are by itself URIs.

> and the WG's

Which WG?

> decision that encoded forms in other than UTF-8

There is no "internationalization by %-encoding". %-encoding is an 
escaping convention for URIs. With some care, it can be used for 
escaping characters, rather than octets.

Also, it seems that some people are proposing to use %-encoding for mail 
addresses (outside URIs/IRIs, that is). That would definitely be a very 
bad idea. Please stop.

> presentation of URIs.

It's not a question of which is better, RFC 3986 (URIs) or RFC 3987 
(IRIs). There are some places where only URIs are allowed, e.g. in a 
HTTP GET header. There are some places where (modulo some nasty 
details), IRIs are allowed (HTML, XML,...). Both URIs and IRIs in these 
cases are protocol elements.

> Email addresses may _still_ be irrelevant
> whether the right answer for URIng

By this, you mean putting an email address into an URI, yes?

> involves further tuning (or,

Of course. The same question was and is important when a traditional 
email address is used in a plain old URI mailto:.

And in most cases, it should be pretty easy to answer this question. For 
example, <a href='mailto:[email-masked]'> is obviously an 
URI/IRI. When somebody activates that link, there is some machinery 
involved where the browser calls a designated MUA. Where exactly the 
"mailto" is removed, and where exactly the %-escaping is undone may 
depend on the OS in question. For other situations, this will work in a 
similar way.

>> That is an ordinary MUST,

To Frank: I don't see any MUST here, sorry.

>> and obviously the tool or party
> need careful interpretation)

So we have an "it depends". That's exactly what RFC 3986 already says. 
And on top of that, I think RFC 3987 somewhere says that you should keep 
an IRI as long as possible. And that's what RFC 6055 says. And in both 
cases, "as long as possible" may need careful interpretation.

> and, more important, %-encoded

It doesn't contradict the %-encoding at all. %-encoding is just like the 
escaping that may be needed if you put a string into a string constant 
in a programming language.

It is possible to construct some contradiction between the fact that the 
IRI spec says to use NFC and NFKC (in various different degrees of 
strength in RFC 3987 and in the current draft) and the "never change a 
local part" rule. However, that's not a problem between IRIs and EAI, 
but a problem within EAI itself: How am I supposed to write down an 
email address on a napkin or a side of a bus and key it in later without 
at least some recommendation re. normalization?

If there are other transformations in the IRI spec that you alluded to, 
please tell me which ones.

> email addresses, not IRIs or URIs.

Yes, please (except where email headers use URIs/IRIs directly, such as 
in many of the List- headers).

> If mailto doesn't work in
> the headers,

In those headers where it's already used (e.g. List-Unsubscribe:), it's 
already working. If it doesn't work for EAI addresses, then that's 
because EAI is new. Make sure that implementers know they have to do a 
bit of work in this area, and it will start work, the same way as the 
rest of EAI will start to work.

In those headers where URIs/IRIs are not used (e.g. From or To), 
definitely don't try to use a mailto:. That would indeed be a bad idea.

> let's figure out what, from an email point of view,

I very strongly suspect that if you see some huge problems between EAI 
and mailto:, then you have one of the following problems:
a) You are not distinguishing between mail addresses and URIs/IRIs 
containing mail addresses carefully enough.
b) You are asking URIs/IRIs to do something impossible.

Regards,    Martin.