Subject: [EAI] WG LAST CALL: draft-ietf-eai-framework-02

From: Harald Alvestrand <[email-masked]>
Date: Mon, 16 Oct 2006 07:37:23 -0700

Greetings,

the WG chairs believe that the just released draft-ietf-eai-framework-02 
is "just about ready to go" - that is, there are no big outstanding 
issues, and the WG is better served declaring this stable and working 
out the details of the other documents.

We are therefore starting a WG Last Call on this document - of 2 weeks, 
which will expire on October 30, just one week prior to the San Diego IETF.

This should allow us to get all open issues on the document on the 
table, and if any of them need face-to-face discussion in order to 
propose a good resolution, we can do that in San Diego.

Practical details:

- If you want to raise an issue, send a message to the mailing list with 
a subject line that starts with

"ISSUE: framework <section number> - short description of issue"
The WG chairs will track open issues by the start message of the thread, 
and may assign numbers to issues eventually.

- If you want to comment on an issue raised, reply to the message. The 
subject line will then be "Re: ISSUE: ....."

- If you want to reply to a message and make a general comment, rather 
than a comment germane to the issue raised, CHANGE THE SUBJECT LINE to 
NOT start with "Re: ISSUE".

- If you want to reply to a message calling attention to a NEW issue, do 
NOT do so.
Post another message raising the new issue, with a new ISSUE subject 
line instead.
THIS IS VERY IMPORTANT. Many times, people have raised issues and seen 
them forgotten because they were buried in the long thread of discussion 
on a different topic. RAISE ANOTHER ISSUE.

We will not attempt to extract currently open issues from the mailing 
list. The people who raised them can most easily check the -02 framework 
document to see if they are addressed or not.

We will attempt to occasionally post a list of currently open issues 
during the Last Call, and

                   Harald, for the chairs

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 17 Oct 2006 03:46:27 +0200

Copied from <http://permalink.gmane.org/gmane.ietf.ima/647> JFTR:

Yangwoo Ko wrote:

> ... it is not possible to tell from an ASCII address that an email
> sender or recipient is an i18mail user or an ASCII user.

> Too redundant?

No, it's clearer and better than only s/i18mail/ASCII/.  Thanks.

Frank

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 17 Oct 2006 04:07:29 +0200

The "document plan" in chapter 3 tries to get away with updating 2822
in the utf8headers I-D.  That's not good enough for some darker corners
of MIME like UTF-8 in comments in Content-* header fields or preambles.

The utf8headers I-D must explain why a new message subtype (different
from rfc822) isn't desirable, why using message/rfc822 with UTF-8
headers won't cause havoc, and how it might affect application/mbox.

The "document plan" should note this - a separate draft for MIME issues
(separated from the utf8headers I-D) would be too confusing.

Frank

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 17 Oct 2006 04:22:39 +0200

s/bounce/reject/g   A "bounce" is commonly understood as sending spam
(unsolicited NDRs) to innocent bystanders as part of a DDoS.

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 17 Oct 2006 04:28:36 +0200

Copied from <http://permalink.gmane.org/gmane.ietf.ima/644> :

/ditto for POP.???/  Should that be /ditto for [I18Nemail-pop]/ ?

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 17 Oct 2006 04:46:20 +0200

A "delivery to the final delivery MTA" is confusing, should that be
a "delivery by the final delivery MTA", or a "forward to the" MDA ?

Maybe replace "or after delivery to the final delivery MTA" by
"or at final delivery by the MDA" (acronym introduced in section 3).

Frank

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 17 Oct 2006 04:54:04 +0200

The complete I-D is about I18N, it doesn't need this redundant section.

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 17 Oct 2006 04:57:46 +0200

As noted in <http://permalink.gmane.org/gmane.ietf.ima/644> :

For chapter 7 I propose to add a new subsection about any "local part
transformation" schemes like SES, SRS, BATV, or SPF per-user-policies
(or other uses of its %{l} macro as domain label, %{l} = "local part")

Frank

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 17 Oct 2006 05:19:00 +0200

IMO a pointer to IDN (especially 4690) is good enough, there's nothing
special to say about confusable characters in local parts.  

The reference to [Net-Unicode] should be deleted, that I-D is rather
unclear at the moment, apparently it talks about telnet and not SMTP,
discussing the merits of various ASCII control characters like TAB.

"impact any systems or mechanisms that is dependent" - should that be
"are dependent" or "depend" ?

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 17 Oct 2006 05:29:33 +0200

An old one in <http://permalink.gmane.org/gmane.ietf.ima/644>

~~~cut ~~~
s/[DKIM-Charter]/[draft-ietf-dkim-base] + [draft-ietf-dkim-overview]/

The DKIM-threats, SSP-requirements, and SSP-proper texts are not
directly affected (unless SSP tries "per user policies", unclear).
~~~ end ~~~

I've changed my mind, [DKIM Charter] is okay, the details are not
relevant.  But please replace [RFC2663] by [RFC2633] or better by
[RFC3851] ;-)

---

From: "Kari Hurtta" <[email-masked]>
Date: 17 Oct 2006 07:13:23 +0300

> (unsolicited NDRs) to innocent bystanders as part of a DDoS.

It was

    2.  An SMTP relay must

    <...>

      *  Bounce the message so that the sender can make another plan.

      If the message cannot be forwarded because the next-hop system
      cannot accept the extension and insufficient information is
      available to reliably downgrade it, it MUST be bounced.

On generally smtp level reject is not possible on here.   SMTP
level reject is possible only when message is not yet accepted.

And a SMTP connection to next-hop system happens after message 
is accepted. Therefore bounce (NDR) is option, reject is not 
option.

SMTP is store and forward protocol.

/ Kari Hurtta

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 17 Oct 2006 07:51:42 +0200

Kari Hurtta wrote:
 
> SMTP level reject is possible only when message is not yet accepted.

Yes, you can't accept mail unless you know how to report errors later,
or you're confident that there won't be any issues with the delivery.

> a SMTP connection to next-hop system happens after message is accepted

Then you're obviously sure that you can report errors to the originator,
e.g. because you have an SPF PASS on the Return-Path, or you're an MSA
enforcing submission rights, and similar cases.  You cannot accept the
message if you don't know the originator.

> SMTP is store and forward protocol.

And return, that's why it has reverse paths, and stuff like 551 for any
mildly dubious cases.  It can't work if you accept a mail from:<> and
decide to bounce it later (just an example).  The environment is about
80% spam plus about 10% misdirected bounces, EAI has no business to make
that worse.

Frank

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 17 Oct 2006 01:02:54 -0700

> (unsolicited NDRs) to innocent bystanders as part of a DDoS.
while we sometimes bow to "common" usage, I think we should refer to 
existing RFCs for our terminology in this field.

RFC 2821 uses "bounce" a few places, but does not formally define it - 
it seems to be used as a synonym for the (SMTP-level) "reject"; it 
refers to a "notification message" for the DSN notification (section 3.7 
of 2821).

Consistent use of "reject" when SMTP-level reject is intended seems like 
a good idea to me.

                Harald

---

From: Yangwoo Ko <[email-masked]>
Date: Wed, 18 Oct 2006 17:33:29 +0900

> "or at final delivery by the MDA" (acronym introduced in section 3).

In this context, "delivery" is read, at least by me, as the
transmission of data using SMTP. Then, the final delivery
MTA do *NOT* deliver messages any more. It just does something
else such as saving to storage that is not defined in SMTP.

If "delivery" includes "something else", we need to define
what are included in delivery and what not.

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 18 Oct 2006 12:08:25 +0200

> what are included in delivery and what not.

I read it like you said.  But I thought that "deliver" actually is
"something else", whatever MDAs do, excluding forward / reject.
Apparently RFC 4496 uses it this way:

| In the destination email server, a mail delivery agent (MDA) may
| deliver the email to the recipient's mailbox.  The email client
| program of the recipient might use a different protocol (such as the
| Post Office Protocol version 3 (POP3) or IMAP) to access the mailbox
| and retrieve/read the messages.

RFC 2821 apparently uses "deliver" depending on the context, sometimes
it's "whatever happens between MX and message store", and sometimes it
is about the final "delivery" MTA (still getting a chance to reject or
forward instead of "deliver").

Maybe it's not important, but this "delivery to final delivery MTA"
confused me, it's about input to an MDA, not about output from an MDA.

Frank

---

From: John C Klensin <[email-masked]>
Date: Wed, 18 Oct 2006 11:36:30 -0400

--On Wednesday, 18 October, 2006 12:08 +0200 Frank Ellermann
<[email-masked]> wrote:

> I read it like you said.  But I thought that "deliver"
> actually is "something else", whatever MDAs do, excluding
> forward / reject. Apparently RFC 4496 uses it this way:
> 
> | In the destination email server, a mail delivery agent (MDA)
> may | deliver the email to the recipient's mailbox.  The email
> client | program of the recipient might use a different
> protocol (such as the | Post Office Protocol version 3 (POP3)
> or IMAP) to access the mailbox | and retrieve/read the
> messages.

Let's be a little careful here.  Informational documents don't
count in this context and 4496 is particularly problematic.  You
need to pull definitions from either local materials or from
2821.

> RFC 2821 apparently uses "deliver" depending on the context,
> sometimes it's "whatever happens between MX and message
> store", and sometimes it is about the final "delivery" MTA
> (still getting a chance to reject or forward instead of
> "deliver").

The intent was to be very consistent: the "final delivery MTA"
is the one that does not relay or gateway further, which
(important for our purposes) is also the host that controls and
interprets local-parts of addresses.  It may still, as you note,
bounce -- the most obvious case is the only in which it
discovers that the address (which, again, only it can interpret)
does not exist.  "Delivery" is what that MTA does (assuming it
does not bounce or otherwise reject the message).

If that is not clear in 2821, clarifying text would be welcomed
as a contribution to my file of things to do if I can ever open
2821bis again without being overwhelmed by people who feel a
need to make basic changes to the protocol and its requirements.
 
> Maybe it's not important, but this "delivery to final delivery
> MTA" confused me, it's about input to an MDA, not about output
> from an MDA.

I think that is the language 2821 uses, which doesn't
necessarily make it clear (see above).  Consider 

                A      X       B
  Message from --->  SMTP     ----> Whatever else
   network         Server/MTA         happens

"Delivery to the final delivery MTA" refers to the SMTP
transaction at A, with X acting as an SMTP server.   B is not
normally an SMTP transaction.  If it is, then X is probably not
the "final delivery MTA".   If there is such a thing as an MDA
(remember, 2821 doesn't use that term), then it is part of
"whatever else happens" and the mechanism for getting things to
it, B, is, as far as the SMTP final delivery operation is
concerned, just part of "whatever else".

Again, if that isn't clear, then the framework doc isn't where
it needs to be fixed.  It needs to be fixed by an update (or at
least errata) to 2821.   Text welcome.

Drawing from your earlier note...

> "or at final delivery by the MDA" (acronym introduced in

No.  That change -- moving the delivery point from the A -> MTA
transaction to the MTA-> B or B-> whatever-else  transactions--
would, IMO, make things worse and cause other complications.
We should adjust that non-definition ("sometimes called" is not
a definition as far as I'm concerned) and "after... successfully
received" is important, and consistent with the comments above,
but it doesn't change things.  Keep in mind that a final
delivery MTA may, in practice, simply store the message
somewhere without additional processing and without invoking
anything that could properly be called another agent.

Proposed substitute text for section 3, bullet 3...

old: 
		or after the mail has successfully been received by a
		final delivery SMTP server (sometimes called an
		"MDA") involve different... Section 5, below. 

new:
		or after the mail has successfully been received by a
		final delivery SMTP server involve different... Section
		5, below.  Processing that occurs after such final
		delivery, primarily that involved with the delivery to a
		mailbox or message store is sometimes called "Message
		Delivery" processing and the conceptual process that
		performs it is sometimes called an "MDA". 

Improvements welcome.

    john

---

From: John C Klensin <[email-masked]>
Date: Wed, 18 Oct 2006 11:52:20 -0400

--On Tuesday, 17 October, 2006 04:07 +0200 Frank Ellermann
<[email-masked]> wrote:

> The "document plan" in chapter 3 tries to get away with
> updating 2822 in the utf8headers I-D.  That's not good enough
> for some darker corners of MIME like UTF-8 in comments in
> Content-* header fields or preambles.

It was not the intent to "get away with" anything, but to
localize _all_ of those issues into the utf8headers document.
Suggestions for better wording would be appreciated.

> The utf8headers I-D must explain why a new message subtype
> (different from rfc822) isn't desirable, why using
> message/rfc822 with UTF-8 headers won't cause havoc, and how
> it might affect application/mbox.

Agreed to all but the last.  Application/mbox is an
Informational RFC; taking on the job of explaining the impact on
non-standard headers makes the job of this WG impossible.  If
you think application/mbox needs to be considered explicitly,
please initiate an effort to get it onto the standards track
(start with a conversation with the author).  Of course, it such
an effort were initiated now, I assume the ADs would refer it to
this WG for comment about its impact on our work :-(.

My impression, incidentally, is that there is growing consensus
that message/rfc822 cannot be used to encapsulate messages with
UTF-* headers.   I hope that the UTF8Headers document will say
that and explain why.  I'm trying to get a separate draft
written (by someone else -- the lucky party is free to identify
himself) to describe/propose a header structure that would be
suitable for such encapsulation, but it is not, at present, in
this WG's charter or critical path.

> The "document plan" should note this - a separate draft for
> MIME issues (separated from the utf8headers I-D) would be too
> confusing.

Interesting.  Certainly we have different documents for 2822 and
for "MIME issues" in the existing, ASCII-header, world.  But I
would be interested in seeing the sort of note that you propose.

     john

---

From: John C Klensin <[email-masked]>
Date: Wed, 18 Oct 2006 12:30:11 -0400

--On Tuesday, 17 October, 2006 01:02 -0700 Harald Alvestrand
<[email-masked]> wrote:

> Frank Ellermann wrote:
>> s/bounce/reject/g   A "bounce" is commonly understood as
>> sending spam (unsolicited NDRs) to innocent bystanders as
>> part of a DDoS.
> while we sometimes bow to "common" usage, I think we should
> refer to existing RFCs for our terminology in this field.
> 
> RFC 2821 uses "bounce" a few places, but does not formally
> define it - it seems to be used as a synonym for the
> (SMTP-level) "reject"; it refers to a "notification message"
> for the DSN notification (section 3.7 of 2821).
> 
> Consistent use of "reject" when SMTP-level reject is intended
> seems like a good idea to me.

The intent of "bounce" in 2821 was to include both an SMTP
transaction-time rejection (which, by the way, includes
rejection after receiving DATA, giving the intermediate
go-ahead, and receiving the closing CRLF.CRLF) _and_ generation
of a notification message.    That is consistent with usage in
place long before 2821 was written, usage that way partially
driven by the fact that, if A, B, and C are SMTP servers, such
that the message travels
  SubmissionMUA -> A -> B -> C
under normal circumstances (see below), if C rejects the message
coming from B (e.g., issues a 5yz response to a MAIL or RCPT
command), B is going to need to generate a notification message
to A: C has no control over whether A, or for that matter the
user of the SubmissionMUA, sees an SMTP rejection or a
notification message.

The use of the term "bounce" for the case Frank describes is a
recent innovation and, while it may be commonly understood in
some communities, it is not the [only] common usage.  Extending
Frank's suggestion slightly to imply that NDNs are obsolete and
should be generally considered spam would change the whole
Internet mail model from "delivery is assumed unless notified"
to "delivery should not be assumed unless explicitly confirmed".
That would be a major step; I don't see the community as being
willing to accept it, at least yet, but, in any event, these
documents are not the right place for that debate, IMO.

Clarifying in "framework" that the 2821 usage (i.e., both
rejections and notifications) is intended would be reasonable.
Text welcome.

In the same thread, Kari wrote...

(quoting from "framework")
>>       If the message cannot be forwarded because the next-hop
>> system  cannot accept the extension and insufficient
>> information is  available to reliably downgrade it, it
>> MUST be bounced.

Note that knowing whether the next-hop system can "accept the
extension" does not require that anyone accept the message.  One
knows that the extension is not acceptable as soon as the
next-hop host responds to EHLO.

> level reject is possible only when message is not yet accepted.

Yes.

> is accepted. Therefore bounce (NDR) is option, reject is not 

Actually, this is the normal case, but is not required.  In
principle, the following model is consistent with SMTP.  Given
servers A, B, and C, the following sequence of events is
possible and permitted:

   A opens connection to B
   A->B: EHLO [...]
   B->A: (response)
   A->B: MAIL FROM:<sender-mailbox@a> [...]
   B->A: 220 ok
   A->B: RCPT TO:<local-mailbox@c>
   B opens connection to C
   B->C: EHLO [...]
   C->B: (response)
   B->C: MAIL FROM:<sender-mailbox@a> [...]
   C->B: 220 ok
   B->C:  RCPT TO:<local-mailbox@c>
   C->B:  550 Not here, please go away and don't come back
   B->A:  550 user not at C
   A->B:  QUIT
   B->C:  QUIT

and so on.  There have actually been proposals that would
require doing things this way.  They haven't gone anywhere for
reasons that should be obvious, but the sequence is permitted by
all of the relevant protocol specifications. 

> SMTP is store and forward protocol.

Yes.  But while overlapping receipt and forwarding is rare, it
is not prohibited.

    john

    john

---

From: John C Klensin <[email-masked]>
Date: Wed, 18 Oct 2006 12:32:30 -0400

> The complete I-D is about I18N, it doesn't need this redundant

This section is an IESG requirement.  Take it up with the
rule-makers and nit-pickers.    Perhaps it can be made to
disappear when/if the document goes to the RFC Editor.

    john

---

From: John C Klensin <[email-masked]>
Date: Wed, 18 Oct 2006 12:53:19 -0400

--On Tuesday, 17 October, 2006 05:19 +0200 Frank Ellermann
<[email-masked]> wrote:

> IMO a pointer to IDN (especially 4690) is good enough, there's
> nothing special to say about confusable characters in local
> parts.  

An additional pointer to 4690 has been inserted in the Security
Considerations section.  See below.

> The reference to [Net-Unicode] should be deleted, that I-D is
> rather unclear at the moment, apparently it talks about telnet
> and not SMTP, discussing the merits of various ASCII control
> characters like TAB.

RFC 3552 requires a fairly comprehensive security analysis.
Since there have been particular concerns raised about whether
these extensions are possible without introducing new security
surprises, a fairly extensive analysis seems to be in order.
These references are informative and intended to be present in a
"for additional reading" context, not to force the user to chase
back and forth through a large collection of documents to gain
an understanding of the security issues (a practice that 3552
essentially prohibits, at least as I read it).

There actually are confusable character issues here that are
different from the IDNA ones: IDNA essentially operates in lower
case: even though upper case characters can be mapped to lower
ones, there seems to be general agreement that it is better to
always display characters in lower case since, in general, the
lower-case forms are more often distinguishable.  But email
local parts are case-sensitive, so that particular beneficial
trick doesn't work.

As far as Net-Unicode is concerned, I share your unease about
referencing it.  But SMTP, classically, is defined in terms of a
Telnet transport, so that document is not completely irrelevant.
More important, before the appearance of
draft-klensin-idnabis-issues-00 and
draft-alvestrand-idnabis-bidi-00 this week, Net-Unicode was the
only document in the IETF space that even attempted to discuss
the string handling issues relevant to i18nemail... and the two
or three draft-...-idnabis-... documents are _very_ IDN-specific.

In case it is relevant, Net-Unicode has not been abandoned and
won't be.  I've put it on hold until some of the issues with
Unicode, and particularly NFC (which it requires) are resolved.
And, while it falls belong the level of detail in "framework"
(i.e., any discussion probably belongs in "smtpext") and I have
argued against it before, issues involving transformations by
operating systems or intermediate-level transport that SMTP and
MUA implementations can't control may require use to require, or
make some very strong suggestions about, normalized local-parts
on much the same basis that we make special provisions for
blank-padded lines in messages.

On the other hand, if the Net-Unicode draft is actually unclear,
I'd appreciate pointers to specific text that needs to be
clarified.

> "impact any systems or mechanisms that is dependent" - should
> that be "are dependent" or "depend" ?

Fixed.  Thanks.

    john

---

From: John C Klensin <[email-masked]>
Date: Wed, 18 Oct 2006 12:59:21 -0400

--On Tuesday, 17 October, 2006 05:29 +0200 Frank Ellermann
<[email-masked]> wrote:

> The DKIM-threats, SSP-requirements, and SSP-proper texts are
> not directly affected (unless SSP tries "per user policies",
> unclear). ~~~ end ~~~
> 
> I've changed my mind, [DKIM Charter] is okay, the details are
> not relevant. 

The reason for referencing the charter was to avoid getting
involved in race conditions between the two WGs.  Adds nothing
and makes things hard.  So I'm glad you changed your mind.

> But please replace [RFC2663] by [RFC2633] or
> better by [RFC3851] ;-)

Sorry -- don't know how that got by us, unless we copied a typo
out of a message or I was suffering from late-night stupidity.
Reference corrected to 3851.

thanks,
   john

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 18 Oct 2006 18:08:38 +0100

On Wed, 18 Oct 2006 16:52:20 +0100, John C Klensin <[email-masked]> wrote:

> UTF-* headers.

Well I see no such growing consensus, and if there is, then you can count  
me out of it.

I have always assumed (and have often posted comments implying that  
assumption) that (an extended) message/rfc822 was indeed the right tool to  
encapsulate messages with UTF-* headers, most likely with the restriction  
that any email containing it would have to be designated a UTF8SMTP object  
itself (by inclusion of Header-Type: UTF8SMTP), with a consequent  
obligation to downgrade the contained message/rfc822 (or else bounce).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: John C Klensin <[email-masked]>
Date: Wed, 18 Oct 2006 13:12:22 -0400

--On Tuesday, 17 October, 2006 04:57 +0200 Frank Ellermann
<[email-masked]> wrote:

> For chapter 7 I propose to add a new subsection about any
> "local part transformation" schemes like SES, SRS, BATV, or
> SPF per-user-policies (or other uses of its %{l} macro as
> domain label, %{l} = "local part")

Frank, I don't see any way to do this without opening the same
set of problems as trying to describe, in specific terms, what
to do with non-standard headers.    But I don't know what you
have in mind, so please supply specific draft text so that the
WG can look at it.

   john

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 18 Oct 2006 20:00:58 +0200

John C Klensin wrote:

> You need to pull definitions from either local materials or
> from 2821.

We could try this one:

| When the receiver-SMTP makes the "final delivery" of a
| message it inserts at the beginning of the mail data a
| return path line.  The return path line preserves the
| information in the <reverse-path> from the MAIL command.
| Here, final delivery means the message leaves the SMTP
| world.  Normally, this would mean it has been delivered to
| the destination user, but in some cases it may be further
| processed and transmitted by another mail system.

> If that is not clear in 2821

It's in parts verbatim the same as in a paragraph on page 50
of that PS, starting with "when the delivery-SMTP server"...
and later "final delivery means the message has left the SMTP
environment.  Normally, this would mean it had been" etc.

> if I can ever open 2821bis again without being overwhelmed
> by people who feel a need to make basic changes to the
> protocol and its requirements.

The original protocol design and its requirements are fine -
no decent system could accept any mail for forwarding without
a plan how to return possible error reports later.  

Do you want some text clarifying that fundamental point, maybe
some MUSTard that "as indicated by the reverse-Path" actually
means "at this point you're almost sure or almost dead", with
blow by blow quotes from 821 ?

> transaction at A, with X acting as an SMTP server.

Yes, but it's IMO unusual to say "delivery" for this last hop,
if the word "delivery" is often used for "X decides to try B",
i.e. note the reverse-path in a Return-Path and call mailx or
whatever else happens.

> If there is such a thing as an MDA (remember, 2821 doesn't
> use that term), then it is part of "whatever else happens"

I'd say the X in your figure is the MDA.  And also an MX, it
is talking with unknown strangers ("from network").  That's
too simple, all kinds of ugly problems can be avoided if MX
and MDA are the same MTA, we need one hop more to make that
obvious:
              smtp      smtp       not smtp
Message from -----> MX -----> MDA ---------> Message store
network (MTA)      (MTA)     (MTA)           (or whatever)

The rightmost MTA is the final delivery MTA, and that beast
is known as MDA.  With issues like "reject unknown address",
forcing the MX to send an error report if it already accepted
this mail.  Or "decide that downgrade won't work and reject",
with the same outcome.

> if that isn't clear, then the framework doc isn't where it
> needs to be fixed.  It needs to be fixed by an update (or
> at least errata) to 2821

Actually I think it's clear in the text, but I arrive at a
slightly different picture, more like the RFC 4496 figure 2.

> a final delivery MTA may, in practice, simply store the 
> message somewhere without additional processing

Yes, I see MDA as synonym / shorthand for final delivery MTA,
apparently you think it's some later agent.

>     performs it is sometimes called an "MDA".

I think the old text is correct, not this version.  We can just
delete the "MDA" acronym, using your text up to "below", without
the "Processing that" etc.  If anything goes wrong after the
final delivery it's too late for UTF8SMTP and a job for RFC 3834.

Frank

---

From: Tony Finch <[email-masked]>
Date: Wed, 18 Oct 2006 19:08:01 +0100

On Wed, 18 Oct 2006, John C Klensin wrote:
> interprets local-parts of addresses.

Other preceding relays may also control and interpret local parts, e.g.
an incoming border gateway MTA that knows which users are on different
internal email servers.

Otherwise I agree with your message.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
FORTIES CROMARTY FORTH: SOUTHERLY 6 TO GALE 8, DECREASING 5 OR 6 LATER. RAIN
OR SHOWERS. MODERATE OR GOOD.

---

From: John C Klensin <[email-masked]>
Date: Wed, 18 Oct 2006 14:57:28 -0400

--On Wednesday, 18 October, 2006 18:08 +0100 Charles Lindsey
<[email-masked]> wrote:

>> My impression, incidentally, is that there is growing
>> consensus that message/rfc822 cannot be used to encapsulate
>> messages with UTF-* headers.
> 
> Well I see no such growing consensus, and if there is, then
> you can countme out of it.
> 
> I have always assumed (and have often posted comments implying
> thatassumption) that (an extended) message/rfc822 was indeed
> the right tool toencapsulate messages with UTF-* headers, most
> likely with the restrictionthat any email containing it would
> have to be designated a UTF8SMTP objectitself (by inclusion of
> Header-Type: UTF8SMTP), with a consequentobligation to
> downgrade the contained message/rfc822 (or else bounce).

The content, and in particular the headers, of a message/rfc822
body part is expected to be strictly conformant to RFC 822 (not
even 2822).  That is a strong ASCII constraint.   The way I read
that spec, one can't even have a body part inside of one with
"content-transfer-encoding: 8bit", regardless of what the
transport might transmit. 

If I correctly understand what you are suggesting above, you
would expect a legacy MUA or other application that can handle
message/rfc822, but that has never heard of the "Header-Type:"
header, to scan through the entire header set looking for it
(remember that header ordering is, in general, not guaranteed)
and then to take some action on that basis.    We have generally
considered the idea of having to scan all of the headers to find
one that is necessary to the interpretation of headers that
might come earlier -- forcing a multi-pass procedure over the
headers-- as a non-starter.  MIME-Version does not violate that
rule only because it is used to verify that Content-type and
Content-transfer-encoding are correctly understood.  Header-type
is closer to the line and, if used as I think you are suggesting
above, is over it: the parser in a legacy MUA might have done
very nasty things to UTF-8 addresses and other things long
before encountering that header (which it wouldn't understand)
telling it to apply a 7bit interpretation.

Of course, if a message has already been downgraded to the point
that it is entirely ASCII and 822-conformant, then
message/rfc822 is a reasonable way to handle or represent it.
But that isn't the case that most people are thinking about when
they talk about encapsulation.

There are also some issues with the nested encoding rule...
discussed below.

In the hope of being less abstract, let's walk through the one
case for which it is absolutely clear that encapsulation is
needed (there is still debate about the downgrade case, although
I think we are getting closer).  

Suppose I'm on a fully internationalized email system -- the
MTAs and MUAs, and everything in between support the SMTP
extension, UTF-8 headers, 8BITMIME, charset=utf8 in text/plain,
and so on.  I receive a UTF8SMTP message that contains forward
and backward-pointing UTF8 addresses, some non-address UTF-8
headers, and (to make the problem a little easier but still
clear), one body part which has Content-type: text/plain
charset=utf8 and C-T-E: base64.

Suppose I want to forward that message to a colleague whom I
know is on a legacy system (no UTF-8 headers or addresses
accepted or understood) but that does support, say,
charset="iso-8859-1".  Assume I know that from out of band
information sources, such as having prior messages bounced.  And
suppose I want to forward the message with a few comments of my
own... in German.

In the classical case (no UTF-8 addresses or headers anywhere),
I would forward the message by using a structure of:

     From: ascii-mailbox1
     To:  ascii-mailbox2
     ...
     Content-type: multipart/mixed; boundary="--xyz"

     --xyz
     Content-type: text/plain charset=iso-8859-1

     My note to her, in German and 8859-1

     --xyz
     Content-type: message/rfc822

     <the entire original message, unchanged and 
       including headers and the 
	    Base64 encoded UTF-8 body part.  This is what 
       "encapsulation" is all about.>
    --xyz--

But, in the example given, I've got UTF-8 header stuff in the
original message.   The outside headers and first body part have
to be the same as the example above, because the receiving
legacy system can't handle anything else.  So now I come to the
second body part.  I have, I think, the following options:

(1) Associate a Content-transfer-encoding with the
"Content-type: message/rfc822" and put the entire original
message, headers and all, into Base64.   Whoops, its internal
body part was already in Base64, so I've broken the nested
encoding rule.

(2) I downgrade the headers of the forwarded message, doing
whatever the downgrade drafts says and making up new addresses
so that the thing is all-ASCII and 822-conformant.  Whoops, the
Alt-Address is probably gone by now, there is no envelope in
which it might be found, and, because this is a MUA-level
message forwarding case, there is no equivalent of rejection or
sending a non-delivery notification... unless you are going to
tell me that such messages cannot be forwarded.  If it is
downgraded, the odds are that the original cannot be exactly
reconstructed, so, whatever this is, it isn't encapsulation.

(3) I construe message/rfc822 as "really" being 8bit clean and
permitted UTF8 headers, adding a "Header-type" field to
reinforce that.   Whoops, I can't transmit this message at all
without 8BITMIME (reducing the problem to Case 2) or, if the
otherwise-legacy server can handle 8BITMIME and my submission
server tries to handle it that way, the header parser in the
receiving MUA (or, worse, the receiving IMAP server or other
header canonicalization agent) discards all of the high-order
bits in the headers of that message/rfc822 object, ignoring
Header-Type because it has never heard of it.

If you have another analysis, I'm very interested in hearing it,
because I'd be really happy if we could use message/rfc822.  But
I believe, based on the analysis above and several other efforts
to work through this, that "message/rfc822" means "ASCII,
822-conformant, headers and message" and that we get ourselves
into big trouble by trying to bend that or encourage sender
systems to lie to recipient systems about it.

Remember that one gets away with some transformations in netnews
by assuming that the entire transport and reader environment if
8bit clean or better.  We have explicitly not made that
assumption with SMTP/MIME-based email and, given the behavior of
a number of popular and widely deployed MTAs and MUAs, that
assumption would simply not be sustainable.

     john

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 19 Oct 2006 05:09:26 +0200

John C Klensin wrote:
 
> of a notification message.

Yes, and that vagueness is not more state of the art, "accept
and maybe bounce later" is considered as net abuse unless that
bounce goes to the originator.  On the other hand "reject" is
a simple decision, it's then the problem of the client how to
report an error to the originator.  

If an MTA isn't sure about "downgrade", with a possible result
of "no idea", it can't accept the mail unless it's almost sure
that it can report this issue to the originator.  If the route
forward is clear all is fine, if the route backward is clear
it's also okay, if both are clear it's ideal, and if both are
unclear "accept" isn't possible.  That leaves "reject" as a
neutral statement in 4.1 covering all cases.

> the message travels
> to A

IOW B is supposed to know how to inform A if C rejects the mail.

> C has no control over whether A, or for that matter the user of
> the SubmissionMUA, sees an SMTP rejection or a notification 
> message.

That's B's problem, it accepted the mail and the responsibility,
C can't get into trouble if it rejects it.  

> Extending Frank's suggestion slightly to imply that NDNs are 
> obsolete and should be generally considered spam

That would be an odd "extension", there are complete protocols
helping B out with its decision "accept" or "reject", with an
outcome of PASS meaning "yes, I can bounce this message later,
therefore I can accept it and see what C decides".

NDNs are not obsolete, they're essential.  But of course they
must go to the originator.  How B manages that, e.g. by using
"enforced submission rights" if it's an MSA, isn't C's problem.

> these documents are not the right place for that debate, IMO.

ACK, let's just use unambiguous terms like "reject", that's 
what we're interested in wrt C.  B's problems are not caused
by EAI.

Frank

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 19 Oct 2006 06:04:09 +0200

John C Klensin wrote:
 
> Application/mbox is an Informational RFC; taking on the job of
> explaining the impact on non-standard headers makes the job of
> this WG impossible.  If you think application/mbox needs to be
> considered explicitly, please initiate an effort to get it onto
> the standards track

I'm not exactly interested to improve this spec., it could be as
simple as registering a new format=i18mail, or maybe not.  

For the framework I-D it's a known case where existing applications
might not work as expected, if that's documented it's okay.

I'm mildly interested because I know one mail service provider
offering a Webmail interface, where users can download two or more
mails at once, encapsulated as x-download/rfc822 mbox (they picked
the MIME type before application/mbox existed, and I can't tell if
they renamed it later).

Maybe some mailing lists use application/mbox for their digests -
all we have to do is to say that there might be a problem, we're
not forced to investigate or solve it.  Maybe in a new 7.6 section:

| 7.6 application/mbox
| Some applications use formats similar to the application/mbox
| format defined in [RFC 4155] instead of message/digest to store
| messages.  As far as such applications assume that all stored
| messages use the message/rfc822 format with US-ASCII headers
| they're not ready for [I18Nemail-UTF8].

Or something in that direction, an "additional issue", reserving
7.5 for wild and wonderful (ab)uses of local parts not addressed
in 7.3.

Frank

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 19 Oct 2006 07:25:24 +0200

> RFC 3552 requires a fairly comprehensive security analysis.

It starts to worry me if the "real" specific issues are hidden
within tons of general security considerations, some of them
"well-known".  As an example, the UTF-8 issues are fascinating,
RFC 3629 explains all details, it's a STD, and implementors 
are supposed to read it if there's a normative reference.

> the string handling issues relevant to i18nemail

I've seen (and read, once) the idna-bidi draft:
http://tools.ietf.org/html/draft-alvestrand-idna-bidi

But I don't find an idna-issues I-D, is that still somewhere in
the pipeline ?

> if the Net-Unicode draft is actually unclear, I'd appreciate
> pointers to specific text that needs to be clarified.

Some questions in:
| Message-ID: <[email-masked]>
| Date: Sat, 29 Jul 2006 08:33:10 +0200
| To: John C Klensin <[email-masked]>
| Subject: Re: First comments on draft-klensin-net-utf8-oo.txt
| References: <[email-masked]>
| Content-Length: 3742

A shorter form could be "what's the idea of a SHOULD NOT for
BEL, BS, HT, VT, and FF" ?  It mentions RFC 854, and there I
found commands starting at 0xF0, I've no clue how that could
work with UTF-8 (needing 0xF0..0xF4 for its own purposes).

In section 4 you have the same paragraph "The NFC tables"...
twice.  In 5.2 there's a "vfersion", and the claim that "UTF-8
is tied to Unicode Version 4" makes no sense for me.

Frank
-- 
<http://purl.net/xyzzy/home/test/utf-8.xml> don't try s/8/4/ ;-)

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 19 Oct 2006 07:44:21 +0200

> This section is an IESG requirement.  Take it up with
> the rule-makers and nit-pickers.

Harald would tell us if we violate this rule without a
good reason, he has invented it, it is a SHOULD, not a
MUST.  In a document with I18N in its title following
that rule literally with a dummy section is pointless -
AFAIK the nit-picking tools don't try to enforce this.

Frank

---

From: Tony Finch <[email-masked]>
Date: Thu, 19 Oct 2006 14:27:06 +0100

> neutral statement in 4.1 covering all cases.

Are you proposing that a server must keep a cache of all destination host
capability lists, so that it can reject non-downgradable messages as early
as possible? I think that is too large a change to existing practise to
make it part of the EAI work.

Tony.

---

From: John C Klensin <[email-masked]>
Date: Thu, 19 Oct 2006 09:47:51 -0400

--On Thursday, 19 October, 2006 14:27 +0100 Tony Finch
<[email-masked]> wrote:

> Are you proposing that a server must keep a cache of all
> destination host capability lists, so that it can reject
> non-downgradable messages as early as possible? I think that
> is too large a change to existing practise to make it part of
> the EAI work.

I think it is more than that.  That ideal case requires that it
know that both the forward  and reverse addresses are both
authentic and deliverable, and that it know those things in
advance of receiving a message between the sending system and
the receiving one.  That would require some pre-seeding at best.
Remembering that there are transient conditions in the mail
system -- "full mailbox" being the most common-- it gets even
more difficult.

Note that even a real-time rejection at the first hop doesn't
_guarantee_ that the report will get through to the originator:
the submission MTA might have validated a source but not an
address, the address could have been one-time-use (stupid, but
possible), and so on.  One could sensibly argue that those cases
are marginal or even irrelevant -- the responsibility has been
successfully transferred and that is all the rejecting server
cares about -- and would probably be right, but it emphasizes
how fragile a notion of "almost sure it can deliver or report"
becomes.

As I have tried to suggest, it is perfectly possible to design
an email system so that anything that is not deliverable as
expected (or within some reasonable delta of "as expected") is
simply dropped and the only way to determine whether a message
got through involves some sort of end-to-end ACK.  Some would
claim that X.400 worked that way; some proprietary systems in
the past certainly did.  One can also try to shift the
preference from "accept, then interpret and process, then send
NDN if needed" (which has been the trend and preference for
years) to "reject as early as possible, ideally as part of the
SMTP transaction".  I've certainly got my servers to reject,
e.g., bad forward-pointing addresses, as early as possible and
that was not the case a decade ago.  But it seems to me that one
can't turn it into a requirement, especially given the
implications of a generic MX backup or relay, which knows the
hosts for which it is holding and forwarding mail but not the
mailbox list or syntax on that host, getting a rejection from
the target host.  In that case, there is almost no choice but to
generate an NDN (or to discard the message, which I guess Frank
prefers).

More generally and more important, the Internet mail system is
not designed that way.  It is designed around the assumption
that a mail message, once sent, is delivered unless a negative
message (not distinguishing between an NDN and a real-time
rejection) comes back.   In principle, one could change that,
but it clearly isn't within scope for this WG (or for any other
effort designed to codify existing, mainstream, practices).

     john

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 19 Oct 2006 15:16:16 +0100

> even 2822).  That is a strong ASCII constraint.

But it is always in order to define extensions to existing standards,  
provided you put in place transitional arrangements so that existing  
implementations do not break (indeed, we are currently in the process of  
extending 2822, at the same time trying to ensure that existing  
implementations do not see the new stuff).

>   The way I read

That may well be true, but not if the message/rfc822 itself (not the thing  
inside it) says CTE 8bit. The real question is whether RFC 1652 tells you  
to change the CTE of the contained email when you hit a non-8BITMIME  
server (as clearly needs to be done). RFC 1652 seems somewhat reticent to  
tell you that :-( .
> and then to take some action on that basis.

I would expect that situation to be no different than when a legacy MUA is  
suddenly confronted with a UTF8SMTP message with UTF-8 addresses in its  
headers. In both cases, I believe, the answer is that the legacy MUA  
should never have been allowed to see the message in the first place, and  
if it did it implies that some prior MTA had advertised the UTF8SMTP  
capability and then failed in its obligation to ensure that the message  
was subsequently handled accordingly.

> they talk about encapsulation.

Please note that I am in no way suggesting message/rfc822 (or even  
message/utf8smtp) as a means of encapsulating a message for subsequent  
restoration to its original pristine state.

> accepted or understood)...

OK. You are John. You received the message from Ålice and you want to  
forward it to Bob.

Essentially, you cannot expect to convey any more to Bob than he would  
have seen if Ålice had sent it direct to Bob (e.g. he might possibly have  
some diffficulty in sending a reply to Ålice - he might be able to send it  
to an alternate Alice, though).

> legacy system can't handle anything else.

No, the headers right at the top need to include
        Header-Type: UTF8SMTP
because the message is going to contain something that is not allowed in  
legacy emails.

> ...  So now I come to the

> so that the thing is all-ASCII and 822-conformant.

Yes. More or less. Your MUA (or possibly your local MSA) enquires whether  
Bob's local MTA does UTF8SMTP (it doesn't), so it either tells you "Sorry!  
I cannot send a message with that atachment" (a bounce), or it sets about  
downgrading it. It spots that the attached message/rfc822 needs to be  
downgraded, at which point it does exactly what Ålice's MUA would have  
done if she had tried to send it directly to Bob.

Let us consider Ålice's From header. If that provided an <alt-address>,  
then it becomes the only remaining address.

>  Whoops, the
> message forwarding case, ...

Ah! You want to use that bit in 5.1.1 of the downgrade document which says  
to use the envelope-from in that case. I always thought that was a bit  
iffy, and said so whenb I commented on that draft on Sept 4th. Indeed, it  
has never been clear to me how the originating MUA was able to provide an  
<alt-address> in the envelope-from if it did not find/put one in the  
header From: at the same time.

Once upon a time, we had an ATOMIC parameter (more likely "DOWNGRADABLE")  
which allowed you to apply an algorithmic transformation. But discussion  
on that suddenly stopped and it disappeared from the various drafts,  
apparently without any consensus to do so.

So the best thing to do is to omit the From header entirely (which it is  
in fact legal to do in a message/rfc822).

But all is not yet lost, because the downgrade process will also have  
created

     Downgraded: From: <RFC 2047 representation of Ålice's original From  
header>

and (unless his MUA is being excessively obstinate, or has not the fonts  
to display it), Bob should be able to see that (though whether it gives  
him enough to reply to Ålice is less certain).

> there is no equivalent of rejection or
> tell me that such messages cannot be forwarded.

Yes. You yourself have often said that, even if downgrading gets too  
complicated for implementors to bother with, a system which always bounces  
rather than downgrades would still be preferable to the present situation.  
Your best strategy if that happens, and you really want to be sure that  
Bob sees it, is to do some cutting and pasting and send it as a text/plain.

> ...  If it is

And I do not claim it is encapsulation in that sense.

If you DO want encapsulation in that sense then, on further thought, I  
have come to the conlusion that the only way to make it work reliably will  
be to let that double encoding happen (by using an application type, even  
application/octet-stream if all else fails). But I would not like to write  
that into a formal standard if we can avoid it, and I do not think we yet  
need to contemplate that.

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 20 Oct 2006 08:22:12 +0200

Tony Finch wrote:

> Are you proposing that a server must keep a cache of all destination
> host capability lists, so that it can reject non-downgradable messages
> as early as possible?

That's not limited to EAI, it's a general consequence of the "report
errors to the originator" design.

For the purpose of the "framework" I-D we're only concerned with this:

| When MUA/MTA is transferring mail and finding its envelope contains
| Non-ASCII addresses, it MUST decide to bounce or downgrade if
| receiving MTA is EAI incompliant.

If it's an MX it's (today) expected to know which addresses are valid
or not, and to reject mails to invalid addresses.  Ideally it would
also know which in theory valid mailboxes are over quota, trying hard
to accept only mails where it can do what it must after it accepted
them, deliver or report an error to the originator.

"Cannot downgrade for address X" (otherwise a valid ASCII address X)
is "simply" an additional piece of information it needs for its job,
otherwise almost the same situation "X is no mailbox here" or "X is
over quota".

In all these "undeliverable" situations it better rejects mails to X.
If it has a good idea how to report errors found later it can accept
the mail without knowing the state of X.  There must be a way forward
or a way backward, ideally both.

Otherwise the MX could be in an unholdable position forced to violate
STD 10 in some way or another - "frivolous", as the PS written by a
"dude or not" "(while popping or otherwise)" puts it. <g>

> I think that is too large a change to existing practise to make it
> part of the EAI work.

It's no change at all, it's just the word "reject", because "bounce"
is a mine-field on bright days.

John C Klensin wrote:

> there are transient conditions in the mail system -- "full mailbox"
> being the most common-- it gets even more difficult.

Yes, I've given up to discuss this on the SpamCop list with folks who
claim that receivers can always solve this without any help from the
senders.  Maybe - if most obvious cases, where that works without the
help of the senders, are eliminated by upgrading or blocking abusive
receivers - it's again possible to discuss the more interesting cases.

> I've certainly got my servers to reject, e.g., bad forward-pointing
> addresses, as early as possible and that was not the case a decade
> ago.

The flood of forged Return-Paths started about 5 years ago as far as
I can judge it - I recall a mail-abuse-FAQ written about 2002, where
it was still discussed as "social problem", not as "protocol design
flaw" (introduced between 821 and 2821)

> In that case, there is almost no choice but to generate an NDN (or
> to discard the message, which I guess Frank prefers).

No, unless my mail provider manages to make its service unusable  - I
just deleted 27,000 spams received in less than 15 hours, and that took
me 6 hours - the MX should get a PASS clearly indicating "please send
NDNs as needed" or a FAIL clearly indicating "please reject".

Of course it can decide it by other means, block lists, crystal balls,
DKIM signatures, or whatever it takes.

Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 20 Oct 2006 09:40:13 +0100

> Ah! You want to use that bit in 5.1.1 of the downgrade document which  
> says to use the envelope-from in that case. I always thought that was a  
> bit iffy, and said so whenb I commented on that draft on Sept 4th.  
> Indeed, it has never been clear to me how the originating MUA was able  
> to provide an <alt-address> in the envelope-from if it did not find/put  
> one in the header From: at the same time.

But, on further thoughts, that envelope-from of Ålice's original message  
IS available if you want to transfer it into the From header during the  
downgrade. You will find it in the Return-Path of her message.

---

From: Tony Finch <[email-masked]>
Date: Fri, 20 Oct 2006 11:01:54 +0100

> In all these "undeliverable" situations it better rejects mails to X.

Agreed, but many SMTP servers are not MXs. You can't expect an outgoing
relay to know which destination addresses can cope with EAI messages at
SMTP time, so you can't require them to reject instead of bounce.

Tony.

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 20 Oct 2006 13:57:29 +0200

> Agreed, but many SMTP servers are not MXs.

If we agree on the MX case we're ready, anything else is simple.  BTW,
I tried to find your 2004 article where you informed me that RFC 821
can't beat RFC 1123 5.3.6(a) because RFC 1123 is also a part of a STD,
but I haven't found it.  

> You can't expect an outgoing relay to know which destination addresses
> can cope with EAI messages at SMTP time, so you can't require them to
> reject instead of bounce.

Yes, but that mailout probably is an MSA, or sends on behalf of an MSA,
it knows how to report errors to the originator - by mail if necessary,
there's no problem with bounces sent to the originator.  The complete
sending side (or MON, Mail Originating Network, in Keith's terminology)
should have no problems.  Ignoring open relays because they're hopeless.

An MX talking with unknown strangers is the critical hop:  It shouldn't
accept UTF8SMTP if a later MTA on the receiving side (MRN) could decide
that it cannot downgrade.  If s/bounce/reject/ doesn't express this in
4.1, then maybe we need an extra-bullet for such "border MTAs" in 4.1 ?

Frank

---

From: Kari Hurtta <[email-masked]>
Date: 21 Oct 2006 21:36:48 +0300

> Yes.  But while overlapping receipt and forwarding is rare, it

Well,  on one point there is problem. With final '.' you need
store mail, if there is more than one recipient for different @domain
part.   B can not act as proxy in that case   (assuming that
it does not tempfail some recipients so that all recipients left
point to same MX)
 
LMTP avoids that problem.

So SMTP  is a store and forward protocol on general case.

/ Kari Hurtta

---

From: John C Klensin <[email-masked]>
Date: Sat, 21 Oct 2006 15:49:04 -0400

--On Saturday, 21 October, 2006 21:36 +0300 Kari Hurtta
<[email-masked]> wrote:

>> reasons that should be obvious, but the sequence is permitted
>> by all of the relevant protocol specifications. 
>> 
> store mail, if there is more than one recipient for different
> @domain part.

Nope.  B can, in principle, have connections open to each
destination domain, can do the needed verification in parallel,
etc.  The odds of A getting a clear and unambiguous SMTP-time
rejection if one of the recipient hosts accepts the message
after DATA and the message body and others do not are not high,
but they are not high any time a failure reply code is returned
after the body data are transmitted with a multiple-recipient
message, whether the recipients are in the same or different
domains.  That is at least one reason why, in practice, NDNs,
and not failure reply codes, are almost always returned if a
problem is encountered after the 354 code is returned in
response to DATA.

I grant this is getting increasingly unlikely.

>   B can not act as proxy in that case
> (assuming that it does not tempfail some recipients so that
> all recipients left point to same MX)
>  

Sure.  So does anything whose batching model is strong enough
that NDNs are just about the only way to report most errors.
But, if I correctly understand his position, Frank would like to
ban the lot of those things as encouraging spammers and/or DoS
attacks, at least barring strongly authenticated senders.
 
> So SMTP  is a store and forward protocol on general case.

In the general, and common, cases, no question about it.

     john

---

From: Kari Hurtta <[email-masked]>
Date: 22 Oct 2006 21:58:43 +0300

"Charles Lindsey" <[email-masked]> writes in gmane.ietf.ima:

> So the best thing to do is to omit the From header entirely (which it
> is  in fact legal to do in a message/rfc822).
> 
>      Downgraded: From: <RFC 2047 representation of Ålice's original
> From  header>
> 
> and (unless his MUA is being excessively obstinate, or has not the
> fonts  to display it), Bob should be able to see that (though whether
> it gives  him enough to reply to Ålice is less certain).

Yes. Obstinate is perhaps correct word :-)

I do not decode RFC2047 encoding on headers, which I do not know syntax
(syntax "*text" is assumed only for X- headers. )

Most MUAs probaly do not show randow headers (if not something like
"All headers" toggle is pressed.)

/ Kari Hurtta

( yes, I checked 'obstinate' from dictionary. )

---

From: Kari Hurtta <[email-masked]>
Date: 22 Oct 2006 22:09:55 +0300

"Charles Lindsey" <[email-masked]> writes in gmane.ietf.ima:

> > Ah! You want to use that bit in 5.1.1 of the downgrade document
> > which  says to use the envelope-from in that case. I always thought
> > that was a  bit iffy, and said so whenb I commented on that draft on
> > Sept 4th.  Indeed, it has never been clear to me how the originating
> > MUA was able  to provide an <alt-address> in the envelope-from if it
> > did not find/put  one in the header From: at the same time.
> 
> But, on further thoughts, that envelope-from of Ålice's original
> message  IS available if you want to transfer it into the From header
> during the  downgrade. You will find it in the Return-Path of her
> message.

draft-ietf-eai-utf8headers-01.txt:

   angle-addr     =  [CFWS] "<" utf8-addr-spec [alt-address] ">" [CFWS]

   utf8-addr-spec =  utf8-local-part "@" utf8-domain

   utf8-local-part=  utf8-dot-atom / quoted-string / obs-local-part

   utf8-domain    =  utf8-dot-atom / domain-literal / obs-domain

   alt-address    =  "{" [CFWS] addr-spec [CFWS] "}"

RFC 2822:

  return          =       "Return-Path:" path CRLF

  path            =       ([CFWS] "<" ([CFWS] / addr-spec) ">" [CFWS]) /
                           obs-path

>From that reading of syntax Return-Path seems allow only ASCII
addresses.

So yes, downgraded envelope address as found from Return-Path.

(Original UTF8 address from MAIL FROM seems not be available however. )

/ Kari Hurtta

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 23 Oct 2006 11:21:34 +0200

John C Klensin wrote:

> I don't know what you have in mind, so please supply specific
> draft text so that the WG can look at it.

Something like that:

| 7.5 Other uses of local parts
|
| Local parts are sometimes used to construct domain labels, e.g.
| the local part "user" in the address [email-masked] could
| be converted into a vanity host user.domain.example with Web
| space at <http://user.domain.example> and catchall addresses
| [email-masked].
|
| Such schemes are obviously limited by among others the LDH rule,
| and won't work for other local parts like the <utf8-local-part>
| specified in [I-D.ietf-eai-utf8headers].

So far it's obvious, but I've no idea which wild and wonderful
uses of local parts might exist in addition to this simple case.

Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 23 Oct 2006 10:32:39 +0100

>   path            =       ([CFWS] "<" ([CFWS] / addr-spec) ">" [CFWS]) /

I think the intention was that utf8hdrs should permit utf8-addr-spec  
everywhere that addr-spec was allowed, except inside an alt-address,  
obviously. So it looks like that case got forgotten.

Actually, I think what utf8hdrs needs to say about Return-Path is

     return = "Return-Path:" path CRLF
     path = ([CFWS] ( angle-addr / "<" ([CFWS] ">") [CFWS])  
/                                  obs-path

Then, if a compliant MDA still has a MAIL FROM with both primary and alt  
addresses intact, it can put them both into the Return-Path (and the  
smtpext doc should say that).

---

From: Yangwoo Ko <[email-masked]>
Date: Mon, 23 Oct 2006 19:30:58 +0900

> | specified in [I-D.ietf-eai-utf8headers].

For these cases, i18n local parts are parts of IDNs and they
should follow rules defined by IDNA. For example, they can be
in UTF-8 native(???) form when used in IDN-aware domain name
slots.

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 23 Oct 2006 14:10:55 +0200

> in UTF-8 native(???) form when used in IDN-aware domain name

On the bottom (DNS) that's get's us back to Charles proposal of
using punycode.  The idea shouldn't be dismissed without some
considerations:

- "old" users have already ASCII addresses, they'd only wish
  to get an additional new UTF-8-address, without any obscure
  punycode-downgrade variant.

- therefore it's pointless to invent special procedures and
  syntax only to support a wannabe-standard-downgrade variant
  within the EAI framework

- "new" users have no address at all, they start with a fresh
  UTF-8-address, and probably need also a downgrade fallback,
  if they want to communicate with ASCII-users

- for that case (new user) their service provider could use a
  standard way to determine and offer a derived ASCII-address,
  and that could be an IDNA xn-- local part 

Of course they could also B32-encode the UTF-8 local part for
this purpose, or use other ad hoc solutions, but the xn-- way
could be better for e.g. derived vanity host names.

It could also help to determine "allowed" UTF-8 local parts,
if the xn-- version clashes with another user, then the UTF-8
local part might be not "different enough" from this other
user wrt confusable addresses.  That's pure speculation, and
possibly it's plain nonsense, but I'd still like to know why
it's a stupid idea... :-) 

Frank

---

From: John C Klensin <[email-masked]>
Date: Mon, 23 Oct 2006 11:12:20 -0400

>   and that could be an IDNA xn-- local part 

I feel like I'm repeating myself.  

(1) Punycode will not work for email local parts because it is
case-sensitive.  Adam Costello proposed a variant a while back
that would be case-preserving, but it wouldn't be IDNA-punycode
any more.

(2) One could, of course, ban case-sensitive i18n local parts,
but no one has suggested that yet and I can imagine its not
being completely popular.  Popular or not, it would violate the
principle that, to the extent possible, we don't want to have a
different set of restrictions for UTF8SMTP addresses than we
have for ASCII addresses.

(3) If your new users wants to use, e.g., subaddresses, and the
associated delivery SMTP uses early dispatching and late
decoding, the use of almost any encoding system on the wire is a
serious problem.  Vis-a-vis the trick DNS name convention you
describe in your proposed subsection, two of the more common
subaddress delimiter characters -- "+" and "/" -- would be
prohibited since they cannot appear in LDH domain names.  But
hyphens would not be intrinsically prohibited, nor would
subaddress systems that don't use explicit delimiters.

> could be better for e.g. derived vanity host names.

Indeed, if all or part of a local-part is going to be mapped to
or from a domain name label, and your expectation is that
nothing other than mail-retrieval software will ever see the
string in a local-part, then a punycode string would be
reasonable.  But that is a very special case, certainly too
special to justify a general rule although, if a particular
system decides to configure things that way (for use with
Alt-Address if needed), I can't see how anyone could object.

> it's a stupid idea... :-) 

The normalizations performed in IDNA (and nameprep/stringprep)
aren't good enough to provide any significant protection against
confusable characters.  The new 'idnabis' work may be a little
better, but mostly by eliminating characters that IDNA maps to
other through compatibility mappings.   And the present IDNA
excludes some strings that one would almost certainly want in
local-parts.  See draft-klensin-idnabis-issues-00,
draft-alvestrand-idnabis-bidi-00, and
draft-faltstrom-idnabis-tables.

    john

---

From: Philip Guenther <[email-masked]>
Date: Mon, 23 Oct 2006 09:53:41 -0600

...
> transport might transmit.

That disagrees with RFC 2046, which says:
    It should be noted that, despite the use of the numbers "822", a
    "message/rfc822" entity isn't restricted to material in strict
    conformance to RFC822, nor are the semantics of "message/rfc822"
    objects restricted to the semantics defined in RFC822.

So conformance to RFC 822 isn't required.  Furthermore, it then continues:
    <...>
    No encoding other than "7bit", "8bit", or "binary" is permitted for
    the body of a "message/rfc822" entity.  The message header fields are
    always US-ASCII in any case, and data within the body can still be
    encoded, in which case the Content-Transfer-Encoding header field in
    the encapsulated message will reflect this.  Non-US-ASCII text in the
    headers of an encapsulated message can be specified using the
    mechanisms described in RFC 2047.

If 8bit data can't be present in a message/rfc822 part, then why permit 
CTE: 8bit on the part itself?  Given that all other subtypes of "message" 
_do_ have a strict 7bit-only restriction, I can only conclude that a 
message/rfc822 part is permitted to contain 8bit data in its embedded body 
(when it recursively declares itself as being a MIME message with CTE: 
8bit, of course).

Such a message would require 8BITMIME transport, of course, and if that 
was lacking then the message/rfc822 part's contents would have to be 
converted from 8bit to quoted-printable or base64, just like a 
multipart/*.  Isn't that what all the 8BITMIME SMTP clients out there do 
in that case?

...
> they talk about encapsulation.

Are those people referring to encapsulation-with-exact-reconstruction as 
distinct from the downgrade mechanism?  If so, how would you know whether 
to encapsulate or downgrade?  If exact reconstruction is necessary, it 
seems to me that needs to be a capability of the downgrade mechanism and 
not some completely separate thing, otherwise we'll have messages flying 
around in both forms, some which are incomprehensible to old clients, some 
which are imperfect representations.

Philip Guenther

---

From: Tony Finch <[email-masked]>
Date: Mon, 23 Oct 2006 22:03:22 +0100

On Mon, 23 Oct 2006, John C Klensin wrote:
>
> Punycode will not work for email local parts because it is

Do you mean case-INsensitive in the second line above? Case handling is a
nameprep matter, not a punycode matter. There's no reason that EAI
couldn't use punycode with a differenc stringprep profile that preserves
case.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
LUNDY FASTNET: NORTHEAST BACKING SOUTHWEST 6 TO GALE 8 DECREASING 5.
OCCASIONAL RAIN. MODERATE OR GOOD.

---

From: John C Klensin <[email-masked]>
Date: Mon, 23 Oct 2006 17:12:11 -0400

--On Monday, 23 October, 2006 22:03 +0100 Tony Finch
<[email-masked]> wrote:

>> that would be case-preserving, but it wouldn't be
>> IDNA-punycode any more.
> 
> Do you mean case-INsensitive in the second line above? Case
> handling is a nameprep matter, not a punycode matter. There's
> no reason that EAI couldn't use punycode with a differenc
> stringprep profile that preserves case.

That is correct.  But it would almost certainly also require a
different prefix (see earlier note) and might well require a
stringprep profile that was more different from the
IDNA/nameprep one than just preserving case.

   john

---

From: Kari Hurtta <[email-masked]>
Date: 25 Oct 2006 17:20:18 +0300

> this WG's charter or critical path.

lucky party?

You didn't promise to stop whining if I don't write it :-)

-rw-r--r--  1 hurtta tek-tik 78680 2006-10-24 14:40 draft-hurtta-eai-encapsulation-00.txt
-rw-r--r--  1 hurtta tek-tik 88588 2006-10-24 14:40 draft-hurtta-eai-encapsulation-00.xml

If I guess correctly RFC-Editor is open after that meeting, 
whatever it was ?

/ Kari Hurtta

PS. Following is brain teaser:

       Received: from {idn-encoded-name}
           by downgrade.example.org with ESMTP
           id JGR17356;
           Wed, 13 Sep 2006 22:27:25 +0300
       Header-Type: UTF8;
          downgrade=encapsulate
       From: { UTF-8 address }
       To: [email-masked]
       Date: Wed, 13 Sep 2006 22:27:25 +0300
       Subject: { UTF-8 subject }
       X-Foobar: XvrT
       Mime-Version: 1.0
       Content-Type: multipart/signed;
               protocol="application/XYZ-signature";
               micalg="ABC"; boundary=12345
       Content-Transfer-Encoding: 8bit

       --12345
       Content-Description: { UTF-8 description }
       Content-Type: Text/plain; charset=UTF-8
       Content-Transfer-Encoding: 8bit

       { UTF-8 text }

       --12345
       Content-Type: application/XYZ-signature

       { signature data }

       --12345--

Lets assume that Text/plain is still readable after 
encapsulation even when reading MUA do not understand
encapsulation (assuming that MUA supports MIME.)

But perhaps hard part is that either signature should not
found or signature should verify OK (on any point assuming
that signature verify OK originally.) And after
decoding encapsulate signature should verify. 

Hmm. draft-ietf-dkim-base-06.txt was on last call. Perhaps
from it there can get also nice example?  (brain teaser?)

I also do not have example for attaching EAI mail for
non UTF8 environment.

( If  EAI mail is attached to  new EAI/UTF8SMTP message,
  then I think that  message/rfc822  can be used.
  However that draft do not assume that. )

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 27 Oct 2006 21:43:31 +0100

> which will expire on October 30, just one week prior to the San Diego  

Sorry for being so late replying to this.

Most of the points I have raised are textual suggestions or niggles. So  
most of them don't amount to "Issues" (or, if they do, they are already  
under active discussion in the WG).

>            Overview and Framework for Internationalized Email
>                     draft-ietf-eai-framework-02.txt
>Abstract
>   Full use of electronic mail throughout the world requires that people
>    be able to use their own names, written correctly in their own
>    languages and scripts, as mailbox names in email addresses.  This
>    document introduces a series of specifications that define mechanisms
>    and protocol extensions needed to fully support internationalized
>    email addresses.  These changes include an SMTP extension and
>    extension of email header syntax to accommodate UTF-8 data.  The
>1.  Introduction
>   In order to use internationalized email addresses, we need to
>    internationalize both the domain part and the local part of email
>    addresses.  The domain part of email addresses is already
>    internationalized [RFC3490], while the local part is not.  Without
>    these extensions, the mailbox name is restricted to a subset of 7-bit
>    ASCII [RFC2821].  Though MIME enables the transport of non-ASCII
>    data, it does not provide a mechanism for internationalized email
>    address.  RFC 2047 [RFC2047] defines an encoding mechanism for some
             ^
             es
>    specific message header fields to accommodate non-ASCII data.
>    However, it does not address the issue of email addresses that
>    include non-ASCII characters.  Without the extensions defined here,
>    or some equivalent set, the only way to incorporate non-ASCII
                                          ^
                                        at present
>    characters in email addresses is to use RFC2047 coding to embed them
>    in what RFC 2822 [RFC2822] calls the "display name" (known as a "name
>    phrase" or by other terms elsewhere) of the relevant headers.
>    Information coded into the display name is invisible in the message
>    envelope and would not be considered by many to be part of the
>    address at all.

However, I think you need to say more than that. Granted that
internationalizing addresses was the original motivation for this work, we
have gone beyond that now, and in my view the possibility to have utf-8 in  
all
headers where appropriate is at least as important as fixing the address
problem. Moreover, this introductory section is our "shop window" to show  
the
world what we are about (and hopefully to persuade it to buy what it sees).

So I suugest that you mention the current 5 different mechanisms provided  
in
various mail standards for squashing 8bit characters into 7bits, and point  
out
how confusing this makes things for people who manage to see the "on the  
wire"
format which, as you say, "leaks". Indeed, much of what you have written in
4.2 might usefully be moved here.

And the Abstract also needs updating to convey the same emphasis.

> 1.2.  Problem statement
>   and behavior that is culturally friendly.  One thing most of us have
>    almost certainly learned from the experience with email usage is that
>    users strongly prefer email addresses that closely resemble names or
>    initials to those involving meaningless strings of letters or
>    numbers.  If the names or initials of the names in the email address
>    can be expressed in the native languages and writing systems of the
>    users, the Internet will be perceived as more natural, especially by
>    those whose native language is not written in a subset of a Roman-
>    derived script.

You may as well add that the same is true of other natural language phrases
that turn up in syntactically constrained contexts, such as URIs,
<parameter>s, etc.
>   ....  That requirement has far-reaching implications:
>    collections of patches and workarounds are not adequate.  Even if
>    they were adequate, a workaround-based approach may result in an
>    assortment of implementations with different sets of patches and
>    workarounds having been applied with consequent user confusion about
>    what is actually usable and supported....

Make it clear that you are talking about what appears on the wire here, not
just what the user sees displayed.

> 1.3.  Terminology

>    In this document, an address is "all-ASCII", or just an "ASCII
>    address", if every character in the address is in the ASCII character
>    repertoire [ASCII]; an address is "non-ASCII", or "an i18mail
>    address", if any character is not in the ASCII character repertoire.
>    Such addresses may be restricted in other ways, but those
>    restrictions are not relevant here.  The term "all-ASCII" is also
>    applied to other protocol elements when the distinction is important,
>    with "non-ASCII" or "internationalized" as its opposite.

Yes, the concepts there are right, but I do not like the term "i18mail" for
an address that actually contains Non-ASCII (not that I have any better
suggestion). Note that you do not actually use the term with this meaning
elsewhere in this draft (though you do use "i18mail user", which is not the
same thing).

>    A "i18mail user" has one or more non-ASCII email addresses.  Such a
>    user may have ASCII addresses too; if the user has more than one
>    email address, he or she has some method to choose which address to
>    use on outgoing email.  Note that under this definition, it is not
>    possible to tell from the address that an email sender or recipient
>    is an i18mail user.

For that last sentence, So What? And actually, you can tell one way round,  
but
not the other.

> 3.  Document Plan
>   In addition to this document, the following documents make up this
>    specification and provide advice and context for it.
>   o  SMTP extensions.  This document [I18Nemail-SMTPext] provides an
>       SMTP extension for internationalized addresses, as provided for in
>       RFC 2821.

Is not LMTP extended in the same way?
>   o  Email headers in UTF-8.  This document [I18Nemail-UTF8]
>       essentially updates RFC 2822 to permit some information in email
                                     ^
                                  and some other standards
>       headers to be expressed directly by Unicode characters encoded in
>       UTF-8 when the SMTP extension described above is used.
                   ^                                  ^^^
           suitably enabled transports, such as       , are

RFC 2822 is carefully designed to be applicable to any transport protocol,  
not
just SMTP. Our utfhdrs draft should be the same.

>    o  In-transit downgrading from internationalized addressing with the
>       SMTP extension and UTF-8 headers to traditional email formats and
>       characters [I18Nemail-downgrade].  Downgrading either at the point
>       of message origination or after the mail has successfully been
>       received by a final delivery SMTP server (sometimes called an
>       "MDA") involve different constraints and possibilities; see
>       Section 4.3 and Section 5, below.

I think it needs to be made clear somewhere that an MTA may not know  
whether
it is the final MDA until it sees the actual message. For example, it may  
be a
lower priority MX record holder which simply queues the message for later
delivery to the primary MX. Or if may look at the local-part (which at  
last it
is allowed to do) and decide that mail for that particular local-part has  
to
be forwarded on (unchanged bar another Received) to somewhere else.

>    o  Parallel extensions to the POP protocol [I18Nemail-pop].

s/POP/POP3/ throughout (you currently use both forms).

Needs another bullet for mailing lists?

> 4.1.  SMTP Extension for Internationalized eMail Address
>   Some general principles apply to this work.
>   1.  Whatever encoding is used should apply to the whole address and
>        be directly compatible with software used at the user interface.

Not clear what you are getting at there. Certainly not true after  
downgrading.
>   2.  An SMTP relay must
>       *  Either recognize the format explicitly, agreeing to do so via
>           an ESMTP option,

s/ESMTP/UTF8SMTP/

New bullet needed
          * Forward it to some other MTA known to support the format and
            willing to relay,
>       *  Select and use an ASCII-only address, downgrading other
                                                 ^
                             "for RCPT TO" and/or "MAIL FROM"
>           information as needed (see Section 4.3), or

And clearly it may also downgrade a limited set of "easy" cases, and bounce
the rest.

>        available to reliably downgrade it, it MUST be bounced.

Or is is "rejected"? You can fight that one out with Frank.

>   3.  In the interest of interoperability, charsets other than UTF-8
>        are prohibited.  There is no practical way to identify them
>        properly with an extension similar to this without introducing
>        great complexity.

But other charsets can still be used in bodies (and it may even be more
convenient or economical to do so).

> 4.2.  Transmission of Email Header in UTF-8 Encoding

Yes, RFC 2822 defines "header" with that meaning, but the rest of the world
uses that term to mean "header field". So it is safer avoid using the term
"header" if it is possible (as here) to use "header fields" instead.

>    There are many places in MUAs or in user presentation in which email
>    addresses or domain names appear.  Examples include the conventional
>    From, To, or Cc header fields; Message-IDs; In-Reply-To fields that
>    may contain addresses or domain names; and in message bodies.

Arrrrrgghhhhh! No! Message-IDs do NOT contain email addresses (they just  
look
like email addresses). Moreover, we explicitly disallow UFT-8 inside them.

>    ... the user will inevitably, if only occasionally, see them
>    rather than "native" characters and will find that discomfiting or
>    astonishing.  ...
                 ^
             or even totally incomprehensible

>    ....  The only practical way to avoid these sources of
>    discomfort, in both the medium and the longer term, is to have the
>    encodings used in transport be as nearly as possible the same as the
>    encodings used in message headers and message bodies.

Agree 100%. Note that "transport" == "on the wire".
>    ....  This is done by
>    extending the encoding models of [RFC2045] and [RFC2231].

Point out that UTF-8 is for all sorts of headers (as utf8hdrs will  
indicate in
detail), not just addresses (though those have the greatest potential  
impact).
In particular, all <unstructured> headers and all headers beginning with  
"X-"
can use it freely (we should mention X-headers explicitly, but that does  
NOT
include arbitrary "optional" headers as per RFC 2822, because they will
usually be structured extensions defined in other standards).

> 4.3.  Downgrading Mechanism for Backward Compatibility
>   o  Figure out a way to downgrade the envelope or message body in
>       transit.

s/message body/message header fields/ - generally (funny MIME stuff  
excepted)
the message body can look after itself.
>   There is also a third case, one in which the client is I18Nemail-
>    capable, the server is not, but the message does not require the
                                         ^
                                      particular
>    extended capabilities.  ...
>5.2.  Downgrading or Other Processing After Final SMTP Delivery
>   When an email message is received by a final delivery SMTP server, it
>    is usually stored in some form.  Then it is retrieved either by
>    software that reads the stored form directly or by client software
>    via some email retrieval mechanisms such as POP or IMAP.

Point out that it may have to examine the local-part before it can  
determine
how or where it is to be stored.
>   ... it MAY either downgrade internationalized
>    emails or specially identify messages that utilize these extensions,
>    or both.

We have provided the Header-Type header to do that "specially identify".

At the end of this paragraph, add:

      "Alternatively, it MAY bounce the message (even if it had previously
accepted it), {Use of "bounce" and "accept" as per Frank}
>7.1.  Impact on IRIs
>   The mailto: schema defined in [RFC2368] and discussed in IRI
>    [RFC3987] may need to be modified when this work is completed and
>    standardized.

Looks like Martin has this in hand. Be prepared to write something more
substantial in here. As I have said, URIs involving I18N addresses are  
nothing
more than "yet another ACE". And introducing IRIs into the List-* headers  
is
orthogonal to what Martin is doing to Mailto. If we say they are (by
extension) allowed in those headers, then they automatically become  
clickable.
What Martin is doing is more related to IRIs that may turn up in message
bodies.
>7.4.  Encoded-words, signed messages and downgrading
>   One particular characteristic of the email format is its persistency:
>    MUA are expected to handle messages that were originally sent decades
      ^^^
      MUAs
>    ago and not just those delivered seconds ago.  As such, MUAs and mail
>    filtering software will need to continue to accept and decode header
>    fields that use the "encoded word" mechanism [RFC2047] to accommodate
                                                            ^
                                              and the mechanisms of  
[RFC2231]
>    non-ASCII characters in some header fields.

Not sure what "filtering software" you have in mind that might need to  
decode
2047 and 2231 stuff.

>   For example, message parts that are cryptographically signed using,
>    e.g., S/MIME [RFC2663] or PGP [RFC3156], cannot be upgraded from RFC
>    2047 form to normal UTF-8 characters without breaking the signature.
>    Similarly, message parts that are encrypted encrypted) may contain,
>    when decrypted, header fields that use the RFC 2047 encoding; such
>    messages cannot be 'fully' upgraded without access to cryptographic
>    keys.

When mentioning S/MIME and PGP, please make it clear that PGP refers to
the use of multipart/encrypted rather than the more commonly used simple
incorporation of a signed PGP text within a message body (or else make it
clear it refers to both PGP usages).

Why oh why didn't they call it "PGP/MIME" :-( ?

>    Similar issues may arise if signed messages are downgraded in transit
>    [I18Nemail-downgrade] and then an attempt is made to upgrade them to
>    the original form and then verify the signatures.  Even the very
>    subtle changes that may result from algorithms to downgrade and then
>    upgrade again may be sufficient to invalidate the signatures if they
>    impact either the primary or MIME bodypart headers.  When signatures
>    are present, downgrading must be performed with extreme care if at
>    all.

Yes, we need to look at DKIM in more detail.
>13.  References

Please can we have all the references arranged in alphabetical order?

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 27 Oct 2006 23:04:28 +0100

> IDNA/nameprep one than just preserving case.

Are you now accepting that the use of Punycode for downgrding local-parts  
is now back on the agenda, subject to a suitable technical specification  
of exactly how to do it (and possibly in conjuction with an ATOMIC, or  
more likely DOWNGRADABLE, keyword)?

---

From: John C Klensin <[email-masked]>
Date: Fri, 27 Oct 2006 18:41:28 -0400

--On Friday, 27 October, 2006 23:04 +0100 Charles Lindsey
<[email-masked]> wrote:

>>>> case-sensitive.  Adam Costello proposed a variant a while
>>>> back that would be case-preserving, but it wouldn't be
>>>> IDNA-punycode any more.
>>> 
> Are you now accepting that the use of Punycode for downgrding
> local-partsis now back on the agenda, subject to a suitable
> technical specificationof exactly how to do it (and possibly
> in conjuction with an ATOMIC, ormore likely DOWNGRADABLE,
> keyword)?

Only the co-chairs can "accept" anything onto the agenda, I'm
just a participant.  If you are asking my personal opinion, the
answer is "no, I still believe it would be likely to have a
negative impact on the installed base if it appeared in a
downgraded message delivered to a legacy delivery MTA".

    john

---

From: Frank Ellermann <[email-masked]>
Date: Sat, 28 Oct 2006 01:58:53 +0200

> more likely DOWNGRADABLE, keyword)?

When I asked about it what I had in mind was a "standard" way to determine
downgraded addresses if needed.  With a feature to identify classes of"too
similar" local parts.  John said that's more like "too naive" => forget it.

The other aspect was "something better than B32 if used as a domain label",
that might still be interesting.  But what's the point of these keywords ?

The sending side knows its ASCII-address, and then it can use it directly
instead of a "do it yourself" keyword.  Without keyword the complete issue
of "determine an ASCII address" is a local problem of the sending side,
they can use whatever they like (including but not limited to a "standard"
solution), nobody else needs to know how they did it.

Is your keyword proposal somehow related to abuse and forged addresses, or
what's the idea ?  Obviously [ATOMIC] is shorter than [yn-cruft@example],
but it needs far more places knowing how to "do it yourself".

Frank

---

From: John C Klensin <[email-masked]>
Date: Sun, 29 Oct 2006 17:20:14 -0500

Hi.

I'm trying to go through "framework" and do a final pre-IETF
review and I'm still confused about what to do about the general
issue associated with this thread.  As I understand it, we seem
to have two points of view represented in the WG:

(1) If an MTA is not absolutely, 100%, sure that either

	1.1. It will be able to deliver a message, all the way
	to the destination mailbox, in UTF8SMTP form, or if any
	intermediary that needs to do so will be able to
	downgrade correctly, i.e., there is no possibility of a
	subsequent NDN.
	
	1.2  It has a certain, authentic, and deliverable
	reverse-path address

then it MUST reject the message, where "reject" implies a 5yz
code in the SMTP transaction flow, presumably before the DATA
command is acknowledged with a 354 code.

One of the consequences of this approach is that intermediate MX
hosts are expected to know which addresses on the destination
host are valid or not, without asking that host at message
arrival time.  When the MX intermediary is a boundary (e.g.,
firewall or LAN gateway) host, that may be a reasonable
expectation.  When it is a standby backup against the
possibility of a service outage at or near the target host, it
may be a very difficult one.  

(2) UTF8SMTP messages are handled as has been traditional with
Internet mail, i.e., they are forwarded if possible with the
understanding that, if they reach a point at which downgrading
would be required, they may be rejected in a way that will lead
to an NDN message.

The purpose of this note is not to restart the discussion unless
someone has something really new to say.  I just want to be sure
I have correctly stated the two points of view and, if not or if
there are other points of view, get people to tell me/ us about
them. 

    john

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 30 Oct 2006 09:50:47 -0000

> 	1.2  It has a certain, authentic, and deliverable

I would understand here that the Return-Path "<>" is considered  
"deliverable", in the sense that every MTA knows the route to the  
bit-bucket.

> would be required, they may be rejected in a way that will lead

But I am not clear what the precise difference is from (1). What is an  
"understanding" other than what you get from (1)? I assume (2) is meant to  
be more liberal than (1), but it is not clear how. I agree that (1) seems  
a little draconian, but if senders cannot ensure that a decent Return-Path  
is made available, they have only themselves to blame if NDNs get lost.

---

From: John C Klensin <[email-masked]>
Date: Mon, 30 Oct 2006 11:54:52 -0500

--On Monday, 30 October, 2006 09:50 +0000 Charles Lindsey
<[email-masked]> wrote:

> I would understand here that the Return-Path "<>" is
> considered"deliverable", in the sense that every MTA knows the
> route to thebit-bucket.

Yes, of course.
 
>> would be required, they may be rejected in a way that will
>> lead to an NDN message.
> 
> But I am not clear what the precise difference is from (1).
> What is an"understanding" other than what you get from (1)? I
> assume (2) is meant tobe more liberal than (1), but it is not
> clear how. I agree that (1) seemsa little draconian, but if
> senders cannot ensure that a decent Return-Pathis made
> available, they have only themselves to blame if NDNs get lost.

Two differences.  The first is that Frank is worried about faked
reverse paths being used to either spread spam or mount DoS
attacks.  So a key part of (1) is "authentic", not merely
deliverable.   Second, if one is concerned about the use of
bounce messages as attacks of one sort or another, "getting
lost" is not the problem.   (2) is basically our existing model:
if SMTP-time rejection is not possible or not convenient, NDNs
get sent out if the address appears valid, i.e., if there is a
valid DNS entry for the host listed in the reverse path and an
SMTP connection can be established.  If those conditions are not
met (or if the reverse path is "<>"), then the NDN is dropped.
And, since the NDN contains a <> reverse path, if the NDN
message cannot be delivered at the destination host, it is
dropped there.

    john

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 30 Oct 2006 20:03:33 +0000

>4.1.  SMTP Extension for Internationalized eMail Address
 [...]
 >       be directly compatible with software used at the user interface.

Why? Can it be applied to LHS and RHS separately?

 >7.4.  Encoded-words, signed messages and downgrading
[...]
 >   Similarly, message parts that are encrypted encrypted) may contain,

Repeated word "encrypted"

Regards,
Alexey

---

From: John C Klensin <[email-masked]>
Date: Mon, 30 Oct 2006 15:21:39 -0500

> Why? Can it be applied to LHS and RHS separately?

This was an attempt to explain, in short form, why it would not
be reasonable to, e.g., require that the local part be in UTF-8
while the domain part was in punycode.   If you can think of
better wording, or think it needs to be made more explicit,
please send text.

> Repeated word "encrypted"

fixed.
thanks and regards,
   john

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 30 Oct 2006 20:36:10 +0000

> Is not LMTP extended in the same way?

Good point.

 [...]

> convenient or economical to do so).

You are right, but I think John was only talking about message headers here.

> Arrrrrgghhhhh! No! Message-IDs do NOT contain email addresses (they 
> just  look
> like email addresses).

Yes, I've stumbled on this text as well.

I think Message-ID/In-Reply-To are examples of the "or domain names" part.

> Moreover, we explicitly disallow UFT-8 inside them. 

 [...]

> Not sure what "filtering software" you have in mind that might need 
> to  decode
> 2047 and 2231 stuff.

E.g. Sieve mail filtering, which can run inside MTAs or MDAs.

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 30 Oct 2006 21:36:25 +0000

John C Klensin wrote:

>I have correctly stated the two points of view and,
>
Your description looks correct to me.
I just want to add "+1" for the (2) choice. It would be very nice to 
have (1), but I just don't think it is very practical for many existing 
deployments.

>if not or if there are other points of view, get people to tell me/ us about
>them. 
>  
>

---

From: John C Klensin <[email-masked]>
Date: Mon, 30 Oct 2006 17:18:33 -0500

--On Monday, 30 October, 2006 20:36 +0000 Alexey Melnikov
<[email-masked]> wrote:

> Charles Lindsey wrote:
> 
>>> 3.  Document Plan
>>>   In addition to this document, the following documents make
>>>   up this specification and provide advice and context for
>>>    it. o  SMTP extensions.  This document
>>>   [I18Nemail-SMTPext] provides an SMTP extension for
>>>       internationalized addresses, as provided for in RFC
>>>       2821.
>> 

Not in the charter.  In other words, one thing at a time.  If we
can get this right for SMTP and then move toward standards-track
specs, certainly an LMTP upgrade should be part of that effort.

>  [...]
> 
>>>   3.  In the interest of interoperability, charsets other
>>>   than UTF-8 are prohibited.  There is no practical way to
>>>        identify them properly with an extension similar to
>>>        this without introducing great complexity.
>> 
>> But other charsets can still be used in bodies (and it may
>> even be more convenient or economical to do so).
> 

Yes.  But I will clarify the text.

>> "header" if it is possible (as here) to use "header fields"

Done

>>>    There are many places in MUAs or in user presentation in
>>>    which email addresses or domain names appear.  Examples
>>>    include the conventional From, To, or Cc header fields;
>>>    Message-IDs; In-Reply-To fields that may contain
>>>    addresses or domain names; and in message bodies.
>> 
>> Arrrrrgghhhhh! No! Message-IDs do NOT contain email addresses
>> (they  just  look
>> like email addresses).
> 

Yes.  But I have restructured the sentence to clarify this and
noted that Message-IDs may be a special case.

>>>   One particular characteristic of the email format is its
>>>   persistency: MUA are expected to handle messages that were
>>>    originally sent decades
>> 
>>      ^^^
>>      MUAs

fixed

>>>    ago and not just those delivered seconds ago.  As such,
>>>    MUAs and mail filtering software will need to continue to
>>>    accept and decode header fields that use the "encoded
>>>    word" mechanism [RFC2047] to accommodate
>> 
>>                                                            ^
>>                                              and the
>>                                              mechanisms of  
>> [RFC2231]
>> 
>> Not sure what "filtering software" you have in mind that
>> might need  to  decode
>> 2047 and 2231 stuff.
> 

Yes, that is what we had in mind along with some non-standard
techniques.  Do you think Sieve should be explicitly referenced
as an example and, if so, should the reference be to 3028 or to
something else?

     john

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 31 Oct 2006 02:44:58 +0100

John C Klensin wrote:

 (1)
> When it is a standby backup against the possibility of a service
> outage at or near the target host, it may be a very difficult one.

Especially at times where it's needed, when all "call forward" or
similar approaches cannot work.  But that's a "normal" situation
for the backup MX (= no special considerations wrt EAI), it should
tempfail / reject very radically, independent of the address syntax.

Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 31 Oct 2006 10:56:07 -0000

On Mon, 30 Oct 2006 16:54:52 -0000, John C Klensin <[email-masked]> wrote:

> lost" is not the problem.

Ah! I see it now. But how can some intermediate MTA tell whether an  
address is 'authentic' or that it is being used to commit some DOS? And  
aren't these attacks just as possible on non-UTF8SMTP messages, in which  
case it is not our job to fix them? The only difference in the case of  
UTF8SMTP is that such messages may create more bounces than regular  
messages usually do.

So I think the most we can do is draw attention to these problems under  
Security Considerations, in which case count me as +1 for (2).

---

From: John C Klensin <[email-masked]>
Date: Tue, 31 Oct 2006 07:16:31 -0500

--On Tuesday, 31 October, 2006 10:56 +0000 Charles Lindsey
<[email-masked]> wrote:

>> Two differences.  The first is that Frank is worried about
>> faked reverse paths being used to either spread spam or mount
>> DoS attacks.  So a key part of (1) is "authentic", not merely
> Ah! I see it now. But how can some intermediate MTA tell
> whether anaddress is 'authentic' or that it is being used to
> commit some DOS?

Well, that is the problem.  It is clearly possible if all
intermediate MTAs are clearly under the administrative control
of the sender and have access to their user/address tables (for
backward-pointing addresses) and all target MTAs are clearly
under the administrative control of the target receiver host and
have access to its user/address tables (for forward-pointing
addresses).  There are several "in between" cases that I
consider important; I gather Frank is willing to essentially
prohibit them in order to fend off spammers.

But I may not have this right; see next note.

> Andaren't these attacks just as possible on
> non-UTF8SMTP messages, in whichcase it is not our job to fix
> them? The only difference in the case ofUTF8SMTP is that such
> messages may create more bounces than regularmessages usually
> do.

I believe that is the case.  The fact that the same argument is
coming up in the discussion of Dave Crocker's "email model"
document on the SMTP list may support that conclusion.

> So I think the most we can do is draw attention to these
> problems underSecurity Considerations, in which case count me
> as +1 for (2).

Ack.

    john

---

From: John C Klensin <[email-masked]>
Date: Tue, 31 Oct 2006 07:29:10 -0500

--On Tuesday, 31 October, 2006 02:44 +0100 Frank Ellermann
<[email-masked]> wrote:

> John C Klensin wrote:
> 
>  (1)
>> When it is a standby backup against the possibility of a
>> service outage at or near the target host, it may be a very
>> difficult one.
> 
> Especially at times where it's needed, when all "call forward"
> or similar approaches cannot work.  But that's a "normal"
> situation for the backup MX (= no special considerations wrt
> EAI), it should tempfail / reject very radically, independent
> of the address syntax.

Frank,

I don't understand what you are saying above.  Perhaps a word is
missing; perhaps I'm just confused.

What I often refer to as a "backup MX" is a mail server that
doesn't fate-share (with regard to either location or
connectivity) with the server on which the target mailbox is
located.  It generally doesn't receive mail unless the target
server either goes off-net or offline.   When the target server
is off-net or offline, the backup MX receives all traffic bound
for the target server and stores it for future delivery --
either until the target server is available again or until the
messages can be transmitted in some other fashion (there is,
e.g., a long and important history of transferring mail queues
to magnetic storage and transmitting them to target servers via
automobile or airplane when the target server has suffered an
extended network outage).  

Historically, such servers often exist as the result of
reciprocal agreements ("you provide backup for my mail servers
and I'll provide backup for yours"), sometimes even between
entities that normally treat each other as major competitors but
nonetheless trust each other's integrity.   And, in the normal
case, the arrangements are rarely used -- a few sequential days
once a year would probably be considered a lot.  

For those cases, keeping up-to-date user (or valid mailbox)
lists on the backup MX host would be burdensome and probably
even be considered a serious breach of privacy of business
information.

But, as I understand it, your proposal would prohibit those
kinds of arrangements entirely, since the backup MX host would
not have knowledge of the authenticity of sending addresses
(systems generally don't) and would have no way to know whether
a target mailbox address was valid.  It wouldn't have up-to-date
tables of valid mailboxes for the reasons cited above, and it
couldn't do real-time forward checks (VRFY?) because, by
definition, the target destination host would be unreachable.   

I don't think that conclusion, and prohibition, is tolerable, at
least without an email regime in which only trusted hosts are
permitted to send mail (and global trust is somehow
established), all email is authenticated wrt validity of sender
and address, and all relays were required to verify the
authentication and credentials.  Such a regime would have many
advantages, IMO, but we are nowhere near getting there.  YMMD.

And, as Charles has pointed out, all of these issues exist
without any i18n extensions, so they are not really the province
of this WG.

       john

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 31 Oct 2006 22:59:42 +0100

> AFAIK the nit-picking tools don't try to enforce this.
FWIW, I've argued against a mandatory "internationalization 
requirements" section at every turn, arguing instead that the issues 
need to be discussed when we need to, in the form that's best suited for 
them.

And I still can't see any requirement for it in 
<http://www.ietf.org/ID-Checklist.html>, so I guess someone's been 
listening.

                      Harald

---

From: John C Klensin <[email-masked]>
Date: Tue, 31 Oct 2006 19:17:15 -0500

--On Tuesday, 31 October, 2006 22:59 +0100 Harald Alvestrand
<[email-masked]> wrote:

> requirements" section at every turn, arguing instead that the
> issues need to be discussed when we need to, in the form
> that's best suited for them.
> 
> <http://www.ietf.org/ID-Checklist.html>, so I guess someone's
> been listening.

I will take that as direction to remove it from the working
draft :-)

     john