Subject: Re: [EAI] REVIEW: draft-ietf-eai-smtpext-08.txt

From: John C Klensin <[email-masked]>
Date: Fri, 14 Sep 2007 13:07:57 -0400

--On Friday, 14 September, 2007 12:23 +0200 Harald Alvestrand
<[email-masked]> wrote:

> SM wrote:
>> consideration.
>> 
>> In Section 2.5.  ALT-ADDRESS parameter usage and response
>> codes:
>> 
>>   "When messages are rejected because they require UTF8SMTP,
>>   the response code "550" is used, defined in [RFC2821],
>>    meaning "mailbox unavailable".  If enhanced mail system
>>    status codes [RFC3463] are"
>> 
>> Shouldn't that be a "553", i.e. mailbox name not allowed?
>> 
>> If the SMTP server does not advertize the UTF8SMTP extension,
>> I don't  see how we are getting to the stage where we get
>> that response code as  we cannot use the ALT-ADDRESS
>> parameter in the MAIL FROM or RCPT TO  unless we ignore the
>> absence of UTF8SMTP  support.
> oops - I misunderstood.
> 
> you can get into this situation if the SMTP server advertises
> the UTF8SMTP extension, but knows that it may have to forward
> the message to some other system that does not support
> UTF8SMTP - for instance, it may have a local IMAP server that
> doesn't support UTF8SMTP, but have a local POP server that
> does - so for a POP recipient, UTF8SMTP-requiring mails would
> be OK, but for an IMAP recipient, it would have to reject them.
> 
> Other situations are surely imaginable.

Harald,

It feels to me as if this may be another section of the
hypothetical "recommendations to implementers and for
configuration" doc, which I hope someone is planning to write.
Consider that, in the general case, the SMTP server cannot have
any idea how the user will pick up the mail since a given user
may use POP on even-numbered days, IMAP on odd ones, and
something else entirely in any month in which there are five
Fridays.   The 821/2821 requirement that a server either deliver
the mail or not, i.e., that it not deliver and report an error
further constrains the situation.

So, in the example you give, if the POP server is to be
supported, the SMTP server would need to return a positive
response in all cases.  I think that would leave it in a major
tangle if it did not support some sort of post-SMTP-delivery
downgrading, but that isn't a topic for the smptext document.

The situation you describe could arise if the SMTP server had
UTF8SMTP capability but the mail store to which delivery was to
be made for that particular user would not accept 8bit delivery
and no post-delivery downgrading was available.  But, while I
can imagine a few edge cases involving delivery SMTP servers
using different mail stores (with different capabilities) for
different users, it would, in general, be fairly bizarre to
configure things that way.

    john

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: Yangwoo Ko <[email-masked]>
Date: Sat, 15 Sep 2007 10:42:34 +0900

> downgrading, but that isn't a topic for the smptext document.

Post SMTP delivery downgrade is described as "MAY" in the framework 
document. Do we need to change it to "SHOULD" or stronger?

Anyway, I have mixed feelings about this. In general, final SMTP server 
should not be sensitive to configurations of IMAP/POP and others, 
because SMTP is, as far as I understand, a hop-by-hop protocol and it 
should not try to convey semantics beyond the current hop. On the other 
hand, as an end user, it will be very helpful to see where and how a 
mail delivery fails in end-to-end or sending MUA <--> receiving 
POP/IMAP/or others) manner.

By the way, do we have an end-to-end mail capability checking protocol?

---

From: John C Klensin <[email-masked]>
Date: Fri, 14 Sep 2007 23:02:57 -0400

--On Saturday, 15 September, 2007 10:42 +0900 Yangwoo Ko
<[email-masked]> wrote:

>...
> Post SMTP delivery downgrade is described as "MAY" in the
> framework document. Do we need to change it to "SHOULD" or
> stronger?

I don't think so.  The reality is that, in most cases (at least
in my experience), delivery MTAs hand things off to mail stores
or their management software and not to POP or IMAP servers
separately.  If the mail store is not 8-bit-header-capable, then
Harald's case isn't going to apply because either everything
will be downgraded on the way to the store or because the MTA
won't offer the UTF8SMTP extension at all.  If the mail store is
8-bit-header-capable, then, presumably, the message gets
delivered into the store without any downgrading.   The
interesting cases arise 

(1) if either the POP or IMAP server (but not both) are
8-bit-header-capable despite working off of a capable message
store.  However, while the situation may not be under the
control of the user or sysadmin, it will generally be easier to
upgrade the relevant remote message access server than to modify
the message store machinery to provide downgrading.  

(2) If, despite having adequate capability in the IMAP or POP
server, some of the clients are not 8-bit-header-capable.  This
is the only case I consider likely, but it is very far removed
from anything that would affect the smtpext document.  Moreover,
the "fix" is as likely speaking very sternly with the user about
offering to receive mail containing 8bit header or address
material when the mail-retrieving agent is not capable of
actually receiving that material or about upgrading the MUA or
agent as it is about downgrading in that case either.

> Anyway, I have mixed feelings about this. In general, final
> SMTP server should not be sensitive to configurations of
> IMAP/POP and others, because SMTP is, as far as I understand,
> a hop-by-hop protocol and it should not try to convey
> semantics beyond the current hop. 

Because this effects only the final delivery MTA, the hop-by-hop
character of SMTP isn't involved.  But I believe you are correct
anyway.

> On the other hand, as an end
> user, it will be very helpful to see where and how a mail
> delivery fails in end-to-end or sending MUA <--> receiving

But, generally, one can't tell even today in the non-UTF8SMTP
cases.   And, when one does find out, it is via an NDN message
of some sort, not an SMTP reply.  These processes and decisions
are just too far downstream to be reflected in SMTP replies.\

> By the way, do we have an end-to-end mail capability checking

Some work on message tracking/ tracing only.

    john

---

From: Harald Alvestrand <[email-masked]>
Date: Sat, 15 Sep 2007 08:55:01 +0200

> configure things that way.
I used a bad example (since most IMAP servers I'm familiar with also 
have POP capability).
A more realistic one would be one where the IMAP server was UTF8SMTP 
compliant, but the mailing list handling software (Mailman in the case 
of my installation, for instance) was not upgraded.

---

From: John C Klensin <[email-masked]>
Date: Sat, 15 Sep 2007 06:53:58 -0400

--On Saturday, 15 September, 2007 08:55 +0200 Harald Alvestrand
<[email-masked]> wrote:

>> UTF8SMTP capability but the mail store to which delivery was
>> to be made for that particular user would not accept 8bit
>> delivery and no post-delivery downgrading was available.
>> But, while I can imagine a few edge cases involving delivery
>> SMTP servers using different mail stores (with different
>> capabilities) for different users, it would, in general, be
>> fairly bizarre to configure things that way.
> I used a bad example (since most IMAP servers I'm familiar
> with also have POP capability).

Yep

> A more realistic one would be one where the IMAP server was
> UTF8SMTP compliant, but the mailing list handling software
> (Mailman in the case of my installation, for instance) was not
> upgraded.

Yes, but the answer is the same in most cases: too far behind
the network-facing side of the delivery MTA to generate
real-time reply codes.

    john

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 19 Sep 2007 20:35:23 +0100

> downgrading, but that isn't a topic for the smptext document.

But surely this is just the distinction between 'Reject' and 'Bounce' that  
we have discussed before. If an MTA returns a 2xx response, then the most  
it is claiming is "I think I know what to do with this message, so I am  
accepting (allowing you to send) it; but the next man in line might think  
otherwise, in which case I promise to send a bounce (DSN) up the Return  
Path".

That can apply whether the "next man" is the next hop to another MTA, or  
whether it is some complex situation in the local  
storage/delivery/forwarding system. So I see no reason why the (supposedly  
final) MTA cannot accept the message, but later bounce it because it  
subsequently transpires that there are the wrong number of Fridays in this  
particular month. Just so long as it does not lose it without trace.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: John C Klensin <[email-masked]>
Date: Fri, 21 Sep 2007 17:31:18 -0400

--On Wednesday, 19 September, 2007 20:35 +0100 Charles Lindsey
<[email-masked]> wrote:

>> Consider that, in the general case, the SMTP server cannot
>> have any idea how the user will pick up the mail since a
>> given user may use POP on even-numbered days, IMAP on odd
>> ones, and something else entirely in any month in which there
>> are five Fridays.   The 821/2821 requirement that a server
>> either deliver the mail or not, i.e., that it not deliver and
>> report an error further constrains the situation.
>> 
> But surely this is just the distinction between 'Reject' and
> 'Bounce' that we have discussed before. If an MTA returns a
> 2xx response, then the most it is claiming is "I think I know
> what to do with this message, so I am accepting (allowing you
> to send) it; but the next man in line might think otherwise,
> in which case I promise to send a bounce (DSN) up the Return
> Path".

Yes.

> That can apply whether the "next man" is the next hop to
> another MTA, or whether it is some complex situation in the
> local storage/delivery/forwarding system. So I see no reason
> why the (supposedly final) MTA cannot accept the message, but
> later bounce it because it subsequently transpires that there
> are the wrong number of Fridays in this particular month. Just
> so long as it does not lose it without trace.

I think we agree, so am no longer sure what the issue is here.
A problem would arise iff someone or something required that the
delivery MTA _reject_ (not bounce/ generate an NDN) message if
it could not deliver an i18n email message all the way to the
POP client.

     john

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 24 Sep 2007 13:03:47 +0100

> it could not deliver an i18n email message all the way to the

Yes.  think my only worry is that, as we all know, a final delivery agent
has all sorts of other agents hiding behind it for storage of the message
and for delivering it to clients via POP3, IMAP, procmail and doubtless
many others. Traditionally, the interfaces between the MTA and these other
agents in invisible from the outside, and the IETF steers clear of trying
to specify them.

Nevertheless, it is often these 'hidden' agents that will actually
initiate the bounces/DSNs when they are unable to cope with the message as
received (e.g. wrong number of Fridays in the month). OTOH, RFC 3461 (and
hence our own DSN draft) is written on the assumption that these DSNs etc
will be sent by an MTA, and any status codes etc giving the reason for the
failure will be based on the return codes from RFC 2821 or from RFC 3463,
which may not cover some of the more obscure reasons that may apply.

However, this is not really our problem (except insofar as downgrading
disabilities may provide a fresh crop of such obscure reasons). It is a
proplem for email in general which we should probably steer clear of.

---

From: "Frank Ellermann" <[email-masked]>
Date: Tue, 25 Sep 2007 01:10:34 +0200

John C Klensin wrote:
 
> Charles Lindsey <[email-masked]> wrote:

>> If an MTA returns a 2xx response, then the most it is 
>> claiming is "I think I know what to do with this message,
>> so I am accepting (allowing you to send) it; but the
>> next man in line might think otherwise, in which case
>> I promise to send a bounce (DSN) up the Return Path".
 
> Yes.

No :-)  The DSN has to go to the "originator as indicated
in the Return Path", therefore the MTA has to be confident
that the Return Path in fact indicates the originator.

Otherwise it better rejects "dubious" mails instead of
risking to violate this RFC 2821 requirement, where
"dubious" includes any instance of "final delivery is
less than almost certain".  

 Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 25 Sep 2007 12:37:04 +0100

> that the Return Path in fact indicates the originator.

So far as an MTA is concerned, the 'originator' is what was in the SMTP  
MAIL FROM. That later gets copied into a Return-Path header as part of the  
final delivery process.

So when, on this list, we speak of a "Return-Path", we mean either the  
MAIL FROM address or whatever is in the Return-Path header according to  
the stage of processing we are talking about. But editors of drafts need  
to be careful to state the requirements more precisely, of course.

So I think what I informally wrote, and John agreed to, is correct.

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 27 Sep 2007 13:54:56 +0200

> less than almost certain".  
Out of scope for this WG to change mail's behaviour in that way, Frank.

And you knew that already...

---

From: "Frank Ellermann" <[email-masked]>
Date: Thu, 27 Sep 2007 22:54:25 +0200

>> that the Return Path in fact indicates the originator.
[...]   
> Out of scope for this WG to change mail's behaviour in that

Just citing one of the most important statements in RFC 2821
and its logical consequences, when Charles and John discussed
that receivers are free to assume that Return-Paths are okay.

> And you knew that already...

...and John knows that I managed to read the right hand side
of the first non-empty line in RFC 2821, there should be no
problem with a joke here.  For the less funny sides of it 
see <http://permalink.gmane.org/gmane.ietf.smtp/6211> on the
SMTP list.

 Frank