Subject: Re: Sieve reject at SMTP time possible with which implementations?

From: "Guðbjörn S. Hreinsson" <[email-masked]>
Date: Sat, 1 Nov 2003 09:47:52 -0000

> >>The problem with filtering before storing in the end users mailbox is as Ned
> >>says, the concept of the mail envelope.  At the SMTP level, the envelope
> >>probably has many recipients, 
> >>
> s/probably/very occasionally/, so it's not as much of an issue.  I think 
> a 'refuse' that works perfectly as long as there aren't multiple 
> *envelope* recipients (which, keep in mind, is much rarer than multiple 
> recipients of any kind) ( and handles the special case by causing an MDN 
> to be sent on behalf of the rejecting recipients) is much better than no 
> 'refuse' at all.

Having multiple RCPT TO's is quite common. What you define as much 
better is false-positive which for me would be much worse. I would rather 
have a filter miss than a false hit. 

> "Sieve evaluation as typically implemented requires having the entire 
> message on hand.
> In SMTP this means the only response code that's left is the final one, 
> where the server accepts or rejects the message on behalf of all 
> recipients. This becomes a problem when there are multiple recipients 
> and some reject the message and some do not.  In this case, reject MUST 
> cause an MDN to be sent on behalf of the rejecting recipients. (Idea: 
> allow MTA to tempfail the rejecting recipients and accept the others, 
> until all recipients are rejecting the email; then it can be refused.)"
> [I now reject that latter idea:] Is the additional complexity introduced 
> by specifying how the tempfail would work to make SMTP rejects work even 
> in the relatively rare case of multiple recipients on the envelope 
> really worth it?  Making this mandatory would make the implementation of 
> this spec considerably more work, I think.   I don't want to make it 
> optional, I'd rather leave it out than make it optional.  (By the way, 
> tempfail means to send a 4xx SMTP response code)  It would work as 
> follows:  when there are multiple recipients and some reject the message 
> and some do not, a tempfail response code is sent.  The recipient server 
> has to maintain state about the message so it can recognize it when (if) 
> delivery is reattempted. (some spamware never reattempts delivery!)  
> When delivery is reattempted, the rejecting recipients are tempfailed, 
> but the others aren't.  (This is perfectly acceptable, per RFC2821.) 
> When the third delivery attempt is made, the previously rejecting 
> recipients are not tempfailed and (assuming) all the recipients in this 
> delivery are confirmed to be rejecting recipients, the whole email can 
> be permanently refused.  (Unfortunately, some spamware reattempts 
> delivery even after permanent failures!)  I'd be surprised if a spec 
> requiring this would be broadly implemented, even though I think it 
> would reliably provide the desired result.  I don't want to write a 
> Standards Track document that never gets past the I-D stage!

I'm sorry, I think this is a direction in a very wrong direction. Just to 
wait until you have all the message and the remote MTA states ".CRLF" 
is a very dramatic effort for an MTA. It can turn traumatic since the 
filter may take more than two minutes to finish it's work, especially 
if it's so loaded because it's waiting for 500 ".CRLF"'s before filtering 
on the messages. 

Some MTA's do this though, but they usually are very streamlined for 
just that. They usually also wait for X kbytes before issuing a denial 
or keep accepting the message stream. That may be non-standard but 
implementers are known to turn corners, but I don't think RFC authors 
are allowed to. 

I think it would be better to modify SMTP ti better suit what you want. 
 
> >>so who's configuration do you use?  It's only
> >>when you are about to put a message in a users INBOX will you know that you
> >>can take action on behalf of all the recipients (cos there is only one).

Some messaging environments have local deliver agents that can deliver 
to multiple mailboxes where the message has multiple recipients. It's very 
efficient for the messaging environment, i.e. a 'feature'. 

> >Yeah- what one would need to do is do an attempted delivery for
> >each SMTP "mail to" recipient and deal with rejections based on
> >that attempt.  You would have to have access to each individual's
> >filter program to do this.  Furthermore you would need something
> >other than SMTP, since once you have said "OK" to the "mail to"
> >you can't take it back later. 
> >
> No.  Actually RFC2821 specifies when the server takes responsibility:
> "In sending a positive completion reply to the end of data indication, 
> the receiver takes full responsibility for the message (see section 6.1).'
> Until then, the server can drop the message and connection and be 
> RFC compliant, even if it has said OK to all the recipients.
> Also there is no such thing as a "mail to" command; I assume you mean 'rcpt to'.

I think what the originator of the "mail to" text meant is that after you 
have said "250 recipient <xxx> ok" you can not selectively take that 
statement back. I.e. you can not say later "5xx this recipient blocked 
your message". 

If you drop the connection willfully w/o trying to tell the remote mta that 
you are going to do so then you are not really rfc compliant either, but 
it's a minor offense. But the remote MTA will just go ahead and contact 
you again with the same message, what will you do then? Just keep 
dropping connections?

> >  Or you would have to have a way
> >of enforcing only one recipient per SMTP session.  (accepting
> >the first "mail to" and giving 4xx responses for each subsequent
> >one would be very hackish.)  
> >
> I would be open to something like: an implementation MAY tempfail 
> recipients after the first one to facilitate per-user 'refuse' actions.  
> An implementation could, for example choose to do this for mail from any 
> blacklisted IPs, and safely do so even using aggressive blacklists that 
> list IPs that send both ham and spam.  It's not clear whether this would 
> violate any RFCs.  RFC 2821 does not allow "Rejection of messages (for 
> excessive recipients)" < 100 but a 450 rejection of that sender because 
> the mailbox is unavailable for policy reasons seems acceptable to my 
> reading.

Same thing, you are opening your box up to a lot of connection attempts.
Don't forget, multiple rcpt to's are very efficient (for messages you want 
to receive). It would be a shame to hack it away.

In general, instead of this hacking and SMTP/MTA unfriendly behaviour, 
why not have the SIEVE script/scripts/engine respond to every command 
that is sent to the MTA. The script can then state "pass/fail" or more elaborete 
verdicts to the MTA for each statement. Wouldn't this be easier and more 
SMTP friendly than what is described above?

Rgds,
-GSH

---

From: "Mark E. Mallett" <[email-masked]>
Date: Sat, 1 Nov 2003 13:09:20 -0500

> the receiver takes
>   full responsibility for the message (see section 6.1).'
> Until then, the server can drop the message and connection and be RFC 
> compliant, even if it has said OK to all the recipients.

Only if you ignore the context in which I was making those comments:
i.e. about a message with multiple recipients and about not being
able to selectively reject each recipient after the DATA.

> Also there is no such thing as a "mail to" command; I assume you mean 'rcpt 

My apologies, yes that's what I meant.  An embarrassing error, but I
hope people can read through it.  Sorry about that.

> >Here it would be great to have an SMTP
> >enhancement allowing post-DATA rejections.
> > 
> >
> SMTP already allows this.  It's clearly permitted by RFC 2821 to send a 
> negative completion reply to the end of data indication.  Were it not, 
> this I-D would be a non-starter!

Again, I was only talking about post-DATA per-recipient rejection.
Clearly one can reject the entire message after the DATA (that's what
the entire "accept only one recipient" hack comment was based on).

yours,
mm

---

From: "Matthew Elvey (FM)" <[email-masked]>
Date: Sat, 01 Nov 2003 14:04:36 -0800

On 11/1/2003 1:47 AM, Guðbjörn S. Hreinsson sent forth electrons to convey:

>have a filter miss than a false hit. 
No, there is no false positive. The non-rejecting recipients get the 
message.  (The rejecting recipients don't, but that's their intention.  
Labeling that a false positive 'refuse' isn't correct, no matter how bad 
the reason for the rejection.)
Multiple RCPT TO's on email that some folks would refuse with Sieve is 
far from probable, for every user population I can think of, and 
certainly for all of them considered together, but sure, it happens.
I think any efficiency lost by restricting multiple RCPT TO would be far 
outweighed by the efficiencies gained by 'rejecting' spam instead of 
'refusing' it.
I think the net computer resources used would be reduced, and far more 
importantly, the net human resources used would be reduced.
I think there's is zero chance to "modify SMTP" or turn Sieve into a 
Mieve that can direct every stage of an SMTP session.
Let us know of any objections you still have to the multiple RCPT TO 
scheme currently specified by the spec, which is to simply accept the 
message when at least one RCPT TO wants it, causing no false positives, 
as discussed above.

>>message on hand....
Yes, Cyrus Sieve is likely to stay that way, I'd wager.  But, any Sieve 
script evaluation that doesn't use the (not-yet-standard) body test can 
(potentially, that is it's possible to code any implementation so that 
they) run as soon as the header has been received.

And apologies to Mark - I think I misunderstood what he was 
communicating and should have been more friendly than I was - especially 
as I don't want to discourage feedback.

---

From: "Mark E. Mallett" <[email-masked]>
Date: Sat, 1 Nov 2003 18:22:19 -0500

> as I don't want to discourage feedback.

me?  wow, I didn't notice any personal negativity or unfriendliness.
Maybe I'm too dense :-)

mm

---

From: "Guðbjörn S. Hreinsson" <[email-masked]>
Date: Sun, 2 Nov 2003 10:18:39 -0000

> the reason for the rejection.)

Forgive me. I though from reading your message that the intention was 
to refuse the whole message (all routing recipients as specified by the 
RCPT TO's) after the message has been "sent" (as by sending the "." 
and CRLF) by sending a "5xx Filtered xxx". 

So to understand this. SIEVE will reject per RCPT TO at the time 
of the RCPT TO? Not later. 

> outweighed by the efficiencies gained by 'rejecting' spam instead of 

So you want to perform the above by only having a single RCPT TO 
per submission? Does that include the session? Do you allow RSET? 

> message when at least one RCPT TO wants it, causing no false positives, 

Ahh, ok. So a single "positive" for any RCPT TO will outweigh any other 
RCPT TO. So if I have a filter, the filter "hits" on a specific RCPT TO, but 
not on any other RCPT TO, the message is allowed through?

But what's the point then? If this is for catching SPAM messages I think 
they will get the message very quickly and circumwent it...

Furthermore, if you filter on the whole message and then check for the 
RCPT TO's applicability to the filter you are probably wasting a lot of 
resources.

My feeling is there are too many IFs here. This kind of thing needs to be 
analyzed and verified as to what effect it may have on SMTP servers. 

Rgds,
-GSH