Subject: Re: action reject and smtp RCPT TO:

From: Дилян Палаузов<[email-masked]>
Date: Sat, 01 Mar 2008 19:14:21 +0100

Hello,

	I read the discussion from Juli 2007 about the reject concerns. There 
is still something that is not addressed in 
draft-ietf-sieve-refuse-reject-06 -- early evaluation of scripts. More 
concrete I mean that a script can be executed after the RCPT TO and 
before the DATA stage, and then has a stop;.

draft-ietf-sieve-refuse-reject-06
2.1.1. Rejecting a message at the SMTP/LMTP protocol level

    Sieve implementations that are able to reject messages at the SMTP/
    LMTP level MUST do so and SHOULD use the 550 response code.  Note
    that if a message is arriving over SMTP and has multiple recipients,
    some of whom have accepted the message, Section 2.1.2 defines how to
    reject such a message.

Does this mean, that a recipient cannot be rejected after RCPT TO: and 
before DATA: in a multi recipient message? In the discussion I have not 
seen why this shall not be possible.

	Със здраве,
		Дилян

Kjetil Torgrim Homme wrote:
> On Wed, 2008-02-13 at 21:09 +0100, Дилян Палаузов wrote:
>> Hello,
>>
>> If sieve scripts are written in the form:
>>
>> if (envelope tests....) { reject; stop;}
>> some other tests and actions;
>>
>> the scripts can be applied once after the RCPT TO: command and once 
>> after the DATA command. If the first script calling is successful (stop; 
>> is reached without trying tests, that require headers or body), and the 
>> reject action is performed, then the sieve script can reject the message 
>> at the RCPT TO: level.
> 
> see the thread "Re: List of open issues with Sieve reject draft
> (draft-ietf-sieve-refuse-reject-02.txt)" from 2006-07-11.
> 
>> The application of such scripts is, e.g. for mailing lists, with scripts 
>> like
>>
>> 	if (recipient is a mailing list and sender is not allowed to post 
>> there) {reject; stop;}
>>
>> which will save the consequent NDRs, send from the mailing list 
>> software. This approach is not considered in 
>> draft-ietf-sieve-refuse-reject-06, 2.2 Action reject...However 
>> implementations MAY refuse delivery over protocol ..., if and only if 
>> all of the following conditions are true:  2. ...
> 
> I agree it would be beneficial to allow early evaluation of Sieve
> scripts, but I believe we need a separate draft to clarify how it should
> work.  it would be preferable if the refuse-reject document had some
> text to indicate the possibility.
>

---

From: Philip Guenther <[email-masked]>
Date: Sat, 1 Mar 2008 15:58:45 -0700

On Sat, 1 Mar 2008, Дилян Палаузов wrote:
...
> Does this mean, that a recipient cannot be rejected after RCPT TO: and before 
> DATA: in a multi recipient message?

Yes, that's exactly what it means.

> In the discussion I have not seen why this shall not be possible.

Because doing so in SMTP is impossible without unstandardized extensions. 
(How is the server supposed to indicate that a message is being rejected 
for rcpt A but not rcpt B?)

Philip Guenther

---

From: Дилян Палаузов<[email-masked]>
Date: Sun, 02 Mar 2008 00:16:53 +0100

Hello,

>> before DATA: in a multi recipient message?
> 
> Because doing so in SMTP is impossible without unstandardized 
> extensions. (How is the server supposed to indicate that a message is 
> being rejected for rcpt A but not rcpt B?)

MAIL FROM: <[email-masked]>
250 OK
RCPT TO: <[email-masked]>
250 OK
RCPT TO: <[email-masked]>
550-You are not permitted to post to [email-masked] from
550 your [email-masked] address.
RCPT TO: <[email-masked]>
250 OK
DATA ...

Why shouldn't be a sieve script allowed to set the 550-response here?

	Със здраве,
		Дилян

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sun, 2 Mar 2008 14:27:22 +0100

> being rejected for rcpt A but not rcpt B?)

MAIL FROM:<[email-masked]>
250 OK
RCPT TO:<[email-masked]>
     if envelope :all :is "from" "[email-masked]" {
         reject;
     }
550 I reject mail from [email-masked]

Arnt

---

From: Ned Freed <[email-masked]>
Date: Sun, 02 Mar 2008 15:52:17 -0800 (PST)

> draft-ietf-sieve-refuse-reject-06 -- early evaluation of scripts.

Of course it s possible to, say, evaluate a script that only uses envelope
"from" tests at the MAIL FROM stage, or to evaluate a script that only uses
envelope tests at the RCPT TO stage, but I really have to question the utility
of an extended discussions of the issues surrounding using sieve this way. The
overwhelming majority of scripts we end up with in practice make extensive use
of header and other tests that depend on message content and henace cannot be
evaluated until after the DATA phase.

And while the reject action is almost certainly the case where early evaluation
offers the most advantage, it isn't the only test or action where early
evaluation brings up issues, some of them fairly subtle. Consider spamtest - it
is possible in many cases to compute a spam score based solely on envelope
information. But better results are usually produced if content is considered.
So how should this be handled? Should

    require ["spamtest", "reject, "relational", "comparator-i;ascii-numeric"];
    if spamtest :value "ge" :comparator "i;ascii-numeric" "3"
    {
	reject "Keep your spam";
    }

be evaluated at RCPT TO when possible, or should it have to wait until after
DATA because a better score is available then, or perhaps it should be
evaluated twice? And all this totally ignores the implementation issues with
actually using exist spam filtering software with only an envelope.

Now, has the issue of early evaluation been one described in the sieve base
specification and given due considerstion in all the other sieve documents (the
currentdate test defined in date-index is an obvious example of a place where
such considerations should be described), not only would I have no problem with
adding such a discussion to this document, I would have insisted on it. But
that's not now things are. We haven't dealt with this concept at all and IMO
trying t do so as s side note in this one extension is not a good idea.

If early evaluation is indeed of sufficient interest to warrant specification
then the way to handle it at ths point is with an additional informational
specification. But as I stated previously, I am dubious that's there's
sufficient operational utility here to bother.

				Ned

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 3 Mar 2008 09:13:44 +0100

FWIW: My sieve interpreter does exactly what we're (not) allowing: I 
fetch the sieve script as part of verifying that the rcpt to address 
exists, and evaluate until some test fails for lack of data. I think 
the draft as it stands is fine.

Arnt

---

From: Ned Freed <[email-masked]>
Date: Mon, 03 Mar 2008 06:54:19 -0800 (PST)

> FWIW: My sieve interpreter does exactly what we're (not) allowing:

AFAIK nobodhy is talking about not allowing this. The question is whether or
not we should specify all of the nuances involved in supporting this mode of
operation, and if we're going to whether or not this is the right place to do
it. I remain to be convinced of the need to specify this and I'm opposed to
doing it in this document.

> I

Our implementation doesn't work this way but nevertheless allows some types of
Sieves to be evaluated at RCPT TO or even MAIL FROM time.

				Ned

---

From: Дилян Палаузов<[email-masked]>
Date: Mon, 03 Mar 2008 16:54:26 +0100

Hello,

	I think draft-ietf-sieve-refuse-reject-06 implicitly forbids sending 
550 after RCPT. In Section 2.1 (Action ereject) the SMTP 550 way is 
shifted to Section 2.1.1 (Rejecting a message at the SMTP/LMTP protocol 
level) and Section 2.1.1 shifts the problem to Section 2.1.2 (Rejecting 
a message by sending a DSN). Or may be I am wrong?

	About the spamtest issue: it shall be evaluated after data. If I 
remember correctly, there was an idea to extend the envelope test to 
check against the sending host (apart from sender and recipient) and 
check using external lists if the sending host is blacklisted. Then the 
mail can be rejected according to the user's preferences and not due the 
site policy. Can we leave for now spamtest out the discussion?

	The idea of the early evaluation in multi-recipient messages is to 
reduce the amount of generated NDRs, when some of the recipients are 
mailboxes (accept spam) and others are mailing lists (who do not want to 
discard messages from non list members, ignoring the spaminess). The 
problem with NDRs is that they might end in a spamtrap/honeypot and 
blacklist your server. With early evaluation all this is avoided... And 
once again: does draft-ietf-sieve-refuse-reject-06 forbid this behaviour?

 > the draft as it stands is fine.

I think this is wrong, as it does not allow combining keep and reject. 
The reject action shall be executed after RCPT TO:, if the script 
terminated successfully (= reached <EOF> or stop; without making 
header/body tests, or invoking the keep action) being executed there.

	Със здраве,
		Дилян

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 3 Mar 2008 17:48:18 +0100

> I think this is wrong, as it does not allow combining keep and reject.

In my code, keep (even implicit keep) is a command which cannot complete 
at RCPT TO time for lack of data. So it works.

I agree with Ned that going into this level of detail in the document is 
not helpful. This is a very minor point, and spending text on it will 
help very few of the document's readers. For most readers it's just a 
weird digression, and digressions are not good.

Arnt

---

From: Philip Guenther <[email-masked]>
Date: Tue, 4 Mar 2008 09:33:25 -0700

>> Yes, that's exactly what it means.

My apologies for my unclear statement.  The above was in relation to this 
text from the draft:
    Note that if a message is arriving over SMTP and has multiple
    recipients, some of whom have accepted the message, Section 2.1.2
    defines how to reject such a message.

In light of the quote, I interpreted the "after RCPT" in your question as 
meaning "after being accepted at RCPT time".  That is, I thought you were 
asking if it was possible to perform individual rejection of recipients 
that were accepted at RCPT time.

As for your actual question: IMO, the possibility of rejecting individual 
recipients by rejecting their RCPT commands is permitted by the draft.

Philip Guenther

---

From: Ned Freed <[email-masked]>
Date: Tue, 04 Mar 2008 13:53:43 -0800 (PST)

> a message by sending a DSN). Or may be I am wrong?

I'm sorry, but you are quite simply wrong about this. First, just because
something isn't discussed doesn't mean it is disalllowed. And section 2.1.2 is
quite clear in stating that it applies to rejections occuring after message
data has been received. That's the only sensible interpretation the phrase "may
receive a message via SMTP" can possibly have. And the text goes on to talk
about the fact that this restriction doesn't apply to LMTP because LMTP has
per-recipient replies after DATA, which only makes sense if we're talking about
responses after message data has been transfererd.

Now, while I think the current text is already quite clear, in order
to forestall further argument I suggest changing the first paragraph
of section 2.1.2 to read:

   An implementation may need to respond to the transfer of message
   data when more than one RCPT TO has been accepted by the server,
   and at least one recipient but not all of are refusing delivery
   (whether the refusal is caused by a Sieve "ereject" action or for
   some other reason).  In this case, the server MUST accept the
   message and generate DSNs for all recipients that refused delivery.
   Note that this exception does not apply to LMTP, as LMTP is able
   to reject messages on a per-recipient basis after message data
   has been transferred.

> 	About the spamtest issue: it shall be evaluated after data.

Why? What if the necessary information is already available and nothing in the
message body is capable of overriding it? Why would a ban on early evaluation
of scripts that use spamtest make any more sense than a ban on early evalaution
of scripts that use reject?

> If I
> check using external lists if the sending host is blacklisted.

However, no such extension has been defined or even, aside from a passing
mention, discussed.

> Then the

Not if what we're talking about is explicit adding text discussing early
evaluation of sieve scripts. In that case I insist that the discussion
encompass the effects of early evaluation on every test and action we have
defined so far. When it comes to implementation methods we either have to
discuss a given approach thoroughly or not at all.

> blacklist your server. With early evaluation all this is avoided...

We're all well aware of these issues.

> And

No.

> I think this is wrong, as it does not allow combining keep and reject.

I fail to see anything in this implementation description that would create a
problem for combining keep and reject. And even if it did, since the draft
strongly recommends implementations to forbid this combination (our
implementation definitely does this), I fail to see the point of any of this.

> header/body tests, or invoking the keep action) being executed there.

This goes much too far - it actually recommends if not requires implementations
implement early evaluation - and I am adamantly opposed to it.

				Ned

---

From: Дилян Палаузов<[email-masked]>
Date: Wed, 05 Mar 2008 00:17:13 +0100

Hello,

> Why? What if the necessary information is already available and nothing 
> in the message body is capable of overriding it? Why would a ban on early 
 > evaluation of scripts that use spamtest make any more sense than a
 > ban on early evaluation of scripts that use reject?

Doing heuristic spam analysis can combine a lot of rules, some giving 
positive and other negative weight on the result. The fact that the 
envelope increases the spaminess, does not mean that the body of the 
message will not decrease it. At the end the total spaminess can be 
acceptable for the recipient, even if the envelope went above the limit. 
So to be accurate, the spamtest shall be done after DATA .

In the cases when the spam software works only on the envelopes, or 
cannot decrease the spaminess due additional rules, then spamtest can be 
evaluated before DATA as well.

> However, no such extension has been defined or even, aside from a passing

Ned said on 8th October 2007 to look on 
draft-freed-sieve-environment-01, when it comes out, as the idea of 
remote-host and remote-ip will be presented there. Right now I see it is 
really at http://tools.ietf.org/html/draft-freed-sieve-environment-01.

> implementation definitely does this), I fail to see the point of any of 

If reject is done at RCPT time and there is no DATA afterwards (due the 
lack of accepting recipients), keep cannot be made. (Keep what?)

The need of combining keep and reject was addresses in the discussion 
from 2006-07-11 as a sort of logging the process.

> implement early evaluation - and I am adamantly opposed to it.

This is my view on Arnt's FWIW: Arnt said that reject is executed, if no 
test or actions failed before reaching "reject". I answered, that this 
approach does not allow to combine keep and reject/fileinto and 
reject/notify and reject/whatever and reject. Instead, reject shall be 
applied during early evaluation only if the script terminates 
successfully. (The other option is to make reject if no actions/tests 
failed yet).

	Greetings,
		Dilian

---

From: Ned Freed <[email-masked]>
Date: Tue, 04 Mar 2008 18:18:26 -0800 (PST)

> So to be accurate, the spamtest shall be done after DATA .

Sigh. All of this is completely obvious and completely irrelevant. For the
third time, there can be cases where the analysis done on the envelope is
sufficient and no, repeat NO, analysis of the message content is capable of
overriding. In such cases the message might as well be refused at the envelope
stage and making some arbitrary rule saying this mustn't be done is every
bit as inappropriate as the rule against doing envelope time rejects
you're arguing against (despite the fact that no such rule exists).

> evaluated before DATA as well.

Which is exactly what I've been saying from the start.

> really at http://tools.ietf.org/html/draft-freed-sieve-environment-01.

Correct, but the environment extension gives you access to the necessary
information but doesn't do the whitelist/blacklist part, which necessary for
this to gain any sort of equivalency with spamtest.

I actually favor the more general approach described in the (now expired)
draft-melnikov-sieve-external-lists-01.txt, modulo a few changes I suggested
on the list some time back. This provides a generic external list access
mechanism which can be usedd for all sorts of different purposes.

However, now that you have brought up environment, its another extension that
potentially interacts with early evaluation in interesting ways. I hadn't
really thought about it before and the initial set of environment items doesn't
include anything that depends on message content (nor do any of the exxtensions
we''re planning on implementing), but it is not beyond the realm of possibility
that some environment item might be defined that could not be evaluated at RCPT
TO time.

> from 2006-07-11 as a sort of logging the process.

Yes, I recall the discussion. I still think its a totally bogus thing to do and
the draft still recommends that they be incompatible.

> successfully. (The other option is to make reject if no actions/tests

OK, got it. Arnt has already clarified that this is in fact a nonissue for him.
It is for us as well - we don't stop evaluating at reject or anything else for
that matter besides stop or an error of some sort.

				Ned

---

From: Barry Leiba <[email-masked]>
Date: Thu, 20 Mar 2008 16:54:11 -0400

Ned Freed said...
> doing it in this document.

Where to do it is, I think, the key point.  And so I'll point out that 
Randy Gellens has a draft in lemonade, currently called "Notifications 
Architecture": 
http://tools.ietf.org/wg/lemonade/draft-ietf-lemonade-notifications/

The plan is to fold some Sieve-related stuff into it and perhaps rename 
it to "Notifications and Filter Architecture".  That might be a 
reasonable place to put this sort of discussion.

Barry