Subject: error reporting with run time errors

From: Rudy Gevaert <[email-masked]>
Date: Wed, 12 Dec 2007 10:46:33 +0100

Hello,

I have subscribed to this list to report a (possible) problem with error 
reporting.  Please accept my apologies if this is not the correct place 
to report it or if I'm not following the list policy.

RFC3028 and the latest draft of RFC3028bis state in section 2.10.6.

2.10.6.  Errors

    ...

    When an error happens, implementations MUST notify the user that an
    error occurred, which actions (if any) were taken, and do an implicit
    keep.

We are using Cyrus Imapd and their sieve implementation.  I found out of 
the following problem last week.  Our incoming mail relays accept email 
with a size limit of 12MB.  We have users that have sieve redirect 
scripts on the backend mail stores.  As you know to forward the email a 
sendmail forward is used (as stated in the RFC, but this is left out in 
the latest draft).  Because of an unfortunate oversight the postfix that 
runs on the mail stores (and thus it is postfix providing the sendmail 
binary) did have its message size limit set to 10MB.  Forwarding 
messages greater than 10MB didn't work.

In my logs I saw that like this:

Nov 22 17:29:40 manaslu postfix/postdrop[23167]: warning: uid=980: File 
too large
Nov 22 17:29:40 manaslu postfix/sendmail[23166]: fatal: (...@...)(980): 
Message file too big
Nov 22 17:29:41 manaslu mail6/lmtp[9186]: sieve runtime error for 
[email-masked] id <[email-masked]>: Redirect: 
Sendmail process terminated normally, exit status 69

I understand that the RFC says the user needs to be notified, but this 
doesn't happen.  I agree that the issue is implementation specific, but 
I want to raise the following questions.

a) Who is the user?
b) How must the user be notified?

For (a) you would be assume it is the user who set the sieve script. 
But actually it should be (in this case) the sender that at least gets a 
message that the email could not be forwarded (it delivered, but not to 
the final destination).

(b) I would say by email, but depending who the user is, there could be 
problems.  E.g. if he is forwarding all incoming email to an other 
address, it depends if he will be notified (mail could be lost).  If the 
forwarding mechanism is broken the mailadmin should be notified.

Maybe the RFCbis could be clearer on some points?

Thank you in advance,

Rudy
-- 
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Rudy Gevaert          [email-masked]          tel:+32 9 264 4734
Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office
Groep Systemen                    Systems group
Universiteit Gent                 Ghent University
Krijgslaan 281, gebouw S9, 9000 Gent, Belgie               www.UGent.be
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Wed, 12 Dec 2007 13:31:32 +0100

On Wed, 2007-12-12 at 10:46 +0100, Rudy Gevaert wrote:
> messages greater than 10MB didn't work. [...]
>
> doesn't happen.

I'm not sure that the redirect action failed in this case.  it
successfully passed the message on to Sendmail.  in general, you can't
guarantee that the next hop will be successful.

> I agree that the issue is implementation specific, but 

yes.

> message that the email could not be forwarded (it delivered, but not to 

why?  there's nothing wrong with notifying the sender as well (but
please limit backscatter!), but it's the recipient (script owner) who
needs to be notified that his filter failed so that he can fix it.

I think it is too onerous to require the "redirect" delivery to be
successful.

if a "redirect" delivery failure causes the script to fail, the Sieve
implementation can not rely on MDN to do the reporting, it has to keep
track of delivery success or failure itself.  this is a bit awkward,
since it has to maintain its own queue, you can't just hand it off to
your MTA via "Sendmail-API", since there is no back-channel to the
submitter.  what's worse -- the script can't terminate until the
delivery is done, which can take days!  what would happen to commands
after the "redirect" in the script?  should they be postponed?
obviously not.

the Cyrus implementation does not perform actions as it runs through the
script, it collects a list of actions which should be taken, and
performs these *after* the script has finished.  I think this is a
perfectly reasonable approach.

BTW, in Cyrus (at least 2.2.x), a redirect is implemented as a message
submitted with the cyrus system user as the envelope sender, so failed
redirects go to the mail admin.  this is not a Sendmail forward, since
that keeps the envelope sender from the original message.  it would
probably be good to make Cyrus compliant here.

> forwarding mechanism is broken the mailadmin should be notified.

the implementation MUST be able to do a "keep".  if it can't do that, it
should not have accepted the message in the first place.  

> Maybe the RFCbis could be clearer on some points?

publication of 3028bis as RFC 5228 is imminent, so it will almost
certainly have to wait until 3028ter.

some text to clarify success or failure of "redirect" might be a good
idea.
-- 
best wishes,
Kjetil T.

---

From: Rudy Gevaert <[email-masked]>
Date: Wed, 12 Dec 2007 15:11:52 +0100

> guarantee that the next hop will be successful.

I think it depends on what you see as 'failed' or 'succeeded'. :)

In this case it was given successfully to sendmail, but sendmail didn't 
forward it, it even exited with status 69.  But that seems to be ok for 
the cyrus implementation.

> needs to be notified that his filter failed so that he can fix it.

Now that I gave it an other thought I think the user who set the forward 
should be notified.  The notification should be put in his inbox.  I'm 
saying this because:
- you can't rely on the redirect action to actually work, so if the 
forward doesn't work the notification will be put in his inbox
- the redirect action could be set up for a specific action

The only problem that remains is that the user still has login to his 
mailbox to see the notification if the redirect action doesn't work (in 
that case).

> after the "redirect" in the script?  should they be postponed?

Sorry, I don't understand what you are trying to say.

> perfectly reasonable approach.

I can't comment on this, I would have to look at the source.

> probably be good to make Cyrus compliant here.

This is already the case in 2.3.10

> some text to clarify success or failure of "redirect" might be a good

I would think so too.

Rudy

---

From: Ned Freed <[email-masked]>
Date: Wed, 12 Dec 2007 13:44:38 -0800 (PST)

> Sendmail process terminated normally, exit status 69

I don't know the underlying architecture here, but it isn't obvious
to me that this is a redirect error. Redirect reinjects a message into
the transport infrastructure. That message can bounce at some subsequent
point even though the redirect itself was succesful.

FWIW, the way we handle redirect in our implementation is to put a copy of the
message with new recipients in what amounts to a submission queue. The
submission may fail at a later point but that doesn't prevent the redirect from
completing succesfully in almost all cases.

> a) Who is the user?

The owner of the sieve.

> b) How must the user be notified?

That's implementation-specific. Again FWIW, we do this by sending a special
notification message to the sieve owner.

> For (a) you would be assume it is the user who set the sieve script.

Yes, but note that admins can and do set up sieves for users but in such a case
it isn't clear they're the ones who should receive error notifications.

> message that the email could not be forwarded (it delivered, but not to

Maybe, maybe not. It's a question of intent. If the redirect is to, say,
a remote archive and the main copy of the message was delivered normally,
notifying the sender about a copy they don't know and don't care about, 
may end up causing more problems than it solves.

> address, it depends if he will be notified (mail could be lost).

Remember that in the event of an error the sieve result is ignored and
a keep is performed. This mitigates at least some of the issues of
using email to make the report.

> If the

It really depends on how it's problem.

> Maybe the RFCbis could be clearer on some points?

A bit late for that now that 3028bis is about to come out as an RFC.

				Ned

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Thu, 13 Dec 2007 16:23:00 +0100

> the cyrus implementation.

this is a quality of implementation issue, IMHO.  as long as only *some*
types of errors can be discovered, and the types depend on
implementation strategy, I don't think it is appropriate for the Sieve
spec to make a list.

> mailbox to see the notification if the redirect action doesn't work (in 

yes, this is unfortunate.  a Sieve implementation is allowed to use
other notification methods if they're available, though.  (e.g. the user
allows XMPP)

> Sorry, I don't understand what you are trying to say.

not sure if I'm able to state it more clearly, but I'll try.

current status is (as I understand the spec) that a redirect action
succeeds as long as it is syntactically correct.  the Sieve
implementation isn't even required to check the supplied address for
syntax validity.

if you want the success of the redirect action to depend on the success
of the actual forwarding of the e-mail, you need a tight integration of
the sending action in your Sieve implementation.  the server you're
redirecting to may be down, and then Sieve will have to queue the
message, and postpone the remainder of the Sieve script until the
redirect action returns with a definite answer.  this is not workable,
IMHO.

-- 
regards,
Kjetil T.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 13 Dec 2007 16:44:22 +0100

Kjetil Torgrim Homme writes:
> as long as only *some* types of errors can be discovered, and the 
> types depend on implementation strategy, I don't think it is 
> appropriate for the Sieve spec to make a list.

I agree. That said, if the actual redirection has no possiblity of 
succeeding I don't think the redirect command should succeed. The 
details aren't IMO specifiable, but that simple statement is a useful 
rule.

Not that it matters at this time - 3028bis has come too far and 3028ter 
is far away.

Arnt