Subject: [sieve] Error reporting, was Re: [draft-melnikov-sieve-external-lists] 3. Security Considerations, Paragraph 2

From: Cyrus Daboo <[email-masked]>
Date: Tue, 08 Sep 2009 09:21:48 -0400

Hi Jeffrey,

--On September 8, 2009 2:54:09 AM -0400 Jeffrey Hutzelman <[email-masked]> 
wrote:

>> It may be straightforward when dealing with a simple LMTP server-based
>> implementation, but let's please not forget that such implementations
>> have the
>> serious issue that errors reported after message data transfer cannot in
>> general be reflected back as an SMTP error.
>
> No, but one could certainly treat lists referenced in this way as a part
> of the data needed to run the script, and fetch them up front, before the
> script begins executing.  Of course, that presumes that one can "fetch"
> the list.  The problem becomes much worse when the list is a query-only
> service, because then you're going to need to talk to the service while
> running the script, and _that_ could fail.

This part of the thread brings up an issue we have mentioned in the past 
but never pursued - namely the ability to provide better error reporting.

We have talked in the past of having something equivalent to a DSN 
generated by the SIEVE implementation and sent to the owner of the script 
when runtime errors occur, so that they get immediate feedback on that with 
sufficient detail to know what went wrong and possibly how to fix it.

Of course sending such details to most users may not be good in practice as 
a lot of users have no idea that they are using SIEVE because their 
filtering front-end obscures that fact.

So in reality the majority of these reports would likely go to the SIEVE 
system admin. Chances are most SIEVE systems already have some form of 
error log that admins can look at, so maybe the DSN idea is not worth it. 
However, I think it is still worth having the discussion.

-- 
Cyrus Daboo

---

From: [email-masked]
Date: Tue, 08 Sep 2009 07:47:52 -0700 (PDT)

> when runtime errors occur,

Actually, that's how our implementation works currently. This was the
only error reporting mechanism we could come up with that got the information
about the failure to the right person.

For us, Tte hardest part of it turned out to be figuring  out who the script
owner is. We have a lot of different types of sieves in our implemetantation,
and several of them didn't have the bookeepting necessary to track an owner
address. So we have to go in and retrofit that.

> so that they get immediate feedback on that with

Well, I wouldn't calll the feedback immediately, both in the sense that an
error may not crop up until just the right message is received. And when it
does crop up, you have to read your mail to find out about it.

But since we don't have anything like an always-up IM channel to the user for
reporting sutff like this, a DSN was the best we could do.

> filtering front-end obscures that fact.

Maybe we've just been lucky, but we do send notices of seive errors to the end
user owner and so far this has not been a problem. There are a lot of things I
don't like about our sieve generation front end, but I have to say that it
always seems to construct syntactically valid and executable scripts. So we
haven't seen cases of inscrutable errors being sent to end users.

If this did become a problem we'd probably aad an option to force the owner
address for such sieves to some sort of administrator.

> However, I think it is still worth having the discussion.

While the DSN reporting idea has worked well for us and I wouldn't have a
problem talking about it in some sieve specification, I don't think it offers a
solution to the inaccessible list problem. For one thing, a list server failure
would, if it generated these sorts of things, result in a deluge of such
notifications. Keep in mind that we're talking about environments where the
number of sieve evaluations is best measured in units of hundreds per second
per server.

				Ned

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 08 Sep 2009 23:10:02 +0100

Cyrus Daboo wrote:

> Hi Jeffrey,
>
> --On September 8, 2009 2:54:09 AM -0400 Jeffrey Hutzelman 
> <[email-masked]> wrote:
>
> This part of the thread brings up an issue we have mentioned in the 
> past but never pursued - namely the ability to provide better error 
> reporting.
>
> generated by the SIEVE implementation and sent to the owner of the 
> script when runtime errors occur, so that they get immediate feedback 
> on that with sufficient detail to know what went wrong and possibly 
> how to fix it.

This is already described in the base Sieve spec.
I've recently implemented this in Isode M-Box. Before that we were 
already logging such failures to logs.

> Of course sending such details to most users may not be good in 
> practice as a lot of users have no idea that they are using SIEVE 
> because their filtering front-end obscures that fact.

This can be indeed a problem. There is also a question on how frequently 
such errors should be reported. Reporting them on each failed Sieve 
script invokation is going to be quite irrating to users.

> So in reality the majority of these reports would likely go to the 
> SIEVE system admin. Chances are most SIEVE systems already have some 
> form of error log that admins can look at, so maybe the DSN idea is 
> not worth it. However, I think it is still worth having the discussion.