Subject: Re: Filtering Infrastructure (was "Re: Who will write filtering scripts?")

From: "Jack De Winter" <[email-masked]>
Date: Fri, 28 Mar 1997 17:24:01 -0500

>ACAP does not verify syntax on options.  There really isn't any "mapping"
>when storing filters in ACAP.  The way I see the filter dataset, it is very
>simple: just a set of entries, each entry consists of an entry (name)
>attribute and a filter attribute.  The entry attribute is any unique handle
>assigned by the filter generator (probably the client), and the filter
>attribute is just a text string.  For example:

Well, to be precise, ACAP does not enforce any syntax, but the gatweway
between ACAP and the data store may.  For example, ACAP will accept the
data, pass it to the routines to store that data, and those routines may
cause the data to be rejected.  This was dicussed at the ACAP meeting,
where I believe the consensus was that the ACAP datasets did not actively
do syntax checking but the backend was allowed to reject data.  My argument
was for a good method to find out any format or syntax to allow for a generic
client to determine 'what is acceptible'. (I will be coming up with a 
'datasets' dataset that I will hope that people will use for their dataset
definitions. )

>\user\fred\filters
>	dataset.entry = "make-money-fast"
>	filter = "if = ( subject, "make-money-fast" ) then trash"
>
>	dataset.entry = "peggy-sue"
>	filter = "if = ( from, "[email-masked]" ) then file
>			("personal")"
>
>This way the dataset knows very little of the structure of the filters, and
>it is easy to extend.

Hrmmm... I can see why you might want to do this, but I see problems.  I
might want to create a script for a given sub-mailbox or a script where
I took mail I wanted to deal with and do something, and as the default
bounce the message.  These would be hard to convey with an entry by
entry basis, as would any idea of including other files.  I would rather
see something like:

\sieve-filter\users\fred -> filter.name = "main filter"
filter.script = "if () if() dodefault"

and have the backend verify the script when the script is set up.
For example, my backend verifies the script and only generates and
updates the 'compiled' script when it detects no errors.

>Yes, there needs to be a way for a site to advertise which filter
>extensions it supports, and there needs to be a default action for
>unrecognized filter clauses or actions.  I think filter extensions should
>be advertised as part of advertising filtering support, in an IMAP
>CAPABILITIES.  This does beg the question of how to advertise non-IMAP
>support, but at least it is a start.  I agree that default action should be
>to file in INBOX.

This gets hairy fast... how do you differ between a script that is supposed
to be run from an IMAP section and one that is not?  Unless you have a
modifier at the top that says 'ENFORCE-EXTENSIONS <x> <y>' where <x> and <y>
are extensions that must be supported, it gets difficult.

The other question is what to do when a script fails?  If you have one
operating mode 'when MTA is about to put in mailbox', you do not have to
worry about different modes and you just make sure to only update the
current script version when the script is valid.  Adding different modes
will start to make the script knowledgable about where it is run from,
how to send any errors to the user to get them to fix the script.

regards,
Jack

-------------------------------------------------
Jack De Winter - Wildbear Consulting, Inc.
(519) 576-3873		http://www.wildbear.on.ca/

Author of SLMail for 95 & NT (http://www.seattlelab.com/)

---

From: Tim Showalter <[email-masked]>
Date: Fri, 28 Mar 1997 17:48:30 -0500 (EST)

(I trimmed the To/CC lines, since everyone's reading the list anyway...)

The filtering draft I've been working on was based on a document that
happened well before we got as worried about ACAP as we are now.  It is
basically a flat-file script/program that has a bunch of if/elsif/endif
clauses.  I rather like the format, because it means it can be edited by
hand, and it has no particular dependancies on ACAP.

If we take the if/elsif/endif clauses and push them into sparate ACAP
entries, you get the same thing, except you have no guarantee on order of
evaluation, which is a fairly useful thing to have if you'd prefer to toss
out any message you don't filter.

if any-of (header ("subject") contains ("HEY-TIM-ACTUALLY-READ-THIS"),
	header ("from" "sender") contains ("wall" "wcw" "weiler" "rob") then
	fileinto "INBOX"
elsif header ("to" "cc") contains ("[email-masked]") then
	fileinto "INBOX.mta-filters"
else
	reply message
I dropped your message, because I don't know you, and I think
your message is unsolicited.  If this is not the case, send mail with the
subject "HEY-TIM-ACTUALLY-READ-THIS".
.
	toss  # throw message away, it's probably spam
endif

(This script might actually comply with the syntax I was working on.)

As I understood the example, it could be difficult to get this sort of
split.  Did I take too slim a view?  The above is a really paranoid
approach ...

I want to advocate an approach that allows savvy script authors to bunch
together similar cases to save space and time.

If the message is from anyone at CMU.EDU then
	If the message is from wall then
		put it in INBOX.from.wall
	If the message is from rob then
		put it in INBOX.from.rob
	If the message is from shadow then
		toss
	stop
Else [ Sort on mailing lists here or something ... ]
	If ...
	... Endif
	stop
Endif

toss

> to be run from an IMAP section and one that is not?

What's the difference, other than mailbox names?

I suggest a "normal" action, which is "do whatever you'd do if you weren't
actually filtering this stuff."  Under IMAP, it's file into INBOX.

> Unless you have a

I suspect this is a good idea to have anyway.

> how to send any errors to the user to get them to fix the script.

I believe the thing to do when a script fails is
   * drop the message into INBOX (or the local equivalent)
   * drop another message into INBOX, saying "An error occurred...",
     although the filtering agent has to be sure not to do this too often

-- 
                                           Tim Showalter [email-masked]

---

From: Randall Gellens <[email-masked]>
Date: Fri, 28 Mar 1997 15:40:43 -0800

At 5:24 PM -0500 3/28/97, Jack De Winter wrote:

> do syntax checking but the backend was allowed to reject data.

As I recall it, there was general rejection of the idea that ACAP, as a
protocol, would enforce syntax.  There was some agreement that one could
have an implementation that would apply some syntax or content rules, but
it should be pretty exceptional.  Your example was using ACAP to store
server configuration, where you wanted to be sure not to let anyone store
"Hi Mom!" in the "Maximum number of threads" attribute.  That might be OK,
but having an ACAP server apply all sorts of rules to the data would mean
that you then have to invent a way for clients to discover the rules, and
you end up with a very complex mess.  If syntax enforcement is required,
then it needs to be in the protocol.  I think we've decided it doesn't need
to be in ACAP.  Maybe I was jet-lagged and wasn't following the discussion,
but that's how I remember it.  I suppose this is a question for the ACAP
list.

> ... I took mail I wanted to deal with and do something, and as the default
> entry basis, as would any idea of including other files.

I really think we need to keep the filtering spec as simple and bare-bones
as possible, otherwise it will never get consensus.  I recall the BOF in
San Jose saying explicit inclusion of other filters was too complex for now.

> ...and have the backend verify the script when the script is set up.

I don't think the ACAP back-end should do anything except store and
retrieve the data.  No syntax checking.