Subject: Re: MTA Filters BOF request, LA IETF; Proposed Charter

From: SunilPaul <[email-masked]>
Date: Tue, 13 Jan 1998 00:23:37 EST

In a message dated 98-01-12 17:04:37 EST, [email-masked] writes:

<< with few exceptions, the mail transport system should not be 
 doing mail filtering.  the mail transport's job is to deliver
 mail, not to make filtering decisions on behalf of the user.
 if recipients want to filter their own mail, that's their 
 business, but then it's a UA-level thing.

mail filtering at the MTA is a must to avoid congestion and overloading of
"downstream" servers ... large ISPs like AOL (where I was Internet Product
Manager) implement MTA-level filtering out of necessity: a significant portion
of their traffic (ranging 5-50%) is spam or spam-related (responses, etc)...
not that existing filtering is helping that much ... :)

sp
-- 
Granola: Dedicated to "End Spam as we Know it"
Sunil Paul, CEO
415.575.0862
Now hiring Sr. S/W Engrs, QA, N/W Admins, & Anti-Spam Technicians
...respond to [email-masked] or see www.end-spam-as-we-know-it.com

---

From: Keith Moore <[email-masked]>
Date: Tue, 13 Jan 1998 00:48:44 -0500

> not that existing filtering is helping that much ... :)

the only reason you can really 'filter' spam now is that the
spammers are pretty lazy and generate obviously bogus messages...
but they're getting smarter.  filtering could help in the long
term only if some kind of labelling were mandatory.

Keith

---

From: Tomas Fasth <[email-masked]>
Date: Tue, 13 Jan 1998 11:57:21 +0100

> not that existing filtering is helping that much ... :)

Please, hold just there for a moment.
My opinion is that large ISP anti-spam issues are quite different from
UA-filtering. I think it will improve the discussion a lot if we can
agree on that.

If you think otherwise, try to imagine the potential admin nightmare of
having "downstream" mail account users being able to put individual
filtering rules into action at "upstream" mail relays. It gives a chilly
feeling ...

As far as IETF is concerned, it might not want to endorse such
scenarios.

I think Tim made it quite clear in what contexts SIEVE are being
designed to work in.
The key words are "at point of delivery". Which I want to extend to "as
directed by and in the name of" individual mail accounts. Well, before
you flame me I better add, at least this is one very precise context
among other not so precise.

I made a point several months ago on this list that having rules of
individual accounts applied earlier than this point was probably not
desirable for many reasons. Some persons argued otherwise. So, no
consensus on that I'm afraid.

Tomas

---

From: Steve Hole <[email-masked]>
Date: Wed, 14 Jan 1998 16:47:13 -0700

Message-ID: <[email-masked]>
Priority: NORMAL
X-Mailer: Simeon for Win32 Version Mercury a6 Build (6)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; CHARSET=US-ASCII

> UA-filtering. I think it will improve the discussion a lot if we can

Exactly.   The issues and requirements for dealing with SPAM are far
from well defined.   It may be that Sieve turns out to be a useful tool
for dealing with SPAM, maybe not.   That was not the origin of Sieve --
it was conceived to do unattended rule based processing on messages.

==> Begin Soapbox

I believe the following to be obvious to everyone.  However, we are
arguing for the start of a new standards track activity so it is
important that everyone have a good understanding on this.

The requirements for "unattended UA actions" (rather than filtering) are
reasonably well defined.   There are a number of highly successful
proprietary mail systems that support this type of functionality.
Make no mistake, implementing the same level of functionality in an
Internet client is not an option for any vendor serious about using the
Internet mail architecture as a competitor to proprietary systems.

To execute unattended actions you need rules.   Rules bind message
matching criteria together with actions.   If a message matches some
criteria, execute one or more actions.  We have been doing this for some
time, and there is good engineering experience as to the advantages and
pitfalls.

It seems to me that there are two basic types of actions.  I'll call
them "active" and "passive" actions.  Active actions result in new
messages being injected back into the mail delivery system.  Passive
actions may result in further processing of the message, but the message
is "terminal" -- no new messages result from the actions.  Examples of
active actions include: reply, forward, bounce and reject (DSN or
MDN). Example of passive actions include: file into, annotate, ...
Active actions are usually perceived as more dangerous than passive
actions because it affects the entire infrastructure.

Note that active actions may have a time sensitivity characteristic to
them.  The action must occur within a reasonable time or its usefullness
is diminished or even negated.   For example, reply for purposes of
vacation is useful only if done quite soon.   It is not useful if it
waits until the recipient user returns and initiates processing.

There are a number of engineering issues that must be resolved to
provide an unattended UA action service in a distributed Internet:

1.  When do the rules get applied?

There are two choices here: (1) they get applied when the message
is being delivered into the message store or (2) they get applied
when the mail management tool tries to access them.

(1) is useful for both IMAP and POP clients and for both active and
passive actions.   (2) is useful for POP clients and passive actions.
(2) is totally useless for time sensitive active actions and for IMAP
clients that do NOT want to start moving messages around as a result of
connecting. 

This argues strongly that, to be useful, rules should be applied at
delivery time.

2.  Who applies the rules?

If we accept the arguments made in 1 that rules should be applied at
delivery time then the obvious choice is that the delivery agent (acting
as a UA of course :-) should apply the rules.  I don't believe there is
another logical choice -- perhaps an asynchronous third party process.

3.  How are rule sets communicated to/from a distributed UA that is an
active user interface?

Rather than argue about the semantics of what a distributed UA is, let's
just agree on some things.  User's use an application (a UA) to read
their mail -- lets call it their mail management tool.  They also want
to configure whatever rules for unattended actions that are associated
with their mail, executed by a different UA, in the mail management
tool.  Using anything other than the mail management tool to *configure*
unattended UA rules is unacceptable.  They want to configure their
unattended UA from their attended UA.

 a. The mail management tool and the delivery agent must agree on the
    rule language.

 b. The rules must be held on a per-user basis by: (1) the delivery agent
    itself, or (2) a trusted third party service.

 c. Transfer of rules from the mail management tool to the delivery
    agent must be secure.  This means, at a minumum, that the transfer
    mechanism must be authenticated.

Now we get to the nut of it.   I think that a. just stands.   If you
accept that the rules will be configured in one application and executed
in another, there is no option but to standardize the language.   This
is the core work for our new working group.

I believe that there are any number of workable solutions for b and c.
There are some that make more sense than others.  We probably should
make some recommendations here, but I don't think that it should be part
of the language specification.   Perhaps a second document.

Some obvious choices are:

 Transfer Protocol            Rule Storage
 -----------------            ------------
 ACAP                         ACAP
 LDAP			      LDAP
 HTTP                         File
 FTP                          File
 SQL                          Database
 NFS/AFS/DFS/... 	      File
 
The first two mechanisms were designed for this kind of thing and have
all the desirable semantics for doing it.   The last four are hacks but
could be made to work.  I'm sure there are more.

===> End Soapbox

Cheers.
---  
Steve