Subject: RE: [Asrg] 3. Requirements - Non Spam must go through

From: Jason Steiner <[email-masked]>
Date: Fri, 11 Jul 2003 10:53:44 -0700

"Walter Dnes" <[email-masked]>
> 
>   Welcome to...
> 
>   Walter's 1st (f)law of content-based spam detection;
>   You can *NOT* use content-based spam detection against email from a
> mailing list that discusses spam.

Sure you can. In fact, I do.

>   Think about it for a minute.  On any spam-discussion mailing list,
> it is on topic to show examples of spam.  If a spam-detector is working
> properly, it *WILL* detect the spam-samples, and flag the
> spam-discussion email as spam.

Think about it for a minute. If a human can distinguish between examples
of spam under discussion and actual spam, then it is theoretically possible 
for a content-based filter to duplicate that reasoning. And it turns out to
be possible in practice as well as theory.

>   Corallary 1
>   Content-based spam detection is imperfect, but if you *INSIST* on
> using it, the best approach requires that you...
>   a) absolutely whitelist any spam-discussion mailing-lists

A whitelist *is* one way to make the distinction (after all, a human says, 
"This comes from a user of the ASRG list, therefore it is something I
probably want.") But fuzzier methods work as well.

>   b) do *NOT* include emails from spam-discussion mailing-lists in the
>      filter's "learning" mode.

To the contrary, there are still enough differences between the two  that
a well-designed filter can learn the difference almost as easily as it learns 
the difference between "normal" spam and email. Think about the headers
in general, as well as the way that such example messages usually
get forwarded.

>   Walter's 2nd (f)law of content-based spam detection;
>   Even 100% correct (0% false positives and 0% false negatives)
> content-based spam detection that properly flags 14 megabytes of spam and 1
> megabyte of non-spam is useless given an inbox with 5 or 10 megabytes
> of capacity.

Now *this* is very true. However, content-based filtering can be 
an important part of a 2-tier anti-spam solution. Spam steals both
system resources and human time. A content-based filter can
address the latter, and provide information used to 'train' the source-
based defenses that address the former.

jason

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: "Pete McNeil" <[email-masked]>
Date: Fri, 11 Jul 2003 14:31:03 -0400

>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]] On 
>Behalf Of Jason Steiner
>Sent: Friday, July 11, 2003 1:54 PM
>To: [email-masked]
>Subject: RE: [Asrg] 3. Requirements - Non Spam must go through

<snip>

>> content-based spam detection that properly flags 14 
>megabytes of spam 
>> and 1 megabyte of non-spam is useless given an inbox with 5 or 10 
>> megabytes of capacity.
>
>an important part of a 2-tier anti-spam solution. Spam steals 
>both system resources and human time. A content-based filter 
>can address the latter, and provide information used to 
>'train' the source- based defenses that address the former.
>
>jason

Absolutely, positively! Version 3 of Message Sniffer (under development)
uses precisely this approach... and as it turns out it is not only
possible for a content filter to train a source based filter, but it can
even train itself to recognize new patters of content. I can't share
results on this yet - because they are all highly experimental and
otherwise meaningless, but I can tell you that the theory holds up very
well in practice.

One actual case I can point to: Message Sniffer has been used in at
least one installation where the IP source of messages trapped by
Message Sniffer (the content filter) is driven forward to an IMGate box
(MTA Gateway running Postfix). In this system, the content filter
captures 4-6% of spam continuously (indicating continued "learning")
while the MTA Gateway box rejects the rest of the spam based on IP ACLs.
Performance reports on this system have been extremely good, and as a
methodology this seems to be a very scalable solution since the
"expensive" content filtering load is significantly reduced while it's
effects are still highly leveraged.

Systems like this should do very well establishing policys for the
"anonymous" case... that is: SOFT CONSENT, and SOFT DENIED CONSENT. For
example, in the rejection policy for SOFT DENIED CONSENT, the REJECTION
could include an action item to add an entry to the DENIED CONSENT
policy by source address with an appropriate expiration. (The expiration
should also be "abuse sensitive").

If consent policies can be structured in this way they will be able to
provide an adaptive framework that can take advantage of any current or
future technologies for detecting abuse... In the same vein, the
non-abuse case can also be leveraged where consent token methodologies
can be adaptive - recording new senders in the CONSENT policy
automatically under special conditions so that email address changes can
be accomodated without intervention. Again in this vein is the ability
for a CONSENT policy to contain a provisional component for "sensing"
the email that the RECEIVER sends such that automated mechanisms might
be employed to add CONSENT entries as needed - that is, an "auto
whitelist" feature.

Even though the consent policy structure will be a standardized
mechanism, there's no reason any number of services and utilities can't
be employed by that consent policy provided the services are available
where the policy is employed. To that end, the structure of a consent
policy should be standardized in such a way that "any" available
mechanism can be employed as part of the local consent model for a
RECEIVER in their given environment. The additional goal of making
consent policy documents portable should also be well considered.

I think an IMPLEMENTATION section should be included to tie local system
resources into the consent policy in a way that is platform independent.

_M

---

From: [email-masked] (Justin Mason)
Date: Fri, 11 Jul 2003 13:25:23 -0700

>otherwise meaningless, but I can tell you that the theory holds up very

SpamAssassin does this for the AWL and Bayes auto-learning components,
fwiw.

--j.