Subject: [Asrg] Why are we still here?

From: "Peter Kay" <[email-masked]>
Date: Thu, 30 Dec 2004 07:12:15 -1000

I've been a part of this list for some time now, in mostly lurking fashion.
My question to the group is, if you read all the anti-spam vendors (me
included), they all claim high-9 catch rates and near-zero false positives.
If this is the case, why are we still here trying to eliminate spam when one
would think that, if the claims are true, the spam problem has been solved?
I can only think of one of three reasons:
 
1. The vendor claims are false.  In the real world, you still get lots of
undesireable email.
2. The vendor claims are true, but we're a bunch of perfectionists searching
for infinite resolution of Pi.
3. The filtering paradigm is a non-solution due to increased use of
recipient resources (bandwith, storage, processing, end-user time, etc)
 
I'm jaded here because I just don't get any spam at all. To me, this group
is firmly stuck in #2.  But I'd love to hear everyone's feedback/experience
on "Why are we still here?"
 
Peter Kay
Titan Key Software
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Devdas Bhagat <[email-masked]>
Date: Thu, 30 Dec 2004 23:16:55 +0530

> included), they all claim high-9 catch rates and near-zero false positives.

Let me see vendors successfully handling abuse desk mailboxes, or
mailboxes for sales, abuse, postmaster, and helpdesk at the edge MTA.

I am not really interested in per recipient mail filtering as much as
trying to make sure it doesn't get sent out in the first place.

> on "Why are we still here?"

#3 for me. In the opinions of the people that I know of here who are
handling significatint amounts of mail not destined for their own inbox,
that seems to be true too.

Devdas Bhagat

---

From: der Mouse <[email-masked]>
Date: Thu, 30 Dec 2004 13:17:24 -0500 (EST)

> My question to the group is, if you read all the anti-spam vendors
> (me included), they all claim high-9 catch rates and near-zero false
> positives.  If this is the case, why are we still here [...]?

> 1. The vendor claims are false.  In the real world, you still get
>    lots of undesireable email.

> 2. The vendor claims are true, but we're a bunch of perfectionists
>    searching for infinite resolution of Pi.

>    recipient resources (bandwith, storage, processing, end-user time,

I think there is some truth in each.

(1) is true to an extent.  Some of those claims are, doubtless, utterly
fabricated; some more (probably a lot more) are true but only for a
particular kind of user, kind of site, kind of whatever.  My own
filtering, for example, is probably somewhere around two nines - but to
get that I do some things that some sites Just Can't Do, such as
rejecting HTML out of hand.

(2) is probably true to an extent too, though I think it more likely
that we're rejecting the vendor "solutions" not so much because we're
perfectionists but because we're not interested in proprietary turnkey
"solutions".  I know I'm not - open-source your filters and I may
consider them, but I'm not interested in anything closed-source or
otherwise proprietary.  (If nothing else I simply have no budget - I
can and do pay with my time, but if it takes money it's a nonstarter.)

(3) certainly has truth to it too.  When your pipe is redlined with SYN
packets, it doesn't matter how little processing you do on each one, to
cite just one example.

/~\ The ASCII				der Mouse
\ / Ribbon Campaign
 X  Against HTML	       [email-masked]
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

---

From: Laird Breyer <[email-masked]>
Date: Fri, 31 Dec 2004 13:18:52 +1000

> 1. The vendor claims are false.  In the real world, you still get lots of

This is true. Vendor claims generally mean nothing and tend to be
misleading to sell their wares. There are many issues, such as:

1) Nobody verifies a vendor's claims except anecdotally, because it's too
much work or needs large resources. So you get junk like this instead:

http://www.zdnet.com.au/insight/software/0,39023769,39172027,00.htm

If you read the damn thing, you find that the filters weren't tested, but
rather the reviewers felt it was a priori reasonable to give them all 
comparable good performance marks.

2) A consistent testing methodology that can be applied to several products
of different types does not exist. 

3) Any test represents a point estimate at some point in time and at some
point in the network. This is quickly outdated as spammers evolve fast
(not that fast, but in a matter of months). 

4) There are billions of spam mails circulating each day, how does one
generalize from a test involving a few thousand mails in a lab?

5) Nobody agrees on just what spam is, making 1-4 above irrelevant to some.

> for infinite resolution of Pi.

Some of us are perfectionists ;-) However, I think the main issue is that
the spam umbrella contains a large number of different subproblems for 
different people, and solving one subproblem doesn't solve them all.

There's also no static spam problem, it evolves. So a formerly successful
solution or assumption can become outdated or useless. The virus industry
is a good example. 

> recipient resources (bandwith, storage, processing, end-user time, etc)

I think that's true and false. It is a solution for end-users with
personal computers and fat enough pipes to get the input. 
It is only an ingredient for middle-men or providers. 

I also think that the vast majority of users world wide don't have
effective filtering set up, even though a small subset of power users
already use systems which can filter everything out that you throw at
it. So some people don't actually have a spam problem any more, but 
they are in the minority.

> on "Why are we still here?"

I'm firmly a proponent of Bayesian end-user filtering. I view the
user problem as an information overload problem, which will only grow over
time. This overload will exist even if spam (ie unsolicited bulk mail
using stolen resources) was reduced to irrelevance. 

For example, postal services allow bulk mail for an appropriate fee,
so there's no spamming in the postal system as such. But it's still a
disgusting practice from my point of view, as I have to accept, sift
and throw away legitimate, paid for garbage in shrill colours and big
lettering. 

But rather than bore everyone with one sided arguments, I'm here to
learn more about the non-filtering aspects and help criticise them.

-- 
Laird Breyer.

---

From: James Lick <[email-masked]>
Date: Fri, 31 Dec 2004 14:44:11 +0800

> 1. The vendor claims are false.  In the real world, you still get lots 
> of undesireable email.

The vendor claims are only somewhat accurate.  The only low-cost 
high-accuracy system is challenge-response, but the drawbacks of that 
system have already been discussed ad nauseum.  The most effective spam 
filtering systems are able to get somewhere around 95-99% accuracy if 
the filters are upgraded regularly.  After several months, the accuracy 
of any filtering system decreases as spammers figure out ways to avoid 
them. 

It's very difficult to accurately measure the true false-positive rate 
for various reasons:

- Most people don't ever look at their Spam folder, and for those with 
very high spam levels it is impractical to look through it all.  In 
addition some filters reject a portion of detected spam outright, so 
there's nothing to review.
- People don't miss or don't care that some legitimate opt-in mails get 
tossed, and others do care.
- Most senders won't bother to try to re-contact a recipient if the mail 
is bounced or they never hear back
- If the recipient is told that an email was lost or bounced, it is 
often difficult to impossible to determine what happened to it.

> recipient resources (bandwith, storage, processing, end-user time, etc)

This is the most accurate answer to your question.

The main problem is that the most effective filters are expensive 
relative to the usually slim margins ISPs make off users.  Even "free" 
solutions such as SpamAssassin require you to throw large amounts of CPU 
at it to filter any significant volume of email.  ISPs which do minimal 
filtering and leave it to the end-user to filter end up paying increased 
costs for storage, and less satisfied users.  With the exception of 
DNSBLs, most other filters require the ISP to receive the message to 
determine if it is spam, so additional bandwidth is needed.  These costs 
tend to be trivialized by those with their own personal server, but when 
you have to filter for millions of users, the costs are very significant.

Just to give you an idea, the last two upgrades of my mail server I did 
were not because my users were intentially using more resources, or that 
their wanted mail volume increased dramatically, or that I got a large 
increase in users.  The upgrades were solely because the server could 
not keep up with the volume of unwanted mail.  I could turn off the 
filters but then I'd need to add more storage, and my users would yell 
at me that their e-mail was unusable.  Large ISPs are in the same 
situation but a few orders of magnitude bigger.

Unfortunately the fact that filtering is effective in stopping the 
end-user from seeing most spam also makes it hard for the average person 
to understand the urgency of the problem.  They just see that they 
"only" see a few spams per day, and not the couple hundred that got 
filtered out, or the costs of that filtering.

> I'm jaded here because I just don't get any spam at all. To me, this 
> group is firmly stuck in #2.  But I'd love to hear everyone's 
> feedback/experience on "Why are we still here?"

Because there's still work to be done, and if we stand still the 
spammers will figure out how to defeat current technology, and because 
we don't want to have to keep spending more money on e-mail 
infrastructure to handle spam.

-- 
James Lick -- 黎建溥 -- [email-masked] -- http://jameslick.com/

---

From: "Tom Petch" <[email-masked]>
Date: Fri, 31 Dec 2004 06:52:44 +0100

3 dominates for me
Tom Petch

----- Original Message -----
From: "Peter Kay" <[email-masked]>
To: <[email-masked]>
Sent: Thursday, December 30, 2004 6:12 PM
Subject: [Asrg] Why are we still here?

> 1. The vendor claims are false.  In the real world, you still get lots
of
> 2. The vendor claims are true, but we're a bunch of perfectionists
searching

---

From: Jim Whitescarver <[email-masked]>
Date: Fri, 31 Dec 2004 11:27:06 -0500

Our networks are clogged with spam traffic.
Our servers are overloaded by spam filtering
Spam is increasing exponentially.
Spammers change strategy constantly, what works today, will not work 
tomorrow
What is spam is purely subjective.
Commerce on the net is good not bad.  We are throwing out the baby with 
the bath water.

Perhaps we can filter it well, I have won my war against spam at least 
temporary, but here is a limit on how much we can filter.  I have been 
forced to block over 4 million IP addresses since I cannot handle the load.

Jim
Peter Kay wrote:

> I've been a part of this list for some time now, in mostly lurking 
> fashion. My question to the group is, if you read all the anti-spam 
> vendors (me included), they all claim high-9 catch rates and near-zero 
> false positives. If this is the case, why are we still here trying to 
> eliminate spam when one would think that, if the claims are true, the 
> spam problem has been solved? I can only think of one of three reasons:
>  
>------------------------------------------------------------------------
>

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 31 Dec 2004 22:05:40 +0100

der Mouse wrote:

> I'm not interested in anything closed-source or otherwise
> proprietary.  (If nothing else I simply have no budget - I

Add me.  As a fourth reason I'm just a normal user of some ISPs
doing "something", and that "something" is not 99%.  Or maybe
it is, but then the remaining 1% is still too much for a POP3
user with a modem.  And I refuse to JHD based on the subject 
tag ***SPAM*** inserted by an unknown piece of software where
I don't have write / compile / link / run access.  Bye, Frank