Subject: Re: [Asrg] Please critique my anti-spam system

From: "Michael Kaplan" <[email-masked]>
Date: Sun, 05 Dec 2004 16:04:04 -0500

----- Original Message -----
From: "Bill Cole" <[email-masked]>
To: [email-masked]
Subject: Re: [Asrg] Please critique my anti-spam system
Date: Sun, 5 Dec 2004 14:54:03 -0500

> 
> At 12:53 PM -0500 12/5/04, Michael Kaplan wrote:
> >>  > Billions of spam are sent each day and the problem is getting
> >>  > worse. I believe that this system will profoundly decrease email
> >>  > traffic by
> >>  > eliminating spam.  In time as spam is controlled the number of bounces
> >>  > will decrease.
> >>
> >>  The response of spammers to better blocks and filters has been to send
> >>  more spam. If I want to ensure that 100 user get my spam, and a 90%
> >>  effective filter is commonly used, then I send out 1000 messages. If the
> >>  filter is 99% effective, I send out 10000.
> >
> > This is not a content filter.  If the spammer doesn't have your
> > email address then the spammer can't send you spam.  There is no
> > reason for the spammer to increase the spam load.  Sending spam to a
> > very effective filter is not futile.  Sending spam to a non-existent
> > address is completely futile so the spammer will stop doing it.
> 
> That statement conflicts with available hard evidence. I have a 
> list of just under a dozen addresses under scconsult.com which have 
> not existed for various periods between 6 years and forever, yet 
> which all are offered mail at least daily. I have never accepted 
> mail for bogus addresses in SMTP, so for all of these addresses, 
> all the mail aimed at them while they have been dead (and all mail 
> ever aimed at the ones that never existed) has been refused. For 
> all of those addresses, the rate of mail rejection over the past 18 
> months has risen steadily.

I had imagined a scenario such as what would happen if AOL or some other large provider made this system standard.  Spammers already have a large bounce rate and they still spam, but what would happened if more than 95% percent of the emails sent to AOL addresses started to bounce?  I suspect that at this point there would be a financial incentive for spammers to weed out the invalid addresses.  I have read articles how many big time spammers already routinely weed out invalid addresses, but I am unable to cite these articles off-hand.  I accept that for now I am only able to make reasonable speculation concerning this point.
 
> >>  > The text in these bounces is generic.  An email service 
> >> provider can alter
> >>  > the content of the message to a generic Chinese message if the user
> >>  > wishes.  The instructions for the CAPTCHA can be created in a hundred
> >>  > different languages - the user can decide which one to see.
> >>
> >>  How do I know which language my sender will prefer? Particularly if the
> >>  sender is a new correspondent. If I set my system up to send out such
> >>  bounces in say, Hindi, and you don't understand it at all, how would
> >>  that situation be handled?
> >
> > The text in these bounces is generic. My email provider can
> > recognize one of these generic bounces and substitute the generic
> > Hindi message with an identicle generic English message.
> 
> That is a radical concept. It is not consistent with the way any 
> significant fraction of the real mail system operators deal today 
> with any bounces. You are waving off a very serious problem by 
> hand-waving a profound change in how mail systems are run, in an 
> area where there is a long record of everyone doing their own thing 
> and ignoring standards out of ego and spite.
> 
> >  I'm sure
> > that others can think of other similar ways to handle this situation.
> 
> That's a poor response to a serious critique. If you can't think up 
> a real way to handle a real problem with your proposal, why ask for 
> a critique?
> 
> > Even if this process did not happen then I doubt it would be a big
> > problem.  If your system sends out bounces in Hindi then whoever is
> > trying to correspond with you also likely speaks Hindi.  People who
> > cannot read each others language at all rarely correspond via email.
> 
> That's a remarkably naive response. It's also very much factually 
> incorrect. The Hindi example is a very good one, given the nature 
> of linguistic variety in India.
> 
> Consider this: well over half the spam that makes it past the 
> protections I have in place at the network and SMTP levels is 
> encoded in ways that are not anywhere near downward compatible to 
> US-ASCII. IOW: not only is that mail in a different language than 
> the English I speak, it uses a different alphabet than anything I 
> speak or can usefully piece together from a weak knowledge of a 
> handful of European languages. All of that mail arrives with an 
> envelope sender whose domain part resolves in such a way that I 
> could try to bounce it, but most arrives with a different From 
> header, indicating a strong chance one or the other is forged. If I 
> accept and then bounce all that mail I can't hope to read (some of 
> which could be non-spam from people who expect me to read their 
> language, I suppose...) I would be building bounces in English in 
> response to mail which is not in English, and sending them to 
> addresses that may or may not be the actual senders of the mail and 
> so may or may not understand the bounce and/or the original mail.

You are right, I should be more specific about how the language issue can be resolved.  One common technique used on many websites is to include the image of many different international flags.  If one doesn't speak English then you can click on the flag of your nationality and a window would pop up with translated contents of the bounce.

Every conceivable language would not be covered by this system, but it will likely cover almost everyone who uses email.

If this system became common place then I can see how after a few years a South Korean email service provider could pass on these generic bounces in the Korean language if the user designated in his account preferences that he wanted this to be done.  Having this be done automatically would be a convenience, not a necessity.

The language issue is a valid criticism but it can be significantly mitigated.  If my system is otherwise sound and if it can truly eliminate almost all spam then I doubt that the language issue will be enough to disregard the system.

> Note that after accounting for subscribed mailing lists like this 
> one, the mail that I cannot hope to decipher outnumbers my 
> legitimate mail from unknown strangers by a few orders of 
> magnitude. That is a common situation, and one that  is related to 
> the core flaw in all C/R anti-spam systems: what happens to the 
> spam that arrives is as important as what happens to the non-spam, 
> and what happens to spam splatters innocent third parties.

With my system you would never see any emails sent by strangers lacking legitimate sub-addresses.

There seems to be some concern that this system will effectively 'Joe Job' innocent people.  I don't see the clear incentive for spammers to do this - why not just use a forged address?  I don't personally know anyone who was Joe jobbed.  Is this that common?  Also if spammers know your address then you can solve the problem by activating my system and using a sub-address.

Thank you for your constructive comments.

Michael G. Kaplan
-- 
_______________________________________________
Find what you are looking for with the Lycos Yellow Pages
http://r.lycos.com/r/yp_emailfooter/http://yellowpages.lycos.com/default.asp?SRC=lycos10

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Bill Cole <[email-masked]>
Date: Sun, 5 Dec 2004 16:49:39 -0500

At 4:04 PM -0500 12/5/04, Michael Kaplan  imposed structure on a 
stream of electrons, yielding:
[...]
>There seems to be some concern that this system will effectively
>'Joe Job' innocent people.  I don't see the clear incentive for
>spammers to do this - why not just use a forged address?

As a somewhat tangential note, I think 'joe job' is a bit overbroad 
in this context. Its origin was a very conscious, intentional, 
targeted, and successful attempt to smear a hosting provider who had 
kicked the particular spammer off of his service. That sort of attack 
is pretty rare.

What is very common is simple random forgery of working addresses. 
I'm not convinced that most spammers have thought this through well, 
but at this point I think most of them have figured out that the 
envelope sender's domain had best be resolvable to a plausible path 
for mail delivery, because it is very easy to reject mail without 
that. More recently some unwisely managed mail systems (such as 
Verizon's) have taken up the dangerous practice of blocking inbound 
SMTP sessions while they confirm that the offered envelope sender is 
acceptable to a machine that would have to accept mail for it. This 
sort of ill considered 'verification'  has led to an environment 
where using fully bogus addresses is somewhat less functional than 
using fully functional ones, but the last thing that the low-end 
spammers want to do is provide their own working addresses.

>I don't
>personally know anyone who was Joe jobbed.  Is this that common?

In the random forgery sense, it is extremely common. However, the way 
most forged-sender spam is send and rejected these days pretty well 
hides the extent of that problem from most users, because most of the 
spam that is rejected is rejected in SMTP at the exterior MX for the 
target domain, and only the spammer's own SMTP engine sees the 
rejection, not some real SMTP client that will bounce the rejected 
message. There are still a lot of exceptions to that in absolute 
numbers, but not so much that most people whose addresses have been 
forged will ever know about it.

The idea that ANY email address which is actually used can be kept 
secret from all spammers is fundamentally flawed. As long as people 
are dumb enough to use Windows and other people continue to mail 
them, email addresses will leak to spammers, because the low end of 
the spammer genus has effectively become one with the swarm of 
malware authors who prove daily how bad MS-ware really is.

>Also if spammers know your address then you can solve the problem by
>activating my system and using a sub-address.

There's no need to use a system with an intrinsically abusive C/R 
aspect and/or the text-hostile CAPTCHA model to use tagged addresses. 
For example, I'm on my 4th special address used only for posting to 
this mailing list. The prior 3 all  have had spam (and malware) 
directed at them as a result of what appear to be innocent fools 
reading this mailing list.

-- 
Bill Cole
[email-masked]

---

From: "Mark C. Langston" <[email-masked]>
Date: Sun, 5 Dec 2004 14:04:43 -0800

On Sun, Dec 05, 2004 at 04:04:04PM -0500, Michael Kaplan wrote:
> 
> Every conceivable language would not be covered by this system, but it
> will likely cover almost everyone who uses email.
> 
> 

Not everyone uses HTML-capable MUAs, nor does everyone wish to receive
HTMLized email.  Further, clicking on links in email isn't necessarily a
habit that should be encouraged.

Personally, I think any error (bounce, reject, C/R, etc.) that requires
HTML or subsequent use of a web browser is a non-starter.

-- 
Mark C. Langston                                 The GOSSiP Project
[email-masked]                  http://sufficiently-advanced.net
Factotum                                  Distributed, Peer-to-Peer
http://bitshift.org                        E-mail Reputation System

---

From: Barry Shein <[email-masked]>
Date: Mon, 6 Dec 2004 18:36:20 -0500

> kicked the particular spammer off of his service. That sort of attack 

It's not rare, it's actually a pretty common reaction I see to
aggressively blocking an aggressive spammer, w/in a few hours we see
cascades of joe job traffic I can't help but suspect (tho can't prove)
is related. But there are patterns that would seem to defy total
coincidence like you block an aggressive spammer who uses zombies on
german DSL networks and a couple of hours later you're being hit with
joe jobs w/ all kinds of german fingerprints (e.g., bouncing off of
german ISPs, german language bodies, etc.)

What I will say is:

a) It doesn't go on for very long, "hours" is rare, just a blast or
two is more common (e.g. I guess they send a few thousand joe job msgs
out in one effort and move on.) Or very identical-looking blasts once
or twice a day for a few days like when they think of it they hit the
"joe job 'em" button.

b) It's not that big a deal, really, tho I guess it's a bigger deal to
end-users than companies who have resources. We'll just body pattern
block the joe job and that's pretty much the end of it, or the end of
seeing it.

-- 
        -Barry Shein

Software Tool & Die    | [email-masked]           | http://www.TheWorld.com
Purveyors to the Trade | Voice: 617-739-0202        | Login: 617-739-WRLD
The World              | Public Access Internet     | Since 1989     *oo*