Subject: SPF abused by spammers

From: Markus Stumpf <[email-masked]>
Date: Thu, 9 Sep 2004 15:18:42 +0200

Justin Murdock posted this link on the qmail list:
    http://news.bbc.co.uk/1/hi/technology/3631350.stm
    "CipherTrust [...] found that 34% more spam is passing SPF checks than
    legitimate e-mail."

	\Maex

-- 
SpaceNet AG            | Joseph-Dollinger-Bogen 14 | Fon: +49 (89) 32356-0
Research & Development |       D-80807 Muenchen    | Fax: +49 (89) 32356-299
"The security, stability and reliability of a computer system is reciprocally
 proportional to the amount of vacuity between the ears of the admin"

---

From: "Chuck Mead" <[email-masked]>
Date: Thu, 9 Sep 2004 10:06:35 -0400 (EDT)

>     "CipherTrust [...] found that 34% more spam is passing SPF checks than

This is good news! Personally I hope it hits 100% as that will make it
sooooooo much easier to whack the moles!

--
[email-masked]
Chief Tech and Bottle Washer

---

From: Dean Anderson <[email-masked]>
Date: Thu, 9 Sep 2004 10:58:34 -0400 (EDT)

Isn't that what I said would happen?

		--Dean

>     "CipherTrust [...] found that 34% more spam is passing SPF checks than

Date: Tue, 10 Aug 2004 19:55:57 -0400 (EDT)
From: Dean Anderson <[email-masked]>
To: 'IETF MARID WG' <[email-masked]>
Subject: Analysis of SPF benefits for reduced filtering

It has been reported that AOL is already using SPF to give reduced 
filtering to SPF-using domains. Is this a good idea?

IF you use SPF to provide less stringent anti-spam processing, then you
are MORE vulnerable than you were before. You have shot yourself in the
foot.  Suppose for example that AOL subjects MSN users to less stringent
anti-spam filtering because MSN uses SPF.  MSN is still vulnerable to
viruses as it was before it used SPF, and it is just as vulnerable to
disposable account creation as it was before.  Using SPF will __attract__
abusers to MSN, because they can get more spam through to AOL, because it
is subject to less processing.  Since AOL is doing less processing on the
same spam, AOL users get more spam. SPF is bad for both companies.

And of course, anyone who sets up a disposable domain can also get spam
through to AOL by creating an SPF record for the domain. Disposable
domains along with disposable or stolen accounts is a major problem now,
and it remains a major problem under SPF.

Anything that reduces spam filtering without reducing the number of
abusers will be harmful.

Basically, SPF gives abusers the opportunity to whitelist themselves, or
the opportunity to identify ISPs that may be whitelisted. Any kind of
whitelist that is under the control of the sender, rather than the
recipient is also going to be ineffective and harmful.

Dean Anderson
Av8 Internet, Inc

---

From: Peter Bowyer <[email-masked]>
Date: Thu, 9 Sep 2004 16:10:49 +0100

> From: Dean Anderson <[email-masked]>

(Sorry if I got the attribution wrong - confusing bad quoting and
top-posting going on)

> filtering to SPF-using domains. Is this a good idea?

On its own - no.

Fortunately, it's not what they're doing - they're using SPF for
exactly what it was intended - as part of their reputation system.
They use SPF to confirm that a mail purporting to come from an
organisation on their whitelist is not forged.

Domain on whitelist + SPF Pass = less filtering.

Peter

---

From: Michael Hammer <[email-masked]>
Date: Thu, 9 Sep 2004 11:28:12 -0400

It might be useful if people checked their facts before using rumor or
secondhand reports as the basis of posting to the lsit.

> filtering to SPF-using domains. Is this a good idea?

Please visit: http://postmaster.aol.com/spf/

AOL has NOT said they will do less stringent checking if you publish
an SPF record. What they have said is that in order to remain on their
whitelist (assuming you meet all of the other requirements) you must
publish SPF records.

I think their website is pretty clear.

Mike

---

From: "william(at)elan.net" <[email-masked]>
Date: Thu, 9 Sep 2004 09:11:05 -0700 (PDT)

> whitelist (assuming you meet all of the other requirements) you must

"A requires B" does not mean "B implies A" i.e. A<B does not lead to B=A

So in order to be on their whitelist AOL will require to have published 
SPF records. However those that published SPF records don't automaticly 
get to be on the AOL whitelist!

-- 
William Leibzon
Elan Networks
[email-masked]

---

From: Patrik Fältström <[email-masked]>
Date: Thu, 9 Sep 2004 18:15:40 +0200

> get to be on the AOL whitelist!

It's pretty sad that people like the author of this article we got a 
link to still see 1:1 mapping between "remove spam" and "minimize the 
number of fake sender addresses". I thought "we" as a collective had 
educated enough people to not get articles like this.

Sigh...

More education needed, else any mechanism which try to stop fake sender 
addresses will fall in the same trap -- and be useless. We need ability 
to track spammers (i.e. more correct data about the sender), someone 
tracking and a legal system which penalizes the originator to show that 
"spam is not ok". All three things. One piece in the puzzle is not 
enough.

    paf

Patrik Fältström <[email-masked]>                         Cisco Systems
Consulting Engineer                   Corporate Consulting Engineering
         PGP: 2DFC AAF6 16F0 F276 7843  2DC1 BC79 51D9 7D25 B8DC

---

From: Markus Stumpf <[email-masked]>
Date: Thu, 9 Sep 2004 20:51:41 +0200

> "spam is not ok". All three things. One piece in the puzzle is not 

IMHO we need something different first.
If in real life you throw 100,000 letters on the street no postman will
collect and deliver them to the recipients. You have to carry them
somewhere like a post office to get them distributed and delivered.
A postman will only accept letters and put them in his bag if they are
from a "official" post office.
The disadvantage of the SMTProtocol is that there is no way for a MTA
(as opposed to MSA and MDA) whether the party putting letters in his
mailbag is a "official" post office or just some random user.

So what we really need is a method for a MTA to tell the difference.
DE has 6.9 millions registered domains and according to a survey of
Peter Koch (it is in the archives of this list) there are about 150,000
unique IP addresses of MX hosts for that domains. With each TLD added
the percentage will decrease, as a lot of MX hosts act for domain pools
like
     example.{com,net,org,ccTLD,...}
In DE the ratio domains to MXhosts is 2.1% and with all existing domains
on the Internet it may drop to 1.5%.
I think it is not an assumption with a large error ration (at least it
is in the same order of magnitude) to assume that the number of sending
IP addresses ~ number of IP addresses of MX hosts.
IMHO that is a data set that is more workable with, even more as the
control is with people that tend to know what they are doing
(i.e. managing a MTA).

So IMHO Mta Authorization Records In Dns (MARID) should have dealt foremost
with the fact to provide the information in DNS if an IP is a mailserver
at all. That way we would have received the number of "mailservers" (or
lookalikes) from 4,000,000,000 to about 600,000?

Yeah I know all the gossip about "marking in rev DNS is bad as rev DNS
is poorly maintained". Now, how much "legitimate" mailservers have
broken revDNS records? And this is all that is important, if the revDNS
for "legitimate" mailservers is also "poorly maintained".

But industry and the big money are all for anti spoofing of domain
names to protect their customers. But as long as the software industry,
on which this solutions depend, produces software like described in
    http://weblog.infoworld.com/udell/2004/03/23.html
this authorization thingy will reduce the success of spoofers and
phishers by exactly 0 percent.

Sure - as someone has written in this thread - spammers (ab)using SPF help
to block their IP addresses, but with spammers running botnets of 100,000s
of infected hosts I don't think they really care about that fact at all.
Rather they sit there having a good time and laughing about all those
heated discussions about patents and methods of authorization they do
not at all care about. At least not for another 5-10 years.

The spam problem is more then 10 years old (Cantor and Siegel was 1994)
and now look how far we have already come in the last 10 years.
Currently my maillog show that the spam problem is worse than ever.

	\Maex

---

From: "Michael R. Brumm" <[email-masked]>
Date: Thu, 9 Sep 2004 14:43:23 -0600

Markus Stumpf wrote:
>   "CipherTrust [...] found that 34% more spam is passing SPF checks
>   than legitimate e-mail."

I suggest everyone else do what I did. I clicked on the "Feedback" link up
at the top and filled out the form as follows.

YOUR COMMENT IS ABOUT: Factual Errors
URL: http://news.bbc.co.uk/1/hi/technology/3631350.stm

YOUR COMMENT:
In your article "Spammers exploit anti-spam trap", you completely missed the
point of SPF. Instead of reporting the use of SPF as a positive, you
portrayed it as a negative.

We WANT spammers to use SPF, because then they are easy to identify. If 100%
of spammers used SPF, I could easily blacklist all e-mail coming from their
domains, and then I'd receive no spam.

Please make a better effort to consult technical experts before making a
report. Several others in the media got the story right (or came close), and
it is very disheartening to see that the BBC (which I rely on for unbiased
reporting) got it completely wrong.

Michael R. Brumm

---

From: Jonathan de Boyne Pollard <[email-masked]>
Date: Thu, 09 Sep 2004 18:48:03 +0100

PF> It's pretty sad that people like the author of this article we
PF> got a link to still see 1:1 mapping between "remove spam" and
PF> "minimize the number of fake sender addresses". I thought "we"
PF> as a collective had educated enough people to not get articles
PF> like this.

Does this "we" include those who are, right now, educating people with 
the following?

"SPF reduces inbound spam."
-- <URL:http://spf.pobox.com./execsumm.html>

"If the message fails SPF tests, it's a forgery.
That's how you can tell it's probably a spammer."
-- <URL:http://spf.pobox.com./faq.html>
(That first sentence is a lie, of course.)

"Eventually, as SMTP improves its immunity to spam,
we hope spammers will get discouraged."
-- <URL:http://spf.pobox.com./faq.html>

"'Why should SPF succeed when similar proposals have failed
in the past?'  The spam problem was never as bad in the past
as it is now."
-- <URL:http://spf.pobox.com./faq.html>

"Saving the world from spam is an expensive duty."
-- <URL:http://spf.pobox.com./donations.html>

---

From: Tony Finch <[email-masked]>
Date: Fri, 10 Sep 2004 13:59:06 +0100

On Thu, 9 Sep 2004, Markus Stumpf wrote:
> at all.

That is what CSV does.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
LOUGH FOYLE TO CARLINGFORD LOUGH: EAST OR SOUTHEAST BECOMING SOUTH OR
SOUTHEAST 4 OR 5, OCCASIONALLY 3 AROUND THE SHORE. SHOWERS LATER. MODERATE OR
GOOD. SLIGHT TO MODERATE.

---

From: Dean Anderson <[email-masked]>
Date: Sat, 11 Sep 2004 18:08:31 -0400 (EDT)

> Domain on whitelist + SPF Pass = less filtering.

This is exactly what I said. And this is a bad idea for the reasons I 
posited.

		--Dean

---

From: "Mark C. Langston" <[email-masked]>
Date: Sat, 11 Sep 2004 15:16:31 -0700

On Sat, Sep 11, 2004 at 06:11:07PM -0400, Dean Anderson wrote:
> 
> Their website is clear. And their use amounts to less stringent checking
> for those using SPF, which is bad for the reasons I stated.  And I might
> add that this explains why spammers have quickly adopted SPF.
> 

Combine beahvior-based reputataion systems with SPF queries, and you
acquire a fairly useful aggregation tool for reputation scores.
In this manner, spammers adopting SPF can be a good thing.

-- 
Mark C. Langston            GOSSiP Project          Sr. Unix SysAdmin
[email-masked]   http://sufficiently-advanced.net    [email-masked]
Systems & Network Admin      Distributed               SETI Institute
http://bitshift.org       E-mail Reputation       http://www.seti.org

---

From: Dean Anderson <[email-masked]>
Date: Sat, 11 Sep 2004 18:11:07 -0400 (EDT)

--Dean

---

From: "Peter Bowyer" <[email-masked]>
Date: Sun, 12 Sep 2004 06:56:25 +0100

Dean Anderson <[email-masked]> wrote:
>> It might be useful if people checked their facts before using rumor
>> or secondhand reports as the basis of posting to the lsit.
>>
>> an SPF record. What they have said is that in order to remain on
>> their whitelist (assuming you meet all of the other requirements)
>> you must publish SPF records.
>>
> Their website is clear. And their use amounts to less stringent
> checking for those using SPF, which is bad for the reasons I stated.
> And I might add that this explains why spammers have quickly adopted
> SPF.

But AOL won't have spammers on their whitelist. The only domains for which
an SPF Pass means less filtering are those which are pre-whitelisted. And
SPF will ensure that mail which appears to be from those domains isn't
forged.

What's your problem with this? It's using SPF for exactly what it was
intended, and the fact that spammers publish SPF will have no effect on it
at all.

Peter

---

From: Dean Anderson <[email-masked]>
Date: Sun, 12 Sep 2004 18:32:17 -0400 (EDT)

> SPF will ensure that mail which appears to be from those domains isn't

This is a false assumption.  Any domain that is whitelisted will attract 
spammers to that domain.  There is no domain or ISP that can claim not to 
have spammers ever.

Whether you use SPF as the reduced testing mechanism or some other
mechanism is irrelevant. The 'bad thing' is to ever subject some mail to
less filtering.

> intended, and the fact that spammers publish SPF will have no effect on it

SPF has no effect on anything except to interfere with legitimate email
unbundling.  SPF allows MSN and AOL to conspire to prevent email
outsourciing, or to charge the outsourcers for the "privelege" of having
an SPF record, or to interfere with such outsourced email by having
unreliable nameservice for the outsourced SPF records.  In exchange, SPF
has no impact on spam, whatsoever.  Thats my problem with SPF.

---

From: "Peter Bowyer" <[email-masked]>
Date: Mon, 13 Sep 2004 06:11:31 +0100

Dean Anderson <[email-masked]> wrote:
>> But AOL won't have spammers on their whitelist. The only domains for
>> which an SPF Pass means less filtering are those which are
>> pre-whitelisted. And SPF will ensure that mail which appears to be
>> from those domains isn't forged.
>
> This is a false assumption.  Any domain that is whitelisted will
> attract spammers to that domain.  There is no domain or ISP that can
> claim not to have spammers ever.

They don't whitelist ISP domains - only responsible bulk email senders.
People whose mail you want your users to receive.

> mechanism is irrelevant. The 'bad thing' is to ever subject some mail
> to less filtering.

Wrong. You submit mail which you've already determined as 'truested' by your
own reputation mechanism.

>> intended, and the fact that spammers publish SPF will have no effect
>> on it at all.

> SPF has no effect on anything except to interfere with legitimate
> email unbundling.  SPF allows MSN and AOL to conspire to prevent email
> outsourciing, or to charge the outsourcers for the "privelege" of
> having an SPF record, or to interfere with such outsourced email by
> having unreliable nameservice for the outsourced SPF records.  In
> exchange, SPF has no impact on spam, whatsoever.  Thats my problem
> with SPF.

Well, you're entitled to your view, but your appreciation of the facts in
the case of AOL is clearly inaccurate. And please remember that SPF is an
authentication framework, not an anti-spam tool.

Peter

---

From: Dean Anderson <[email-masked]>
Date: Mon, 13 Sep 2004 11:43:44 -0400 (EDT)

> People whose mail you want your users to receive.

Yes, such as MSN.

> own reputation mechanism.

This "trust" is a false assumption.  MSN doesn't know whether a particular
message is spam, until it gets a complaint.  It doesn't know whether a
computer is virus infected until it gets a complaint.  It can't certify
that it doesn't have spammers or virus infections or worms or etc.  No one
can. If MSN and AOL make that assertion, and that assertion is accepted by
others, this fact alone will make MSN a target for spammers, viruses,
worms, etc.  This is why reduced filtering promotes more spam.

> authentication framework, not an anti-spam tool.

Likewise, you're entitled to your view, and we may have to agree to
disagree.  But besides your false assumptions of trust, you don't seem to
appreciate the fact that DNS cannot be used for an authentication
framework because it is so trivially and easilly spoofed.  We learned the
vulnerability way, way back with the Morris worm and the BSD R-command
exploits.

		--Dean

---

From: Peter Bowyer <[email-masked]>
Date: Mon, 13 Sep 2004 17:02:47 +0100

> > People whose mail you want your users to receive.

If they have whitelisted MSN, it will be because they want to receive
mail which is verifiably from MSN. It's not a choice I would make
myself, but their network - their rules. SPF has helped them with the
verification.

> can.

True - but that's the basis of all reputation mechanisms - there's no
absolute authority to refer to, you decide whose opinion you trust. If
AOL trusts MSN, that's their business.

> If MSN and AOL make that assertion, and that assertion is accepted by

But AOL's whitelist is for their own consumption, not for others to
rely on. I wouldn't choose to trust AOL's view of what mail to trust
for a whole variety of reasons. To be fair, I don't believe they're
suggesting that anyone should - its reason for existence is to help
AOL themselves to accept email from sources they themselves trust.
What process they go through to establish that trust is their business
- might be a respected validation service, might be a suitcase full of
used notes - that's up to them.

> vulnerability way, way back with the Morris worm and the BSD R-command

Yes, I agree there - but DNS spoofing is sufficiently rare and hard
for the average email forger to do that a mechanism relying on DNS
will have a tiny probability of being subject to forgery in the real
world.

Peter

---

From: Jonathan de Boyne Pollard <[email-masked]>
Date: Mon, 13 Sep 2004 20:54:57 +0000

---

From: Jonathan de Boyne Pollard <[email-masked]>
Date: Thu, 09 Sep 2004 18:48:03 +0100

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 14 Sep 2004 08:40:58 +0200

On Thu, Sep 09, 2004 at 06:48:03PM +0100,
 Jonathan de Boyne Pollard <[email-masked]> wrote 

> Does this "we" include those who are, right now, educating people with

spf.pobox.com contains many marketing promises that
are... careless. (Some people could say "dishonest".) They also keep
pretending that there are "RFCs" about SPF. [email-masked] never
replies on such matters.

Nevertheless, here, in the MARID working group, we can say the truth
:-) SPF (or SenderID) does not assert anything about the spaminess of
the message but about its authenticity (a spam can be authentic).

---

From: Dean Anderson <[email-masked]>
Date: Tue, 14 Sep 2004 12:08:12 -0400 (EDT)

> myself, but their network - their rules. SPF has helped them with the

SPF didn't "verify" anything. SPF just helped AOL paint a target on MSN
for spammers and virus writers.  Not that I feel sorry for MSN, if they
were dumb enough to go along with that.  And of course, the "trust" would 
probably be mutual, so MSN paints a similar target on AOL for spammers and 
viruses.  Mutual self anihilation, I guess. Maybe self-immolation.

> AOL trusts MSN, that's their business.

I agree. Reputation and trust can't be based on false assumptions. You
can't accept promises that you know the other party can't keep, even if
they really wanted to. That's true for just about everything, not just
spam. Trust implies honesty: Honesty about whether you can perform the
promise. If you know you can't perform, yet you promise, you are
dishonest.  How can you possibly trust someone who is dishonest?

> will have a tiny probability of being subject to forgery in the real

It is rare only because no one trusts DNS, and r-commands have all been
turned off. There is nothing to exploit, so there is no point to DNS
spoofing.  However, it is trivial, and it is easily done by the average
forger, if they were motivated to spoof DNS.

DNS spoofing of SPF records can result in the ability to send spam, and
the ability to deny service.  Both bad things.  One may be worse than the
other.

		--Dean

---

From: Dean Anderson <[email-masked]>
Date: Tue, 14 Sep 2004 14:30:01 -0400 (EDT)

> > myself, but their network - their rules. SPF has helped them with the

I also forgot to point out the MSN didn't send email, MSN __users__ sent
email.  We already know they are MSN users from the IP addresses in the
Received headers.  We still don't know which user it was.  Only MSN knows
that, and even then, they don't know if it was due to virus/worm/cracking
until they investigate.

> viruses.  Mutual self anihilation, I guess. Maybe self-immolation.

Sorry to respond to my own message, but there is one more thing that our
proverbial big providers get from this: They get to mutually deny
unbundled email services.  That is, they get to force their customers to
purchase connectivity services from them.  There is much more profit
margin in the connectivity than in the email anyway.  Email is usually
free or low dollars.  That just causes customers to spend more money with
them by abandoning paid-for services that now don't have to be provided.
So what if SPF also brings in more spammers and viruses? Preventing
unbundling could be good for the bottom line for some providers.

		--Dean

---

From: Douglas Otis <[email-masked]>
Date: Tue, 14 Sep 2004 11:33:35 -0700

> the message but about its authenticity (a spam can be authentic).

SPF or Sender-ID validates the MTA as authorized to send messages on
behalf of the mailbox domain.  Neither of these mechanisms authenticate
the message as originating from the specific mailbox domain.  That
assumption of authentication will lead to grievous errors.

-Doug