Subject: Re: [Asrg] SPF is only useful to dupe the ignorant...]

From: Fridrik Skulason <[email-masked]>
Date: Sat, 11 Sep 2004 10:08:16 +0000

At 3:43 PM -0400 9/10/04, Barry Shein wrote:
>>Spammers no longer use static domains, and they haven't for years.
>
>Some still do, but they are untouched by SPF anyway.
>
>>Spammers us ZOMBIE PCs.
>
>Yes, and those are quite hard to get into SPF records.

There are two things to consider here.

1) Most compromised machines (and the last time I checked they seemed
to account for 30% of the spam) are personally owned "home" machines,
running Windows, typically with an ADSL or broadband connection.  In
many cases they have dynamic IP addresses, and they simply should not
be sending out mail directly.  Many ISPs block port 25 traffic from those
machines.  However, in theory a spammer could send the mail through
the ISP, just as if it was the user actually hitting the buttons -
on a compromised machine anything is possible.  In those cases it
is possible to get the mail to pass an SPF check.

2) If the ISP allows use of port 25, all the spammer has to do is to
determine which domain the compromised machine "belongs to", and then
send out the mail with a forged sender *in that domain*, as if the
owner of the machine was actually sending the mail.

To the outside (and to anyone checking SPF records), there would not
be a difference bewtween spam sent from the machine and real email
sent by a real human sitting at the keyboard.

SPF is not a "cure-all" - it will help against joe-jobbing in
particular, and (if universally adopted), it will kill off the
current generation of computer worms.  Combined with ISPs blocking
port 25 by default, blacklisting of ISPs that allow spammers to set
up one throwaway domain after another, harsher legal actions against
those using compromised machines for spam and Spamhaus-type
blacklisting of spammers with their own dedicated spam-servers you
would actually see a VERY significant drop in spam.

SPF alone is not going to reduce spam - only change its nature by
getting rid of most joe-jobs (except the "same domain" ones).

--
Fridrik Skulason   Frisk Software International   phone: +354-540-7400
Author of F-PROT   E-mail: [email-masked]       fax:   +354-540-7401

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Bill Cole <[email-masked]>
Date: Sat, 11 Sep 2004 15:35:54 -0400

>running Windows, typically with an ADSL or broadband connection.  In

It looks different to me: more like 80%. That's what I'm seeing 
rejected by the Spamhaus XBL plus a local blacklist made up almost 
entirely of consumer broadband ranges. I have never seen any 1st-hand 
evidence of false positives from either list. However, that 80% does 
not distinguish between spam intended to sell stuff and spam intended 
to spread infection. If you're only looking at spam not matching a 
malware signature, that could explain the discrepancy.

>is possible to get the mail to pass an SPF check.

Yes, but this is a losing tactic. Spammers learned almost a decade 
ago that spamming through the mail servers of consumer ISP's was a 
swift path to disconnection because of how easy it is for an ISP to 
detect and respond to that sort of spamming. There may have been 
regression on the part of ISP's in the last few years in the area of 
keeping their legitimate outbound mail systems from being used to 
spam, but I suspect that Swen has managed to instruct the backsliders.

>sent by a real human sitting at the keyboard.

Right, and this is a serious concern relative to preventing spam from 
being transported.

However, any accurate detection of a domain that will allow a machine 
to pass SPF checks also affixes a truly responsible party to the 
mail. If one publishes an SPF record including a particular IP 
address, one is affirming responsibility for the use of that domain 
name from that IP address.

>current generation of computer worms.

It won't kill Swen. Only the spawning of competent and ethical ISP's 
can do that. Don't hold your breath.

>Combined with ISPs blocking

Some of those actually drive increases, when measured in the wrong ways.

I believe that the excellent Spamhaus SBL and XBL are a part of the 
reason for the explosion in spam over the past year, if you measure 
spam from a point not protected by those lists. Spammers have a long 
history of responding to effective measures by driving more volume, 
and for the XBL especially this only improves the effectiveness and 
creates a cycle of escalating volume. The available mechanisms for 
reliably rejecting spam mean that a competently operated mail system 
can provide users with less spam in their mailboxes today than 3 
years ago, even as the volume of spam being offered up has grown. The 
more the spammers send, the faster they become detectable. As their 
stuff gets detected faster and rejected more, they boost the volume 
more.

>getting rid of most joe-jobs (except the "same domain" ones).

Right. The history of spam evasion is of ongoing identification of 
the various technical gaps in email where the spammers slip through 
and putting traps over them. At some point maybe we'll be able to 
tell who is sending each message easily and reliably enough to 
actually address the social issue of spamming
-- 
Bill Cole
[email-masked]

---

From: Laird Breyer <[email-masked]>
Date: Sun, 12 Sep 2004 12:27:30 +1000

I just want to address the points about zombies:

> spam, but I suspect that Swen has managed to instruct the backsliders.

On the contrary, I believe this is a winning tactic. 

You're talking about historical cases where spammers signed up for an
account themselves, so that when the account was closed, they lost
their net connection, and ability to spam.

The winning tactic is that the spammers are being proxied by
legitimate users unaware of the fact. Who is the ISP going to shut
down? Some grandmother with an infected PC? I claim there is no way
this is going to happen in appreciable numbers.

Take an ISP with 50,000 users. Half or more of those users catch a
virus (not unheard of...) which installs a spamming proxy. In each
case, the proxy detects the ISP mail gateway, and uses it to send spam
on behalf of the user.  Eventually, the ISP detects that half its
customers are sending spam in rolling waves or bursts. None of the
owners of those PCs have any idea what's going on, or would even know
how to fix the problem when it's pointed out.

You are the ISP CEO. Your mission, should you choose to accept it, is
to close down 25,000 user accounts for the crime of spamming, while
keeping your shareholders happy.

> name from that IP address.

In the zombie scenario, the problem isn't who is ultimately
responsible.  The problem is whether the responsible party (the ISP)
can change its behaviour.  How does an ISP change the behaviour of
half its customers? It can't. So how does the ISP stop originating
spam?

It's one thing to pick a handful of customers and talk tough to them
if they don't clean up their PC. It's another thing to talk tough to
more than half your customers, offering them the choice of fixing
their PC, or being disconnected.

So we might soon see large ISPs which 1) acknowledge that a sizeable
fraction of their customers spam unwittingly, and 2) say: so what?

-- 
Laird Breyer.

---

From: Bill Cole <[email-masked]>
Date: Sat, 11 Sep 2004 23:40:25 -0400

At 12:27 PM +1000 9/12/04, Laird Breyer wrote:
>legitimate users unaware of the fact.

I think I neglected to give all the context. There's somewhat more 
than disconnection keeping the spammers from spamming through ISP 
mail servers, there are often rate limiters and similar measures 
making it unworkable.

>Who is the ISP going to shut

It certainly is not happening now, but that situation looks quite 
unsustainable to me and I'm starting to hear rumblings out of the big 
broadband providers that  some most definitely will be doing that in 
the future. They certainly SHOULD be doing it.

>keeping your shareholders happy.

The entire ISP network unable to talk to significant parts of the 
Internet in any way at all because of the problem machines, or an 
outbound mail server that can't function because of the load or can't 
get anyone to accept its mail because it mostly sends spam are all 
worse than dropping half of the customers into a network jail where 
they can't talk to anything (even the ISP's mail servers) other than 
a single webserver that offers them no alternatives other than the 
tools to clean their machines.

This sort of technology is not new. Hotels offering Internet access 
were doing it 4 years ago.

>half its customers? It can't. So how does the ISP stop originating

Step one is to to mature in their understanding of their business. A 
lot of non-ISP's operate networks connected to the Internet that 
generate zero spam. A lot of ISP's who serve markets other than 
consumer residential access manage it as well. There are some 
consumer ISP's managing it. Even AOL comes very close.

>fraction of their customers spam unwittingly, and 2) say: so what?

That's the current situation. It cannot last without all those 
adult-run networks increasingly deciding that accepting any traffic 
other than HTTP requests from mismanaged networks is more trouble 
than it is worth.
-- 
Bill Cole
[email-masked]

---

From: Laird Breyer <[email-masked]>
Date: Sun, 12 Sep 2004 16:15:06 +1000

> mail servers, there are often rate limiters and similar measures 

In the context of zombies, there's the effect on the single user,
and the effect on the spammer.

For the user: with a rate limit, it's a question of who uses up the
allowance first, spamming proxy or human. To the human, the end effect
is probably that their ISP is "unreliable" for email, because it might
reject around half the connection attempts. At best, it's likely to
increase complaints from clueless users to their ISP, or push
customers to rival ISPs with different limits.

For the spammer: with large zombie armies, a rate limit on each user
is not a major problem. Those blackhats who write zombie deployment
infrastructures aren't script kiddies, and can consolidate thousands
of accounts for simultaneous campaigns over 72 hours say, just below
each zombie's spamming limit. It's a matter of having enough zombies,
and zombie creation is subject to completely different rate limits
again (RPC, browser vulnerabilities, etc)

So the important question is probably: will comprehensive ISP
countermeasures be more expensive than the zombie attacks? I have no
idea, but if it's the case, then ISPs will simply learn to live with
the problem, rendering the particular reputation/whitelisting
solutions ineffective as a global force.

> tools to clean their machines.

Interesting idea. You want to set up an automated forced update system
for customers. Presumably, the ISP will pay a yearly fee to the
anti-virus companies to be able to distribute their cleaning software
free of charge to the customers.

I think the complexity of this solution for the ISP is good, because
they only have to handle a few operating system variations, and the
customers hitting each web page are self selecting.

What's the typical delay time between virus discovery and fix, for
major anti-virus companies? That's at least the downtime you'll impose
on each customer, and may be an achilles heel (e.g. Microsoft delay
between patches etc). This could do with some discussion.

> consumer ISP's managing it. Even AOL comes very close.

The whole issue with spam is scale. If spam only affected a few
thousand machines on the internet, virtually any solution would work.
But large consumer ISPs exist because it makes financial sense to offer
cheap access to hordes of people. So the fact that some network operators
manage to be clean when offering (presumably) expensive service to a
small userbase isn't helpful unless their methods scale. AOL however is
a good case study.

> other than HTTP requests from mismanaged networks is more trouble 

I don't know. Email is a killer app, the major reason why people get
an internet account. Dropping email and only offering web browsing is only
a temporary fix, and even webmail is dependent on mail protocols somewhere
along the way. So I'm not sure what this reduction of services would
accomplish for this ISP, over time.

-- 
Laird Breyer.

---

From: Bill Cole <[email-masked]>
Date: Sun, 12 Sep 2004 10:50:17 -0400

At 4:15 PM +1000 9/12/04, Laird Breyer  imposed structure on a stream 
of electrons, yielding:
[...]
>solutions ineffective as a global force.

The expense of tolerating zombies is already having some interesting 
and useful spikes. I am aware of significant corporate networks which 
have taken the approach of shunning major consumer ISP's by ASN (i.e. 
all traffic from all sources on the ISP's network) as a result of the 
mismanagement of those networks. In at least one case, employees 
using those providers are being told quite bluntly the reason for the 
shunning and that there will be no work-around offered to allow them 
to email between work and home or establish VPN links.

It should not take many cases of an ISP being unacceptable to people 
with jobs at Company X before they are effectively unacceptable to 
people with jobs. That's a bad business position for the ISP.

>for customers.

Minor correction: I *want* those people with a proven record of 
infection to be cut off from the net completely and not allowed back 
on by any provider allowing anything more dangerous than WebTV. 
However, I am *aware of* providers choosing to continue their pattern 
of taking the risk of retaining dangerously negligent customers by 
devising cheap ways to provide support in isolating and fixing their 
systems. This is suboptimal, but it is better than the excuse I've 
heard straight from policy-makers at a couple of large 
monopolist-owned ISP's, essentially that they are not willing to fund 
their help desks adequately to allow them to take a phone call from 
every customer they can positively detect to be zombied.

>  Presumably, the ISP will pay a yearly fee to the

Knoppix is free :)

On a more serious note, that presumption speaks to the past, not the 
future. SBC is already giving out McAfee virus software and I would 
be surprised if they are not paying for it or if their competitors 
are not doing the same. Besides that, there are free options for 
cleaning an infected system.

>customers hitting each web page are self selecting.

They only need to handle Windows users for this. Even if there were 
accurately measurable compromise rates for any other systems on 
consumer-grade connections,  in absolute numbers those users are 
easily sacrificed.

>cheap access to hordes of people.

That's a statement of blind faith in the perfect efficiency of the 
market which is not supported by facts or reason. The ISP industry is 
not now and never has been mature, stable, or anywhere close to any 
sort of equilibrium point.

Look at the financial history of the ISP industry and you will see a 
very different reality. Cheap, fast, widely available Internet access 
is the product of charlatans and fools who created, built, and funded 
a lot of catastrophically uneconomic businesses over the past decade. 
Those businesses have momentum of their own driven by the impossible 
dreams and equity investments of fools and the desire of the 
charlatans to ride those dreams and funds as long as they last. It is 
no accident that today there is no major ISP in the US which is not 
either perennially unprofitable and surviving on good money thrown 
after bad or sustained by its owners' other profitable enterprises or 
both. There are no major free-standing ISP's which are not grotesque 
business failures. The continued existence of cheap access is part of 
a larger strategy by companies whose main businesses are  regulated 
traditional monopolies now facing slightly increased competition to 
tie services in ways that discourage customers from switching 
providers of the profitable base service. If you run down the list of 
problem access providers in the US, you will find almost exclusively 
ILEC's and cable operators.

>So the fact that some network operators
>small userbase isn't helpful unless their methods scale.

Their methods only scale when all the competitors are seeking to make 
a profit strictly from Internet access. As long as competitors are 
willing to lose money on providing the cheapest possible Internet 
access, quality providers will remain niche providers.

>AOL however is

You misunderstood me. I'm, not talking about ISP's changing what they 
offer their customers, but about non-ISP's shunning all or nearly all 
traffic from the vast wasteland of consumer broadband. It is 
perfectly feasible and is being done now. In the long run as that is 
done more, it does damage to the value a shunned ISP can offer 
customers. The market will approach equilibrium over time, and it may 
do so by the rest of the net removing value from cheaply-offered 
access.

-- 
Bill Cole
[email-masked]

---

From: Laird Breyer <[email-masked]>
Date: Mon, 13 Sep 2004 11:09:11 +1000

> people with jobs. That's a bad business position for the ISP.

Perhaps. We can only wait and see whether it pans out.

> systems. This is suboptimal, but it is better than the excuse I've 

With the past examples of massive virus infections (going back to code
red etc), I'm just not convinced that the class of people with a
proven record of infections is implicilty small/manageable, and it does
certainly cover both residential and corporate users. 

Your idea of an automatic jail upon spamming detection is good, but
the potential scale of inconvenience to users (whether corporate or
otherwise) means this is a policy decision to be taken high up, I
believe. Or rather, it's a decision which might be vetoed for other business
reasons :-(

> >free of charge to the customers.

Debian testing is, too ;-)

> cleaning an infected system.

If ISPs and IT departments widely implement an automatic
spam->jail->clean system, their main concern will be how long it takes
from detection to cleaning, ie user downtime. (same is true now, but
a spamming system is still useable, mostly, whereas a jailed isn't)

Since viruses and malware evolve precisely to evade anti-virus
services, I think this downtime will relatively often include waiting
for an expert analysis + cleaning fix, ie the particular version of
the installed anti-virus software won't handle the discovered virus
out of the box, but first need an update from the vendor.  

Perhaps you think that's too pessimistic? We've had enough examples of
viruses being changed by a couple of characters in their code to evade
detection, and spyware installing hidden portions of code whose sole
purpose is to detect when the main executable is deleted by an
anti-virus, and to recreate this executable afterwards with a slightly
different signature.

> sort of equilibrium point.

Ok, let's not get into investor motivations etc. You have a point, but
I just wanted to emphasise that consumer ISPs are here to stay,
whatever the reason or mechanics of it. And due to their size, they
are major players, whether we like it or not. And therefore, they
are potential spam attack vectors.

> do so by the rest of the net removing value from cheaply-offered 

Yes, that's possible, but as you pointed out, certain monopolists
can also afford to ignore this projected tide against them if their
ISP related products/business isn't the cash cow.

Secondly, the approached equilibrium itself could be problematic.
What if this equilibrium implies 80% spam in people's inboxes, in
other words spam forever? Wouldn't that be a failure of the
shunning/bullying approach to spam? Conversely, if the equilibrium
settled on 10% spam in people's inboxes say, it would be a relative
success. We just don't know.

-- 
Laird Breyer.