Subject: Re: Will SPF/Unified SPF/SenderID bring down the 'net?

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Tue, 29 Jun 2004 03:42:10 -0700

I agree with greg.

Any spammer trying the proposed mechanism is going to end up causing the
target sysop to come in and see that spf just rejected several hundred
thousand spams.

I can't see how that would stop the sysop using spf, quite the opposite.

I do not have time to analyse arguments in hermeneutic form, nor should this
be necessary given that the original speaker is here.

Doug might think he has made himself clear, but every time I read a post
from him or mathew I see yet another interpretation of the text.

I think it is very obvious that any attempt by the spammers to intimidate in
the manner suggested is going to fail, it fails for the same reason
terrorism fails. The attacker does not have the resources to attack more
than a small number of targets at any given time.

If your macro language offend yee then pluck it out.

There is a separate thread on the factored records issue. I see no reason to
specify anything more than a set of ip addresses with the administrative
flexibiloity that margaret has argued for. It is possible that an argument
could be made for expansion by the username component of the email address.
That is a worthwhile discussion.

The reason I do not want to go down the csv path is that I have yet to see a
clear and concise explanation of the proposal. All I see is fud about other
proposals.

If there is a there there it will be possible to state in a short post the
changes necesary to express csv semantics in spf syntax. 

 -----Original Message-----
From: 	Matthew Elvey [mailto:[email-masked]]
Sent:	Mon Jun 28 23:37:19 2004
To:	Greg Connor
Cc:	'IETF MARID WG'
Subject:	Re: Will SPF/Unified SPF/SenderID bring down the 'net?

Doug just responded rather eloquently to this thread, I'll try my best.

On 6/28/04 10:06 PM, Greg Connor sent forth electrons to convey:

>On Mon, 28 Jun 2004, Matthew Elvey wrote:
>  
>
>>>BLs have a single point of failure that is similar to the problem
>>>of running core DNS, you take down one part of the network and in
>>>time the rest of the net grinds to a halt.
>>>
>>>You have failled to show that there is a dependency that looks anything 
>>>like the dependency that a mail server has on a BL or on core DNS.
>>>
>>>      
>>>
>>What part of "it makes them very resource intensive, so folks stop using 
>>'em" don't you understand?
>>    
>>
>
>
>I am going to agree with Phillip on this one.  SPF queries don't go to a 
>central location, they go to the spammer's DNS server (or that of whoever
the 
>spammer is trying to impersonate.  It sounds like the worst they could do 
>would be to make the receiving mail server really busy, stop their own mail

>from getting through, or pick on one or two other domains that have weak 
>nameservers.  I don't see a type of attack that erodes DNS in general for 
>everybody. 
>
Again, the attacker's immediate goal is just to get folks to stop using SPF!

> Furthermore, the spammer is exposing his own IP during the attack 
>so he should be blocked quickly.
>  
>
Huh? He's got a zombie army, and AGAIN, how do you identify the attack 
as an attack?

>  
>
>>Here's the three paragraphs, with some editing by me.
>>I'm sorry if you don't understand them, but they are clear.
>>It has nothing to to with XML (which is dead, WRT MARID), at least now.
>>
>>Any mechanism introduced that stems the flow of UCE will be subjected to
>>intensive attack.  ...  As the allowable answer from DNS is small, any
chained
>>records further increases vulnerabilities by increasing both resources
>>and time required to process a message.
>>
>>An attacker "jamming" the checking mechanism might set up DNS servers
>>for domains they control that respond erratically and offer complex
>>record sets with small TTLs.  The attacker then sends messages from
>>their domains in an attempt to exhaust resources as a means to have
>>recipients disable the checking processes within the channel.  (If on
>>average a small enterprise uses two outside services, then normally
>>there will be a need to chain these records as it would be prohibitively
>>difficult to administer otherwise. These outside vendors may in turn
>>also outsource for yet more chaining.)         
>>
>>For example, a mail server is receiving 50 messages per second that
>>average 4 K bytes in size.  If using the SPF mechanism, checking DNS
>>data is indeterminate as there is no limit for the number of sequential
>>queries required to converge upon an answer. RFC1035 indicates 5 to 10
>>seconds should be considered a worst case resolver interval.  If there
>>becomes an average of 10 queries with an average of 5 seconds a query,
>>then this limits each process to about 1 message about every minute. 
>>These 10 queries will also add to the traffic at 350 bytes per record a
>>total of 4K bytes of additional traffic for a doubling of the network
>>load.  The mail server may normally handle 1,500 simultaneous processes,
>>but at 60 seconds per process, the mail server is reduced to only
>>running 25 messages a second.  This may still represent the same amount
>>of network traffic, just half as much mail gets through the network.
>>
>>You cannot redefine the size of the emails the attacker sends to make the
attack less effective. 
>>    
>>
>
>
>This seems reasonably clear, but it doesn't identify a damaging attack.  Is

>the attacker trying to get his message through, or just trying to make
trouble 
>for the receiver?
The latter!!! Quoting myself:

>What part of "it makes [SPF] very resource intensive, so folks stop using 
>[SPF]" don't you understand?

>We get a lot of spam (attempts anyway) from domains that don't resolve due
to 
>timeouts. 
Sure, but I bet it's a small fraction.

> That means the spammer is already causing resolvers to time out and 
>mail servers to keep connections open for as long as it takes to time out.

That's generally one query to a possibly non-responding nameserver per 
message, not < 20 or < infinity, .

>So, if the main element of the theoretical attack is "lots of mail sent at 
>once and it makes the resolver do lots of queries that time out" -- I think
we 
>already have the problem today and are dealing with it.  (In several cases
I 
>have had to install DNS servers directly on the mail server box to keep it 
>from bogging down our normal nameserver.)
>  
>
You can deal with it being 100 x worse?

>In other words, there are already a number of DNS queries being done per 
>message, and many of those time out.  5 more or even 20 more DNS queries
are 
>probably not as harmful as, say, increasing the incoming smtp connections,
or 
>consuming SMTP sockets with spoofed SYN packets or something.
>  
>
Possibly.  But these wouldn't achieve the attacker's goal. And only a 
small fraction of  these DNS queries time out.

>Now, I think it's actually interesting to compare this type of attack
scenario 
>with normal spam.  Normal spam may be from a domain that doesn't resolve 
>properly, but if the result is a timeout (for spf or for just resolving the

>MAIL FROM and its MX) then you don't get to go on to the next step.  If the

>DNS responds well enough to keep the connection alive, then the spam is 
>accepted (which consumes more bandwith than the DNS lookups) and sent to 
>SpamAssassin (which consumes CPU, the steps before haven't relied on CPU 
>much).
>
>In other words, it is likely that taking a normal spam run and adding 
>recursive SPF queries that respond erratically and don't cache well, might 
>shift more load onto the resolver, but the more effective it is at weighing

>down the resolver, the more likely the spam will get a 454 answer and not
go 
>on to DATA.  In that case I get my bandwidth back and I get my CPU back, so

>the impact is actually less than a normal spam run, approximately.
>  
>
Well, you get your CPU back, but Doug is arguing that the bandwidth is 
spent on the SPF queries - ~4k/message.
But this is a valid point you make; there is the potential for some win.

>Anyway, if the point of all this is "We should ensure that LMAP queries
aren't 
>allowed to be chained more than X deep" or "We should test to make sure
that 
>complicated SPF queries don't adversely affect the mail server or its dns 
>server" I would agree with that.  I wouldn't describe this as a new attack 
>vector however... anything that starts with "Assume a large number of
incoming 
>SMTP connections..." ought to be familiar territory to any mail server 
>operator :)
>  
>
No.  The second half of the sentence could be something they're 
completely unfamiliar with.  In this case, it may well be.

---

From: Roy Badami <[email-masked]>
Date: Tue, 29 Jun 2004 21:30:31 +0100

>>>>> "Hallam-Baker," == Hallam-Baker, Phillip <[email-masked]> writes:

    Hallam-Baker,> There is a separate thread on the factored records
    Hallam-Baker,> issue. I see no reason to specify anything more
    Hallam-Baker,> than a set of ip addresses with the administrative
    Hallam-Baker,> flexibiloity that margaret has argued for. It is
    Hallam-Baker,> possible that an argument could be made for
    Hallam-Baker,> expansion by the username component of the email
    Hallam-Baker,> address.  That is a worthwhile discussion.

I've seen it argued earlier in the life of SPF that some sites might
not wish to make their complete list of outgoing IP addresses easily
accessible, and that the exists mechanism in SPF (I guess this is what
is meant by a factored record) is of benefit to such sites.  An
attacker who'd comprpomised an ISP's routing could use block records
to rapidly search for domains that list addresses in netblocks the
attacker is able to spoof.  Factored records make that search much
harder.

	 -roy

---

From: Matthew Elvey <[email-masked]>
Date: Tue, 29 Jun 2004 16:38:30 -0700

>attacker is able to spoof.  Factored records make that search much

Yup.  I can't advocate removing the macro abilities of SPF.  Unless we 
make some progress toward showing that Classic SPF with macro abilities 
doesn't create a tempting and relatively vulnerable target, then I see 
CSV as necessary before SPF.  Makes more sense than dropping macro 
abilities.

Philip is saying in essence that the details of an attack scenario and 
of CSV are beyond his comprehension at this time* and he and I appear 
unable to communicate further; each of us feels we have presented our 
case fully, but we are still far apart.  This is unfortunate; I think if 
we dug deeper we might find that SPF can withstand attack as well as 
what's out there now.  Philip's posts to date don't get us there, so I 
think we need to agree to disagree.  I don't mean anything personal; we 
just need to have the kind of discussion one would expect a new crypto 
scheme proposal would engender.

BTW, here's a report of a zombie army of over 100,000 IPs attacking the 
MX host of elvey.com:
http://www.emailaddresses.com/forum/showthread.php?postid=198568#post198568

*e.g. It's not about stack depth.  Also:
"The reason I do not want to go down the csv path is that I have yet to 
see a

clear and concise explanation of the proposal..."
If there is a there there it will be possible to state in a short post the
changes necesary to express csv semantics in spf syntax. "

MOVING FORWARD: 
Here I try to flesh out further how an attack might (not) work.
1) It would make sense to assume the attacker will have most leaf nodes of the SPF records it published be victim nodes, and that some non-leaf nodes could be as well (e.g. include:msn.com)

If we make the (perhaps unwarranted) assumption that SPF checkers rely on local caching DNS resolvers that hold the world's SPF records, then are we safe?  No, with a dynamic SPF record with thousands of leaf nodes, all random macros, no DNS resolver can cache that.  No scheme for determining that the domain or any IPs involved can be blacklisted thereby has been presented or comes to mind.  What if we eliminate macros?  Well, a)I don't think we should but b) does it fix the problem?
Doug's attack scenario made no use of macros, and seems reasonable, so probably the answer is 'no'.

However, let's take a step back.  Before fetching the whole SPF record, the domain will be checked with a reputation service.
Perhaps we say that the first N (N=3?) UDP DNS queries to fetch an SPF record must either fetch the whole record or specify a reputation service that at least gives the domain enough credibility for a fetch of the rest of the record to be warranted.  Thoughts, anyone? 

Perhaps simply the fact that the domain hasn't been blacklisted gives the domain enough credibility for a fetch of the record to be warranted.  Thoughts, anyone? 

A point that I thought was obvious, but some may have missed: if an SPF record's tree can be ten levels deep, it can require thousands of DNS queries to resove.  The number of DNS queries is what needs to be controlled.

PS.  I wrote this post before reading Greg's about limiting posts, which was probably deservedly targeted partly at me. 
I'll bite my tongue for a while.  <Err, just read Meng's post.  I was convinced that if SPF doesn't act as an amplifier, it's fairly safe. Now I realize that even in that case, it still isn't safe.  But neither is CSV.  So why worry?>  Sorry my posts (all to 2 threads) were excessive, and the thread that went downhill.