Subject: Re: [dnsop] I-D ACTION:draft-ietf-dnsop-reflectors-are-evil-00.txt

From: Dean Anderson <[email-masked]>
Date: Mon, 22 May 2006 13:00:53 -0400 (EDT)

In summary, this draft says that we need to be protected from someone who might
scan the internet or a large portion thereof, to find public recursors, and then
perform an attack which can be more easily and more effectively performed on
private recursors, authoritative nameservers, or the roots with the same or
greater multiplication factor. [Somewhat dubiously, a recent discussion on Nanog
asserts that someone, for reasons unknown, did actually construct a large list
public recursors and for a short while, did conduct such an attack using
recursors, even though they could have been far more destructive using the other
variants, not have left an obvious evidence trail in the form of a nameserver
configured with a specially crafted DNS record.]

The draft asserts that we need to be protected from the most obscure variant of
an attack which, if performed with these other types of servers, is more
difficult to mitigate, and is more difficult to detect in preparation, and
leaves no evidence trail in specially crafted large records. The obscure variant
detailed in this draft is of dubious value to any real attack, since the other
variants, which this draft does not address, are essentially the same in
structure, but are more harmful to the target, and more difficult to mitigate,
and more difficult to trace.

More inline.

On Fri, 19 May 2006, Robert Story wrote:

> On Thu, 18 May 2006 16:55:24 -0400 (EDT) Dean wrote:
> DA> So how does limiting public recursive nameservers reduce this attack?
> 
> Because if there were no public recursive nameservers, the attackers
> would have to build a list of authoritative nameservers with large
> records. 

Building a list of high bandwidth authoritative servers is much easier than 
building a list of  public recursors.  

In fact, the attackers better meet their objective with a list of: authoritative
servers with large records, the roots, and/or the private recursive nameservers
used by the target. Particularly useful are those servers operated by the
target's ISP, and upstreams which may also give services for the target.  The
target can't block them without denying themselves nameservice.  Nor can the
block the roots, nor can the root server operators filter out their traffic
without denying nameservice to the target. (This denial would seem to even more
effectively accomplish the attackers' goal. )

Further, searching for public recursive nameservers is much more difficult than
searching for public authoritative nameservers that will send large packets.  
Searching for recursion, like searching for open relays, gives away the identity
of the attacker because they need to collect the results of unusual activity.  
In contrast, requesting your IN-ADDR, SPF, or DNSSEC records is hardly
suspicious.

> (Also, each bot spoofing packets would not only need to know which IP to send
> to, but what record to query for.)

This probably isn't too hard. The large records are SPF, and In-addr, and signed
zones with lots of nameservers or large RRSets.  But the bots already have to
carry a list of thousands of public recursors. BTW, why do you assume its
difficult about adding a field to the bot payload so that it contains nameserver
and query?  As long as the nameservers have suitably high bandwidth, the attack
only depends on the number of queries that can be sent and the size of the
response.  If one cuts the payload in half, but gets the top list of high
bandwidth servers, this is much, much better (for the attacker) than a list of
public recursors, many of which might be low bandwidth, and require an intensive
and suspicious search effort to find.

>  With public recursive nameservers, they just need one authoritative
> nameserver with a large record, and all the public recursive nameservers cache
> that record for the attack. The bots still need an IP to query, but the record
> to query for is fixed.

"You state a disparity where none exists." [Borg Queen to Data] 

Likewise, you to are positing a disparity where no difference exists.  A single
fixed record remains fixed if private recursive nameservers are abused, as well.  
A single authoritative nameserver record remains fixed, also.  The attack
doesn't depend on the number of nameservers found, but on the number of queries
(mainly limited only by the number of bots), size of the response, the bandwidth
of the responding server, and the ability to block those queries or the
responses (mitigation).

Think about mitigation:

- The roots can't block anything for very long, and if you target a block (e.g
the customer block), then root DNS to that whole block would have to be disabled
at the roots. What happens if you have no root nameservice? Total DNS blackout
[well, ironically, this is true only if there aren't any public recursors.  A
conceivable mitigation response would be to shut all my DNS recursion and send
all clients to a public recursor. Joe/Joa]

- Authoritative records can't be blocked without a corresponding denial which
may be worse.  Getting them blocked may be difficult, too.

- My customer probably can't block my recursors without a corresponding denial 
which may be worse.

Mitigation of these requires finding the spoofer, and worse, requires getting 
others to find the spoofer.

In contrast, I can, if necessary block an arbitrary group of nameservers with a
specific, specially crafted record: Netfilter. In the Nanog attack, netfilter 
blocking any DNS responses with "efnet" and a large number of dots.  I can 
filter this myself.

> So yes, a determined attacker could mount an effective attack without public
> recursive nameservers, if they were determined enough to find enough
> authoritative nameservers with large records. 

They don't need _more_ determination to use public nameservers or private
recursors, or roots.  All of these attacks involving other server types are far
___easier___ to mount than scanning for a list of public recursors.  Indeed, I
can detect scanning for public recursors, the same way I detect scanning for
open relays. But I can't infer much suspicious behavior in requests for
authoritive records. Further, if my customer is attacked by many spoofed
requests to my own nameservers, there is not much I can do about it, but try to
stop the spoofing.

Indeed, the suspicious nature of the attack described on Nanog leads one to
conclude that the attack was initiated by people who just think that public
recursors are unnecessary and should be turned off, and would be turned off if
only they "demonstrate the problem".  [we saw this behavior pattern with open
relay abuse, too] The attack described on Nanog was suspicious because:

 - The effects are worse if you use authoritative servers, the roots or the
local private recursor because the effects are harder to mitigate.
 - It is easier to find the other kinds of servers.  No random or exhaustive
scanning is necessary to find authoritative nameservers, roots, or ISP
recursors.  The "big" ones with high bandwidth, that is, the ones most useful
for this kind of attack, are the most well-known domains.
 - And the specially crafted record used in the attack (a long TXT record in a
"defunct domain": Leads back to the person who created the record and the
attack.  Even if the domain is bogus, and actually defunct, the nameserver was
both accessible to, and configured by __someone__; that someone being the
culprit.  Btw, this particular evidence trail is also avoided by abusing the
other server types.

The other attacks all require tracking down the spoofer by the usual methods

> (Which will be really easy once DNSSEC is deployed.) 

Its fairly easy now to find the list of authoritative nameservers. I think CAIDA
or someone runs a query once a year to all authoritive nameservers.

> Of course no matter how many authoritative nameservers they can find to use,
> every public recursive nameserver they can find increases the scale of the
> attack they can mount.

No, this isn't the case.  The attack depends on the number of queries sent
versus the size of the response. It has almost nothing to do with the number of
nameservers abused because one query goes to one nameserver.  Of course, a
nameserver with a great deal of bandwidth is better than a nameserver on a DSL
line.  But if someone, say MSN, creates a large record, game over.

The effectiveness of the attack depends on what can be done to mitigate the 
attack. 

An attack that uses root records can't be mitigated except by tracking down all
the spoofers. Disabling root DNS service is just as bad, if not worse, than the
traffic.

An attack that uses a private recursor also can't be mitigated (except by
tracking down all the spoofers), since that involves turning off the customers
access to DNS.   Suppose someone is spoofing packets for my internet customer 
to several dozen top internet domains?  Suppose somone is spoofing packets to 
the roots?

Oh, and BTW, despite what Joe Abley said, the amplification factor from the
roots is greater than "6 or 7". Without DNSSEC, and without Anycasted roots, it
would be about that.  But with 100+ Anycasted roots, its much greater. With
DNSSEC its much greater still.  It is always interesting that someone would
advocate Anycast performance enhancements while simultaneously forgeting that
those same performance factors don't have any effect on attacks like this.

Oh, and BTW, many root DNS servers have excellent connectivity, and thus make
excellect cooperative hosts for this sort of attack.  In fact, due to this
problem, it may be worthwhile to consider not doing UDP on roots. TCP doesnt'
suffer as badly, since all one would get are a lot of syn acks to non-existant
connections. The multiplication factor is much lower with a TCP version of this
attack.

-- 
Av8 Internet   Prepared to pay a premium for better service?
www.av8.net         faster, more reliable, better service
617 344 9000   

.
dnsop resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/dnsop.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/dnsop/index.html

---

From: John Kristoff <[email-masked]>
Date: Mon, 22 May 2006 14:17:51 -0500

On Mon, 22 May 2006 13:00:53 -0400 (EDT)
Dean Anderson <[email-masked]> wrote:

> Further, searching for public recursive nameservers is much more
> difficult than searching for public authoritative nameservers that
> will send large packets. Searching for recursion, like searching for
> open relays, gives away the identity of the attacker because they
> need to collect the results of unusual activity. In contrast,

Define much more difficult.  I, as well as others, have done some
probing of open recursors.  After recently having probed hundreds of
thousands of servers, in some cases multiple times each in different
ways, only a single admin for a single server sent an email inquiring
about the probes.  Interestingly, this particular admin was reacting
to a log message indicating an abnormal query issued in this case by
fpdns for the purposes of implementation identification.  What he didn't
realize was that this server was an open recursor and that this server
had not long before been seen participating in a spate of amplification
attacks.

Along that note, I've talked to a number of admins from various sized
organizations about their capability to monitor and analyze queries.
Particularly for queries that have occurred in the past, my sense is
that this capability is not widely deployed.

> This probably isn't too hard. The large records are SPF, and In-addr,
> and signed zones with lots of nameservers or large RRSets.  But the
> bots already have to carry a list of thousands of public recursors.

Keep in mind that bots can be updated remotely with what they need.

> They don't need _more_ determination to use public nameservers or
> private recursors, or roots.  All of these attacks involving other
> server types are far ___easier___ to mount than scanning for a list
> of public recursors.  Indeed, I can detect scanning for public
> recursors, the same way I detect scanning for open relays. But I

In practice I think "far ___easier___" is probably of no consequence.

Just curious, did you see my probe to 130.105.14.210 on March 31 of
this year?  I never saw or heard about an email from you, but I'd be
interested to know if you did detect it.  If you prefer, we can
confirm the details offlist to verify.

John
.

---

From: Dean Anderson <[email-masked]>
Date: Mon, 22 May 2006 23:49:19 -0400 (EDT)

On Mon, 22 May 2006, John Kristoff wrote:

> Define much more difficult.  

Umm, you have to find (or create) a authoritive server before you can initiate 
an attack.  Once you have found that server, you can use it via the ISP private 
recursors, which are generally well known, and which a call to the well-known 
help desk number or a perusal of the ISP support site will often reveal.  These 
servers are hard to mitigate.

In contrast, searching for public recursors requires some effort at scanning.  
Just like scanning for open relays.  And attacks are easy to mitigate: one does
not generally do too much harm by shutting off a recursor temporarily. Just like
open relay operations, really.  Perhaps these similar features should also be
included in nameserver software.  Or not. I don't think that it is a likely
attack vector, except to those who (like the no-open-relay folks), expect that
abuse will prove their point.  However, cooperating with them is just like
negotiating with terrorists: it only encourages more bad behavior.  The best
strategy is to ignore them as long as possible, and then take them head on. The 
worst is to give in to demands.

"Much more difficult" can be defined this way: There are multiple "easy outs",
to conduct an attack just as effectively (if not more effectively) before one
gets to scanning for public recursors.

> I, as well as others, have done some probing of open recursors.  After
> recently having probed hundreds of thousands of servers, in some cases
> multiple times each in different ways, only a single admin for a single server
> sent an email inquiring about the probes.

I concede that many people aren't looking at their logs, yet. Emphasis on "yet".  
But some people have logs and are looking at those logs, and others have logs
that are just getting archived.  And yes, perhaps many or most aren't bothering
to log anything.  But there are logs, and there are people watching. CAIDA gets
complaints every year, too, even though they announce their scanning intentions
well in advance, and will not scan anyone who doesn't want to be scanned.

But you have just demonstrated the first reason why real abusers won't do this
scanning: It will generate complaints, and it will also point the finger at the
abusers.  Just like open relay abuse did.

> Keep in mind that bots can be updated remotely with what they need.

Yes, indeed, I suppose they can be. 

> confirm the details offlist to verify.

130.105.14.210 is a customer server. I have nothing to do with its operation.  
If you like, I'll ask them what they are logging.

		--Dean

.