Subject: Re: Comments surrounding draft-iab-dns-applications-01

From: Martin Sustrik <[email-masked]>
Date: Mon, 04 Jul 2011 09:42:16 +0200

On 04/29/2011 04:36 AM, John Levine wrote:

> So my advice would be to back up and write down in one or two
> sentences what problem this document is supposed to fix or at least
> describe, and then see how much of the rest of it might be salvaged.

I was thinking of storing data in DNS and the document proved valuable 
to determine whether its a good idea or not. I mean, not whether it is 
technically feasible, but whether it makes sense at all.

So, while the mission statement may be a bit unclear, it's definitely a 
useful document.

Martin

---

From: "John R. Levine" <[email-masked]>
Date: 4 Jul 2011 10:02:15 -0400

> I was thinking of storing data in DNS and the document proved valuable to 
> determine whether its a good idea or not. I mean, not whether it is 

I agree that such a document would be useful, but this isn't it. 
Applications are poorly suited for the DNS if they need fuzzy matches, or 
range queries, or have query patterns that don't cache well.  (IPv6 rDNS 
has that last problem.)  You can argue about how large query and response 
sizes should be, although DNSSEC has forced an answer to that one.

Applications that map fixed query names to (more or less) fixed responses 
work fine on top of the DNS, even if the query doesn't look much like a 
host name and the response looks nothing at all like an A record, e.g., 
NAPTR.

A document that described the actual technical architecture of the DNS, 
without confusing it with the design of applications built on top of the 
DNS would be a good idea.  I suppose I could try and write one.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

---

From: Mark Andrews <[email-masked]>
Date: Tue, 05 Jul 2011 09:14:41 +1000

In message <[email-masked]>, "John R. Levine" wr
> sizes should be, although DNSSEC has forced an answer to that one.

Reverse IPv6 caches well.  You just can't pre-populate servers with PTR
records for all 2^64 ptr records in a normal IPv6 subnet.  You need to
use tools that add records for nodes that actually exist.  Those tools
are a decade old now.
 
> _______________________________________________
> Ietf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: "John R. Levine" <[email-masked]>
Date: 4 Jul 2011 20:23:50 -0400

> use tools that add records for nodes that actually exist.  Those tools

Over in e-mail land, we've been pondering the behavior of spammers, who 
will likely hop to a different IPv6 address for every spam. If you do rDNS 
lookups, your cache will fill up with useless entries, maybe PTR, maybe 
NXDOMAIN, it hardly matters.  DNSBLs and DNSWLs, if done the same way as 
they are in IPv4, have the same problem.  These issues are well known in 
the mail ops community, where it's now the standard advice not to try rDNS 
lookups on incoming IPv6 mail.

Regards,

---

From: Mark Andrews <[email-masked]>
Date: Tue, 05 Jul 2011 10:56:40 +1000

In message <[email-masked]>, "John R. Levine" wri
> lookups on incoming IPv6 mail.
 
Or you just tune the cache retention times.  For NXDOMAIN/NODATA
that's 3 hours by default for named but could be tuned down to 10
minutes or lower without ill effects.  RFC 2308 recommends 1-3 hours.

I also don't see the point in worrying about this.  Caches cope
with spammers using a different From domains on each piece of email
which is looked up in the DNS.  The worst using a different IPv6
address per email can do is double the cache requirements for the
same volume of email.

LRU cleaning of the cache will cope with this.

Mark

---

From: "John R. Levine" <[email-masked]>
Date: 4 Jul 2011 22:05:33 -0400

> minutes or lower without ill effects.  RFC 2308 recommends 1-3 hours.

DNSBLs already set the min pretty low, e.g. 150 sec for Spamhaus. 
Doesn't really matter how low it is if you have so many entries that they 
force out the useful ones.

> which is looked up in the DNS.

Modern spam filters don't usually look up the author domain, since it's 
usually a genuine address taken from the spam list so it's unrelated to 
the real sender.  Even if they did, universe of domains that exist is a 
vastly smaller set than even IPv4 addresses, and one which caches pretty 
well since so many of them are at large sites like Yahoo and Hotmail.

In any event, we can argue about how good or bad an idea it is to use IPv6 
rDNS, but that's tangential to the issues of deciding what are reasonable 
applications for the DNS.  If you're right and rDNS caches well, it's a 
good application for DNS.  If I'm right and it doesn't cache at all, it's 
not such a good application.

Regards,

---

From: Keith Moore <[email-masked]>
Date: Mon, 4 Jul 2011 23:01:44 -0400

On Jul 4, 2011, at 8:23 PM, John R. Levine wrote:

> Over in e-mail land, we've been pondering the behavior of spammers, who will likely hop to a different IPv6 address for every spam. If you do rDNS lookups, your cache will fill up with useless entries, maybe PTR, maybe NXDOMAIN, it hardly matters.  DNSBLs and DNSWLs, if done the same way as they are in IPv4, have the same problem.  These issues are well known in the mail ops community, where it's now the standard advice not to try rDNS lookups on incoming IPv6 mail.

Yes, but rDNS PTR lookups always have been pretty much meaningless anyway, and will only get worse in IPv4 due to LSN. 

Keith

---

From: Mark Andrews <[email-masked]>
Date: Tue, 05 Jul 2011 13:05:37 +1000

In message <[email-masked]>, "John R. Levine" wri
> force out the useful ones.

DNS[WB]L's are not rDNS.  rDNS is designed to associate a name with
a address.

DNS[WB]L's have never been a good fit to the DNS, rather they have
not been a bad enough fit to require something better to be done.

The naive approach of reversing the address, converting to nibbles
and appending a suffix won't scale.

For IPv6 if you did the reverse of /48, /52, /56, /60 and /64
prefixes, which matches delegation patterns along with NXDOMAIN
synthesis, you would still be fine.  You stop the search on NXDOMAIN
or data with perhaps a new value which says to continue searching
for white listed records.  One could even start with /32 if one is
worried about spammers pretending to be ISPs.

It requires slightly more intelligence in the MTA that faking up a
address lookup.  NXDOMAIN synthesis doesn't have to be done in the
DNS cache.  It can be done in the application provided it keeps
track of the ranges covered by the NSEC{3} records returned with
the negative responses.  DNSSEC validators that support DLV do this
sort of thing.

> well since so many of them are at large sites like Yahoo and Hotmail.

Both are effectively infinite compared to cache sizes.

---

From: "John R. Levine" <[email-masked]>
Date: 4 Jul 2011 23:31:51 -0400

> worried about spammers pretending to be ISPs.

I don't necessarily disagree, but now you've just upgraded the DNS 
(NXDOMAIN synthesis is far from universal) and layered a probing protocol 
on top of it.  We can have a theological argument about whether that 
counts as "using the DNS".

R's,
John

---

From: Mark Andrews <[email-masked]>
Date: Tue, 05 Jul 2011 16:24:43 +1000

In message <[email-masked]>, "John R. Levine" wri
tes:
> (NXDOMAIN synthesis is far from universal)

No, just application smarts is needed.  The DNS already supports this
and it has been used for over 1/2 a decade now.

> and layered a probing protocol on top of it.  We can have a theological
> argument about whether that counts as "using the DNS".
>
> R's,
> John

---

From: Tony Finch <[email-masked]>
Date: Tue, 5 Jul 2011 11:52:34 +0100

John R. Levine <[email-masked]> wrote:
>
> DNSBLs already set the min pretty low, e.g. 150 sec for Spamhaus. Doesn't
> really matter how low it is if you have so many entries that they force out
> the useful ones.

The really important records are the delegation chain NS records. It is
not so necessary to keep leaf records in the cache.

http://nms.csail.mit.edu/projects/dns/

> usually a genuine address taken from the spam list so it's unrelated to the
> real sender.

I think it's still useful to reject mail with an unresolvable return path
domain.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Lundy, Fastnet: Southwest 4 or 5, increasing 6 or 7 later. Moderate or rough,
occasionally very rough later in Fastnet. Showers. Good.

---

From: Tony Finch <[email-masked]>
Date: Tue, 5 Jul 2011 11:56:24 +0100

> not been a bad enough fit to require something better to be done.

Oh come off it, they are just as good a fit to the DNS as reverse DNS is.

> and appending a suffix won't scale.

I don't understand why the setup is OK for reverse DNS but not blacklists.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Hebrides, Bailey, Fair Isle, Faeroes: Southeast 5 to 7, backing east or
northeast 4 or 5, occasionally 6 in north Bailey. Moderate or rough.
Occasional rain. Moderate or good, occasionally poor.

---

From: Tony Finch <[email-masked]>
Date: Tue, 5 Jul 2011 11:58:32 +0100

Keith Moore <[email-masked]> wrote:
>
> Yes, but rDNS PTR lookups always have been pretty much meaningless
> anyway, and will only get worse in IPv4 due to LSN.

Reverse DNS for an LSN is just as informative as automatically populated
reverse DNS (and much more convenient than a whois lookup).

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Portland, Plymouth, Biscay: South 4 or 5, veering west or southwest 5 or 6.
Slight or moderate, occasionally rough later. Rain then showers. Moderate or
good.

---

From: Keith Moore <[email-masked]>
Date: Tue, 5 Jul 2011 08:38:39 -0400

> reverse DNS (and much more convenient than a whois lookup).

correct, in that both are meaningless.   but I think the percentage of lookups that return meaningful information will decrease - though more due to address space exhaustion than LSN, I suppose.

what LSN will decrease is the correlation between source IP address and source host, making any effort to associate a reputation with said addresses less valid.

Keith

---

From: "John R. Levine" <[email-masked]>
Date: 5 Jul 2011 11:06:20 -0400

> I don't understand why the setup is OK for reverse DNS but not blacklists.

One obvious reason is that the most widely used DNSBL server doesn't 
return NODATA vs. NXDOMAIN according to current standards, so probing and 
NXDOMAIN synthesis doesn't work.  I'm planning to fix it, but it'll take a 
while.

Regards,

---

From: Dave CROCKER <[email-masked]>
Date: Tue, 05 Jul 2011 09:30:07 -0700

On 7/5/2011 3:56 AM, Tony Finch wrote:
> Mark Andrews<[email-masked]>  wrote:

+1

This has been an on-going issue, with a kind of 'purity' argument from some 
folk. While concern for the stability of end-to-end DNS operations certainly is 
essential, the degree of resistance to these added uses of the service often is, 
indeed, inconsistent.

In general, the problematic logic requires a devotion to the narrow usage that 
has dominated the DNS, rather than what the design of DNS was intended to 
support.  (Reading the original RFCs is instructive for this broader view.)

However for the current thread, there really does appear to be an impending 
problem.  It seems pretty clear that DNS caching needs to be tailored to the 
types of applications making the queries and that mixed traffic could well mess 
with caching behavior.  (I think it uncontroversial to note that that having DNS 
caching functioning well is an important requirement for stable and efficient 
DNS use.)

For an application that is likely to encounter a different IP address for 
essentially every query, across a very large number of queries, the only 
solution I see available is to use a different cache.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: "John Levine" <[email-masked]>
Date: 5 Jul 2011 17:31:44 -0000

>solution I see available is to use a different cache.

Seems reasonable.  I gather that there are already caches with the
ability to partition themselves so that records from different
subtrees compete for different pools of cache entries.

I'm also sort of surprised that we don't seem to have all that much
experimental data about cache behavior.  The MIT papers that Tony
cited are interesting, but they're also ten years old.

R's,
John

---

From: Dave CROCKER <[email-masked]>
Date: Tue, 05 Jul 2011 10:41:03 -0700

> cited are interesting, but they're also ten years old.

I doubt that partitioning according to DNS structure will work for this, since 
the DNS has no formal semantics to different parts.  Hard-coding knowledge of 
particular tree segments can work for specific examples, in small scale, but 
won't work for the constantly-changing IP Address behavior cited in this thread.

I believe that the solution is to have the applications, themselves, distinguish 
the cache they are using (or the containing library).  A blocklist app needs to 
use a different library/cache than a web browser.

d/

-- 

   Dave Crocker

---

From: Tony Finch <[email-masked]>
Date: Tue, 5 Jul 2011 18:59:27 +0100

> solution I see available is to use a different cache.

This has been operational common sense for many years.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Trafalgar: Northerly 5 or 6 in southeast, otherwise variable mainly westerly
veering northerly, 3 or 4. Moderate or rough. Fair. Moderate or good.

---

From: Dave CROCKER <[email-masked]>
Date: Tue, 05 Jul 2011 11:10:32 -0700

On 7/5/2011 10:59 AM, Tony Finch wrote:
> Dave CROCKER<[email-masked]>  wrote:
>>

In theory, practice is not much different from theory.  In practice...

    <http://thinkexist.com/quotation/common_sense_is_very/193303.html>

d/

-- 

   Dave Crocker

---

From: "John Levine" <[email-masked]>
Date: 5 Jul 2011 18:11:33 -0000

>I believe that the solution is to have the applications, themselves,
>distinguish the cache they are using (or the containing library).  A
>blocklist app needs to use a different library/cache than a web
>browser.

You could do it either way.  Either you could adjust the MTAs to have
a new parameter for the address of the cache to use for DNSBLs vs. the
cache to use for MX, A, and AAAA, or you could tell the cache about
the list of DNSBLs that the local MTAs use (a list which in my
experience is rarely very long.)

Pick your poison, depends which software is older and cruddier.

Or there's the pessimal approach, add a new EDNS parameter so the MTA
can tell the cache what kind of lookup it's making, so you have to
upgrade and debug both of them!

R's,
John