Subject: Seven of nine?

From: John Levine <[email-masked]>
Date: 22 Jun 2005 03:19:03 -0000

Let's say I have an RRset like this with a whole lot of
records of the same type:

foo.example.com IN A	10.1.2.3
		IN A	10.1.2.4
		...
		IN A	10.1.2.99

I know that a lot of DNS servers will shuffle the record order around
when they answer queries for low-rent load sharing.  Will they always
return all of the records, or can they return a subset, either to make
the response fit in a UDP packet, or just because?

I'm wondering because applications like SPF will break in mysterious
non-reproducible ways if they get some but not all records in an A or
MX rrset.

R's,
John

--
to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/namedroppers/>

---

From: Mark Andrews <[email-masked]>
Date: Wed, 22 Jun 2005 14:00:23 +1000

> non-reproducible ways if they get some but not all records in an A or

	RFC 2181 Section 5
--
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: wayne <[email-masked]>
Date: Tue, 21 Jun 2005 23:48:11 -0500

>> non-reproducible ways if they get some but not all records in an A or

Interesting that John mentioned SPF, but this subject actually came up
with CSV and the Additional section of SRV records...  Hmmmm...

> 	RFC 2181 Section 5

Thanks, I knew I remembered reading this somewhere, but doing a search
of the RFCs for "RR set" didn't turn up anything.  (The correct search
is "RRset".)

Now, in the http://www.pool.ntp.org project, we have something like
300-400 NTP servers that need to be rotated through.  We do this by
regenerating the zone file periodically with only 15 or so hosts at a
time.  For a while, we had a djbdns secondary NS for the pool, but we
found that it wasn't giving complete RR sets.

Is this a known problem with djbdns?  Are there other name servers
that have this problem?

-wayne

---

From: "John R Levine" <[email-masked]>
Date: 22 Jun 2005 01:56:53 -0400

> > Will they always return all of the records, or can they return a
> > subset, either to make the response fit in a UDP packet, or just
> > because?

> 	RFC 2181 Section 5

Um, we all know about the TC bit.  The question is not whether servers
will set the TC bit if they have to send a response that's too big, it's
whether they'll edit down the rrset before trying to send it.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://iecc.com/johnl, Mayor
"I dropped the toothpaste", said Tom, crestfallenly.

---

From: Mark Andrews <[email-masked]>
Date: Wed, 22 Jun 2005 16:12:49 +1000

> whether they'll edit down the rrset before trying to send it.

5.1. Sending RRs from an RRSet

   A query for a specific (or non-specific) label, class, and type, will
   always return all records in the associated RRSet - whether that be
   one or more RRs.  The response must be marked as "truncated" if the
   entire RRSet will not fit in the response.

	There is a "all" in the first sentence.  Does that answer your
	question?  If TC is not set then the answer is complete.

	Note: MX record processing also depends upon complete RRset
	to be returned.
 
	Mark

---

From: "John R Levine" <[email-masked]>
Date: 22 Jun 2005 02:16:00 -0400

> 	question?  If TC is not set then the answer is complete.

Yes, thanks.

> 	Note: MX record processing also depends upon complete RRset

I don't see why.  If there's a bunch of MXes at the same distance it
wouldn't be a disaster if a client didn't see all of them.

Regards,

---

From: Mark Andrews <[email-masked]>
Date: Wed, 22 Jun 2005 16:35:48 +1000

> wouldn't be a disaster if a client didn't see all of them.

	You just qualified the contents of the MX RRset. I said "MX
	record processing" not "MX record processing when all the MX
	preferences are equal".

	As MX RRsets can and do have different preference values
	you just might want to look at the error conditions that
	are caused when you don't have a complete MX RRset.

	Try running as each of the MX's in turn then drop various
	RR's and see what errors you trigger especially when some
	of the MX's can see a different set of MX records.

		example.net. MX 0 mx1.example.net.
		example.net. MX 2 mx2.example.net.
		example.net. MX 3 mx3.example.net.

	Trust me when I tell you that you will get mail bouncing
	and/or looping depending upon what records are missing
	and which MX's are running.

	Mark

---

From: "John R Levine" <[email-masked]>
Date: 22 Jun 2005 02:40:56 -0400

> 	are caused when you don't have a complete MX RRset.

If the distances are different, sure, all sorts of bad things can happen.
But if the rrset had a bunch of MXes all at the same distance, any subset
should do at the possible cost of delaying mail if all of the servers in
the subset are unavailable but ones dropped from the set are live.

This is all moot since the language is pretty clear that a server can
return the entire rrset or nothing, but not part of it.

Regards,

---

From: Tony Finch <[email-masked]>
Date: Wed, 22 Jun 2005 14:08:15 +0100

> Is this a known problem with djbdns?  Are there other name servers

Our hostmasters tell me that djbdns has a limit of 8 records per RRset.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
BISCAY: WEST 5 OR 6 BECOMING VARIABLE 3 OR 4. SHOWERS AT FIRST. MODERATE OR
GOOD.

---

From: wayne <[email-masked]>
Date: Wed, 22 Jun 2005 09:36:56 -0500

> Our hostmasters tell me that djbdns has a limit of 8 records per RRset.

That matches my recollection of the problems that the NTP Pool project
found.  (e.g. it was about half the correct RRset size.)

If I recall correctly, even though djbdns truncated the RRset, it
didn't set the TC bit.  Can anyone confirm this?

-wayne

---

From: wayne <[email-masked]>
Date: Wed, 22 Jun 2005 10:02:59 -0500

> Our hostmasters tell me that djbdns has a limit of 8 records per RRset.

Also, is giving incomplete RRsets a bug in just djbdns, or is the DJB
caching thing also broken?

-wayne

---

From: Mark Andrews <[email-masked]>
Date: Thu, 23 Jun 2005 08:59:56 +1000

> caching thing also broken?

	RFC's are clear on what should happen.  Report the bug to the
	author.

	Don't hold up development plans because of buggy implementations.
	Having things break is actually the fastest way to get rid of
	buggy implementations.  People will replace them when they are
	not happy.  This applies whether the vendor is Microsoft, us or
	DJB.
 
	Mark

---

From: John Levine <[email-masked]>
Date: 23 Jun 2005 03:12:57 -0000

>> Our hostmasters tell me that djbdns has a limit of 8 records per RRset.

I took a look at the code, and found that tinydns, the server for
authoritative DNS data, has special case code so that if a name has
more than 8 A records, it serves a random set of 8 of them.  I
verified both from examining the code and by experiment that it only
applies to A records.  Yes, it's an impressively bad idea.

>> Also, is giving incomplete RRsets a bug in just djbdns, or is the
>> DJB caching thing also broken?

I don't think so.  The main complaint about dnscache is that it
doesn't report SOAs in the authority section so that clients can't do
negative cacheing.

R's,
John

---

From: "D. J. Bernstein" <[email-masked]>
Date: 23 Jun 2005 20:13:17 -0000

Sending a huge number of A records is pretty close to the worst possible
load-balancing technique. That's why DNS servers have evolved to provide
better load-balancing techniques.

Mark Andrews writes:
> RFC's are clear on what should happen.  Report the bug to the author.

Changing a DNS record set does not violate the DNS protocol. Changing a
DNS record set at high speed does not violate the DNS protocol.

> Don't hold up development plans because of buggy implementations.

If an experimental DNS client can't deal with modern load-balancing
techniques then that DNS client obviously needs to be fixed.

---D. J. Bernstein, Associate Professor, Department of Mathematics,
Statistics, and Computer Science, University of Illinois at Chicago

---

From: Tony Finch <[email-masked]>
Date: Fri, 24 Jun 2005 00:41:37 +0100

> DNS record set at high speed does not violate the DNS protocol.

However it does mean that tinydns can't be used as a secondary for zones
with large RRsets because it will serve the wrong data.

Tony.

---

From: Alan Barrett <[email-masked]>
Date: Fri, 24 Jun 2005 10:32:03 +0200

> DNS record set at high speed does not violate the DNS protocol.

True, provided you change the SOA serial number every time you change
anything else, and provided you never allow different servers to give
different information with the same SOA serial number, and provided you
do not allow the SOA serial number to increment so fast that it wraps
around before the time specified in various other SOA parameters.

--apb (Alan Barrett)

---

From: John Levine <[email-masked]>
Date: 24 Jun 2005 15:21:14 -0000

>Sending a huge number of A records is pretty close to the worst
>possible load-balancing technique. That's why DNS servers have
>evolved to provide better load-balancing techniques.

Quite true.  But it shows an uncharacteristic lack of imagination to
assume that the only reason a client would fetch A records is to
choose a server to connect to.

>techniques then that DNS client obviously needs to be fixed.

I use this program called tcpserver.  One of the things it does as it
sets up a TCP session is to do a rDNS lookup to get a PTR record, and
then it does a forward lookup on the name and checks so see if one of
the A records it gets matches the original IP address.  If that name
has more than eight addresses and is served by tinydns, that check
will fail randomly.  Could you fix it, please?

I realize that one could put a band-aid on this situation by assigning
each IP a unique alias and using that alias as the rDNS PTR, but that
seems like a lot of special case work to avoid a misfeature if I really
do have more than eight servers in a load-sharing group.

R's,
John

-- 
Regards,
John R. Levine, IECC, POB 727, Trumansburg NY 14886 +1 607 330 5711
[email-masked], Mayor, http://johnlevine.com, 
Member, Provisional board, Coalition Against Unsolicited Commercial E-mail

---

From: "D. J. Bernstein" <[email-masked]>
Date: 24 Jun 2005 21:31:44 -0000

John Levine writes:
> each IP a unique alias and using that alias as the rDNS PTR

That isn't a band-aid. It's eliminating a harmful misconfiguration and
replacing it with accurate DNS information. In djbdns terms, you should
run add-host exactly once for each IP address; the documentation talks
about this in detail, and unless you go digging into obscure tinydns
features then add-host won't let you screw up.

Alan Barrett writes:
  [ regarding fast changes in DNS records ]
> provided you change the SOA serial number

AXFR has bigger problems than its difficulties handling fast-changing
records. For example, it has no way to communicate ``foo.bar has address
1.2.3.4 if this client asks, and address 5.6.7.8 if that client asks,''
even though both BIND 9 and tinydns support that feature.

Fortunately, AXFR is optional, and modern DNS-replication tools do a
much better job of handling the new world of DNS features.

---

From: John Levine <[email-masked]>
Date: 24 Jun 2005 22:12:37 -0000

>replacing it with accurate DNS information. 

I don't get it.  Let's say I have a rack with nine machines for my
outgoing SMTP mail.  They're all similarly configured, and I have a
mail generating back end with a whizzo load balancer that doesn't
depend on DNS.

Why, other than working around this DNS server peculiarity, would it
be important that they each have a different name?

---

From: "D. J. Bernstein" <[email-masked]>
Date: 24 Jun 2005 23:05:04 -0000

John Levine writes:
> I don't get it.

Let's say that X University wants to spread its load among 100 hosts.

Here's the misconfiguration, failing to declare the separate hosts:
x.edu A 6.7.8.1; x.edu A 6.7.8.2; etc.; 1.8.7.6.in-addr.arpa PTR x.edu;
2.8.7.6.in-addr.arpa PTR x.edu; etc. Any client connecting to x.edu, or
trying to check the PTR for 73.8.7.6, has to inspect all 100 A records,
even though it cares about only a tiny fraction of the 100 hosts.

Here's the correct configuration: host1.x.edu A 6.7.8.1; host2.x.edu A
6.7.8.2; etc.; 1.8.7.6.in-addr.arpa PTR host1.x.edu; etc.; and a virtual
x.edu selecting a small random set of addresses. The accurate list of
hosts, together with the random selection, means that the client never
has to look at more than a few A records.

---

From: Dean Anderson <[email-masked]>
Date: Fri, 24 Jun 2005 19:26:04 -0400 (EDT)

On 24 Jun 2005, John Levine wrote:

> the A records it gets matches the original IP address. 

The method of looking up reverse IP addresses and checking their forward
names for a match is itself invalid. You should not use this check.  It
will fail in many more cases than you've listed.  It also opens security
holes that can be easily exploited.  I suggest you fix your security 
vulnerablity. That seems more important than RRsets.

		--Dean

-- 
Av8 Internet   Prepared to pay a premium for better service?
www.av8.net         faster, more reliable, better service
617 344 9000

---

From: John Levine <[email-masked]>
Date: 25 Jun 2005 02:39:47 -0000

>x.edu selecting a small random set of addresses.

I can see how you might want a special purpose DNS server to do that
for load sharing, although if I had a hundred servers, I suspect I
would want something more sophisticated than a random number generator
to even out the load.

But I don't see why that makes it a good idea for a nominally general
purpose DNS server to refuse to serve nine A records, or why if it's a
good idea for A records, it's not an equally good idea for MX records.

I don't have 100 servers.  I might have a dozen, and since they're
SMTP clients, nobody's going to use the A record for load sharing
anyway.

R's,
John

PS: If tinydns limits the answer section to eight A records, why
doesn't it do the same thing in the additional section?

$ dnsq mx y.services.net sdn.iecc.com
15 y.services.net:
412 bytes, 1+1+1+21 records, response, authoritative, noerror
query: 15 y.services.net
answer: y.services.net 86400 MX 42 x.services.net
authority: services.net 259200 NS sdn.iecc.com
additional: x.services.net 86400 A 10.0.0.0
additional: x.services.net 86400 A 10.0.0.1
additional: x.services.net 86400 A 10.0.0.10
additional: x.services.net 86400 A 10.0.0.11
additional: x.services.net 86400 A 10.0.0.12
additional: x.services.net 86400 A 10.0.0.13
additional: x.services.net 86400 A 10.0.0.14
additional: x.services.net 86400 A 10.0.0.15
additional: x.services.net 86400 A 10.0.0.16
additional: x.services.net 86400 A 10.0.0.17
additional: x.services.net 86400 A 10.0.0.18
additional: x.services.net 86400 A 10.0.0.19
additional: x.services.net 86400 A 10.0.0.2
additional: x.services.net 86400 A 10.0.0.3
additional: x.services.net 86400 A 10.0.0.4
additional: x.services.net 86400 A 10.0.0.5
additional: x.services.net 86400 A 10.0.0.6
additional: x.services.net 86400 A 10.0.0.7
additional: x.services.net 86400 A 10.0.0.8
additional: x.services.net 86400 A 10.0.0.9
additional: sdn.iecc.com 86400 A 208.31.42.94

---

From: Robert Martin-Legene <[email-masked]>
Date: Tue, 28 Jun 2005 16:47:58 +0200

On 23 Jun 2005 20:13:17 -0000, D. J. Bernstein <[email-masked]> wrote:
> load-balancing technique.

Indeed.

> DNS record set at high speed does not violate the DNS protocol.

I don't think I understand what you specifically mean here. Maybe it's
written between the lines because everyone else seems to have
understood it, but my logic says it's not allowed.

Would be be kind enough to explain it in further detail?

-- Robert Martin-Legène, .dk