Subject: Re: How long can an NS chain be?

From: "D. J. Bernstein" <[email-masked]>
Date: 5 Jan 1999 17:45:17 -0000

Robert Elz writes:
> If the resolver is too restricted to handle
> chains that occur commonly in practice (ones which other resolvers handle)
> then that resolver will be fixed or replaced.

How exactly is a new implementor supposed to prevent these failures?

Sure, the implementor can test millions of existing domains, but what
happens when people start setting up longer NS chains next year?

The DNS specifications are inadequate. Domains that comply with BIND's
undocumented limits aren't guaranteed to work with new implementations.
Anyone who writes a resolver is threatened with the possibility of
interoperability disasters. Do you want everyone to copy the BIND code?

> This is really similar to the problem of TCP connection establishment.
> All TCPs have a timeout after which they give up and report an error to
> the user (necessary in case the destination host is just down).

RFC 1122 specifies a timeout of 3 minutes by default. This is extremely
conservative. There's no threat for new implementors.

> rules which would be operationally obscene (such as
  [ ... ]
> requiring that servers be always named inside the doamin they serve,
> mandating glue always)

Why would it be ``operationally obscene'' to require the form

   domain IN NS inside.domain

for every NS record?

InterNIC already forces its domains to have glue for secondary servers.
If you have to keep track of an off-site IP address anyway, why not give
it a name within your domain?

---Dan

---

From: Robert Elz <[email-masked]>
Date: Thu, 07 Jan 1999 03:35:06 +1100

Date:        5 Jan 1999 17:45:17 -0000
    From:        "D. J. Bernstein" <[email-masked]>
    Message-ID:  <[email-masked]>

  | How exactly is a new implementor supposed to prevent these failures?

Do you mean the practical failures, or the imaginary theoretical ones?

No reasonable implementation is going to have a problem with any
NS setup that is actually used in practice, and works.   Similarly,
none are going to cope with the worst scenarios that can be dreamed
up as technically possible.

  | The DNS specifications are inadequate.

That may be (in this way and others).   However, right now, no-one is
setting out to rewrite them.

  | Why would it be ``operationally obscene'' to require the form
  |    domain IN NS inside.domain
  | for every NS record?

Because for a large nameserver that means thousands of names with A
records aimed at the address in question, and a similar number of
glue A records in the various parent domains.   An address change
under those circumstances is an enormous task.   Even with the
Internic's unnecessary glue (which is a server operator's private
policy, not a DNS requirement, and which other servers do not require)
there is, in practice, just one A record, and one glue A record, for
all of the domains at a server.  Updating that is a reasonable task
(ie: I, as owner of a name that is in the Internic's zone files as
a glue record can, and have in the past, have them update the associated
A records, which causes the update to apply to all of the domains they have
me listed as a server for).

Further, all of that additional data means that caches everywhere
are storing that (compatively useless) information, perhaps in
preference to more useful data that could otherwise be retained.

It could also almost double the size of all zone files (especially the
big ones with lots of domains delegated to comparatively few servers)
That makes zone transfers more work, and the zone itself require more
resources to manage.

I don't know if you've ever run a large nameserver, but believe me, there
are far more operational problems caused by people who use the above kind
of delegation setup, and then forget to get the glue updated (most often,
never imagine that they're supposed to), than there have ever been from
excessively long NS chains.

kre

---

From: Masataka Ohta <[email-masked]>
Date: Thu, 7 Jan 99 14:54:49 JST

Kre;

> up as technically possible.

Why don't you face the reality?

> setting out to rewrite them.

If glues As not under the referred NS is necessary, the current
specification is not only inadequate but is buggy that new
implementations can not fix it.

Glues MUST be attached to each referral, not to each zone.

Can you give me a real world example or two?

> excessively long NS chains.

??? Are you saying we should have excessively long NS chains?

How long is excessively long?

							Masataka Ohta

---

From: Robert Elz <[email-masked]>
Date: Thu, 07 Jan 1999 18:26:12 +1100

Date:        Thu, 7 Jan 99 14:54:49 JST
    From:        Masataka Ohta <[email-masked]>
    Message-ID:  <[email-masked]>

  | If glues As not under the referred NS is necessary, the current
  | specification is not only inadequate but is buggy that new
  | implementations can not fix it.

I did not dispute that the current spec has some proplems.  What I said was
that right now, no-one is rewiting those specs.   That is, I believe, a
simple fact - if you know of someone who is rewriting 1034/1035 then I think
we'd all like to know about it.

You may also recall that in my first message on this thread that I also
indicated that I thought that an RFC on the general topic of glue would
probably be a good idea.

  | Glues MUST be attached to each referral, not to each zone.

I have no idea where that comes from.   There are many ways that glue can
be handled.

All 1034 says is ...

	To fix this problem, a zone contains "glue" RRs which are not
	part of the authoritative data, and are address RRs for the servers.

Then it says ..

	These RRs are only necessary if the name server's name is "below" the
	cut, and are only used as part of a referral response.

which is the part that is wrong (glue is not "only" necessary in that
case) and in practice, while it would be nice if it were used only in
the referral process, that isn't the way it has worked in practice.

  | Do you mean the practical failures, or the imaginary theoretical ones?
  | Can you give me a real world example or two?

If the org.au zone file (that I manage, though it isn't a huge one, not
even really a large one, just a few thousand delegations) had glue
attached to each delegation it would be almost twice as big.   The nameservers
for most org.au delegations are in .net .com .net.au or .com.au - only a
few of the delegations are to organisations that have their own nameservers
(and hence require glue), a few more are to setups where an secondary name
has been created for a nameserver inside the created domain which would
have had a perfectly good and useable name outside the domain delegated.
(There are 263 glue A records in org.au for 2775 delegated domains which
have 5823 NS records (with 1664 different server names).  (org.au has more
entries that are not delegated, but served directly from the org.au zone
file - mostly just MX records, a few with A records or more).

For com.au, (back in July, which is the last suitable zone file I have lying
around), there were 40569 delegated domains, 92052 NS records, which named 6233
different servers, and 3411 glue A records.   From that you can see that
there's a higher probability that a com.au server will be inside the domain
served than an org.au server, which is not surprising.   Still, only a small
percentage of delegations require glue (and for these, glue is present
everywhere it is required).   Again, there were more domains registered,
but either handled in the com.au zone file directly, or not delegated at all.

I have no access to the COM zone file, counting how much unnecessary glue is
included in that might be a useful endeavour by someone who does.   It would
also be interesting to know whether BIND zone transfers are used to move the
COM zone file around these days (I know that it wasn't once, but don't know
what the current procedures are like).   If it is, it would be interesting to
know how much of the glue that is included survives BIND's sanity procedures.

  | ??? Are you saying we should have excessively long NS chains?

No, I am saying that we don't (in general).   What I said was that there are
far more errors caused by people not updating glue A records when they need to,
than there are problems caused by glue records not being present when they
could be (which would short circuit a possibly long chase from one zone to
another to eventually find some A records to start getting real answers).

  | How long is excessively long?

That was the original question.

kre

---

From: "D. J. Bernstein" <[email-masked]>
Date: 7 Jan 1999 06:03:22 -0000

Robert Elz writes:
> NS setup that is actually used in practice, and works.

Yes or no: Is it ``reasonable'' to limit NS chains to length 5?

  [ blah.com NS ns.blah.com; ns.blah.com A 128.250.1.21 ]
> An address change under those circumstances is an enormous task.

Why? The blah.com nameserver can automatically maintain the ns.blah.com
address as a copy of your munnari.oz.au address. There's no inherent
reason that glue has to be created manually.

If the blah.com sysadmin is automatically maintaining ns.blah.com this
way, why shouldn't he use ns.blah.com as his nameserver instead of
munnari.oz.au?

> are storing that (compatively useless) information,

"blah.com NS ns.blah.com" takes less space than "blah.com NS
munnari.oz.au". Perhaps munnari.oz.au shows up elsewhere in the cache;
but perhaps not. Meanwhile, clients no longer have to chase NS chains.

Have you measured the actual costs and benefits?

---Dan

---

From: Peter Koch <[email-masked]>
Date: Thu, 07 Jan 1999 20:46:04 +0100

> reason that glue has to be created manually.

OK, so if "the" blah.com nameserver can automatically find the address
of munnari.oz.au without manual intervention, why can't everybody else?
And why not hand this out as additional info immediately instead of
creating a new A RR for "ns.blah.com"?

The blah.com servers are not the ones in question here. If your query has
arrived at any server authoritative for "blah.com" you're done. There's
no need for "glue" inside the delegated zone. However, handing out
"additional info" is a fine thing. But if you are going to fake that
dynamically you are running into trouble. Inconsistencies between zones
with identical serial number beeing only one of the issues.
 
> way, why shouldn't he use ns.blah.com as his nameserver instead of

It is an operational hassle to have a single nameserver (process) adressable
by a large number of names. 

> munnari.oz.au". Perhaps munnari.oz.au shows up elsewhere in the cache;

"bla.com NS ns.blah.com", "blah2.com NS ns.blah2.com",
"blah3.com NS ns.blah3.com" together with the three A RR entries do not
only occupy more space, the number of entries needed is 2*n instead of n+1.

I second kre's statement that in the real world DNS far more problems
are caused by well-intentioned but wrong "glue" information than by
"NS chains".

Usually a customer who does not announce a server inside their own zone
(which would require a glue A RR and make you happy) will have (secondary)
service provided by their ISP and some other ISP or DNS provider. While those
servers need not be announced and authoritative for their domain/zone, they'll
live inside it. If you are unable to resolve names in your ISP/DNSP's zones
you loose anyway.

-Peter

---

From: Robert Elz <[email-masked]>
Date: Fri, 08 Jan 1999 05:12:12 +1100

Date:        7 Jan 1999 06:03:22 -0000
    From:        "D. J. Bernstein" <[email-masked]>
    Message-ID:  <[email-masked]>

  | Yes or no: Is it ``reasonable'' to limit NS chains to length 5?

I think it ought be acceptable to limit it to 2.  (That is, a lookup
of the domain requested, leading to a lookup of its servers, which can
mean a chase down from the root in each case - its servers ought to be
directly found though, if its configuration is any good).  But I don't
get to make that decision, the users of the resolver do.   Typically
they won't care about the length of the chain, they'll care about how
long it takes to chase.   It is probably more reasonable to limit a
name lookup to 30 seconds (or something), if you can find it in that time,
great, if not, give up (of course, the user should have someway to set the
limit).

  | Why? The blah.com nameserver can automatically maintain the ns.blah.com
  | address as a copy of your munnari.oz.au address. There's no inherent
  | reason that glue has to be created manually.

Yes, I have often wished for exactly that functionality.   Then I
consider how it would work in .COM   There are something like a million
names listed there (I think, maybe it is more).   Some A records
(including those of nameservers) have TTLs on the order of an hour.
Can you imagine 2 million (or more) queries an hour from each COM
nameserver (2 or more NS's per delegation) just to validate the A
records of the delegated nameservers?  Even if the TTLS were a day,
that's still a pretty high overhead query rate - and that is just
for one zone.

Though you seem to be concerned only with what is in the blah.com zone,
what matters here is what is in COM - if you're suggesting that the
blah.com server should be doing updates of the COM zone, I suspect you
might get come opposition (even if all they can update is the glue).
Both the blah.com and com zones need to monitor the A record as needed.

But in any case, doing what you actually suggested means telling blah.com
(and COM) that the nameserver is really munnari.oz.au, so it knows what
address it should be copying to make the glue for ns.blah.com.    If you're
going to do that you may as well leave the NS being munnari.oz.au, and
have the blah.com server (and the COM server, though it would, in practice
really benefit, as one query of a major nameserver would update the A
for the servers for many domains) find the A (by doing queries), in which
case it wouldn't be glue as such, but cached data, but would be returned in
NS lookups for blah.com just the same as an A record for ns.blah.com
could be (additional section processing is always encouraged).

  | If the blah.com sysadmin is automatically maintaining ns.blah.com this
  | way, why shouldn't he use ns.blah.com as his nameserver instead of
  | munnari.oz.au?

Because the com zone still needs to get the updates, and because it
gains nothing really.    That is, the mechanism you propose to make it
practical works better without the name modification (if it works at all).

  | "blah.com NS ns.blah.com" takes less space than "blah.com NS
  | munnari.oz.au".

In the packets, yes, it does, and that's sometimes an important
consideration.  Usually it doesn't matter - normal NS queries
(including all associated A records) usually easily fit in a DNS
packet.

In caches, perhaps it does, perhaps not.   That depends upon how
the cache is implemented, and what other data it contains.

  | Perhaps munnari.oz.au shows up elsewhere in the cache;
  | but perhaps not.

Yes, but the general assumption here is that munnari.oz.au is a server
for lots of domains (ie: we're talking about an ISPs server generally,
which munnari isn't, but that doesn't matter here).   Typically those
servers are servers for MANY domains - and the chances of one appearing
only once in any cache where size matters is negligible.  That is, if
it appears just once, then there will be very little in the cache, it
won't be big, and no-one will care a lot about its size.  If it's big
(hundreds of Mb or so) then the chances of any of the major ISPs nameservers
appearing as the NS for only one domain cached are pretty small.

  | Meanwhile, clients no longer have to chase NS chains.

That can be avoided by having the A record as additional data in the
NS reply, no matter what the name hapens to be.   The "server name inside
the domain" as a rule is no more than one current technique to more or
less force that to happen.  10 years ago that wasn't needed - A records
were almost always returned with NS records, regardless of the names
involved.   That caused bigger problems than we have now.

  | Have you measured the actual costs and benefits?

Have you?   I am not proposing any changes to anything.

kre

---

From: "D. J. Bernstein" <[email-masked]>
Date: 10 Jan 1999 20:50:42 -0000

I like Dean Anderson's suggestion of documenting what BIND does. The
current situation is highly anti-competitive.

Robert Elz writes:
> It is probably more reasonable to limit a name lookup to 30 seconds

Let's consider

   come.to IN NS ns.greenlight.net
   come.to IN NS black.plant.nl

once again. Suppose that all the servers that know the ns.greenlight.net
address are persistently unreachable.

If the resolver tries looking up ns.greenlight.net, it will spend more
than 30 seconds, and so the original query will fail---even if it is
easy to find the black.plant.nl address and to contact black.plant.nl.

In short, if there's no glue, then your simple strategy has destroyed
the reliability that multiple NS records supposedly provide.

``Use a 25-second time limit on the ns.greenlight.net lookup,'' you say?
This fails if greenlight.net itself has two NS records, the first one of
which is persistently unreachable.

``Look up all the addresses in parallel,'' you say? This can easily lead
to an exponential explosion in the amount of work necessary to look up a
single name. A global limit on the number of queries has the same flaw
as your global 30-second timeout.

  [ NS chain length ]
> I think it ought be acceptable to limit it to 2.

As you know, that makes some existing domains unreachable. What do you
think _today's_ implementors should do?

  [ server automatically copying A record from somewhere ]
> consider how it would work in .COM

InterNIC already has glue for everything in *.com. It's up to the
registrant to notify InterNIC if the glue changes. The necessary forms
generate three orders of magnitude more traffic than a streamlined
protocol would. Obviously InterNIC doesn't care about this cost.

My question still stands. _If_ the sysadmin automatically maintains the
ns.blah.com address as a copy of the munnari.oz.au address, why
shouldn't he use ns.blah.com as his nameserver instead of munnari.oz.au?

When you said this was ``operationally obscene,'' I assumed that you
were thinking of some serious cost for system administrators.

> you may as well leave the NS being munnari.oz.au,

No. The whole point here is to provide glue. You don't want out-of-zone
glue: as you know, if other nameservers provide incorrect glue for
munnari.oz.au, they cause problems for you, not just for themselves.

---Dan

---

From: Robert Elz <[email-masked]>
Date: Mon, 11 Jan 1999 15:24:29 +1100

Date:        10 Jan 1999 20:50:42 -0000
    From:        "D. J. Bernstein" <[email-masked]>
    Message-ID:  <[email-masked]>

  | I like Dean Anderson's suggestion of documenting what BIND does. The
  | current situation is highly anti-competitive.

It already is documented, and easily available.   It just requires
proficiency in reading C rather than proficiency in reading English.
An English (and Dutch, and Spanish, and ...) translation would not
be a terrible thing to do, just don't pretend that any of this is
secret information.

  | ``Look up all the addresses in parallel,'' you say?

If you do "deferred parallel" - ie: try the one you pick first (based upon
previous history of lookups if available), and if that fails after several 
seconds, start on the next (while continuing the non-responding one).  This
has minimum cost in the majority of cases where the servers are available,
and reasonable performance and cost in dealing with the cases where servers
aren't responding.

  | As you know, that makes some existing domains unreachable. What do you
  | think _today's_ implementors should do?

Whatever works.

  | InterNIC already has glue for everything in *.com. It's up to the
  | registrant to notify InterNIC if the glue changes. The necessary forms
  | generate three orders of magnitude more traffic than a streamlined
  | protocol would. Obviously InterNIC doesn't care about this cost.

They don't, because A records change infrequently, and because, in practice
any update of a glue A updates all of the uses of it.   Dynamically verifying
glue would require a query an hour for some A records, and a query a day for
most - rather than the one update a year that is more common for handling
changes.   The DNS TTL isn't designed such that it can be set to long values
(there's no way to revoke an old cached entry).

  | My question still stands. _If_ the sysadmin automatically maintains the
  | ns.blah.com address as a copy of the munnari.oz.au address, why
  | shouldn't he use ns.blah.com as his nameserver instead of munnari.oz.au?

He can, but then he has to handle those updates, and causing the necessary
glue records to be updated - and the com zone admin has to be willing to
handle all the glue noise.    The way most zone admins work now (the big
registries) is they charge an annual fee, and then simply do whatever work
is needed - I have always thought that was silly, and a much better strategy 
would be to charge a much smaller fee, and then charge for each update that
is required, so people that cause the registry no work pay very little, and
those that constantly do updates pay the costs of those.   With such a fee
scheme, your method would impose extra costs.

  | When you said this was ``operationally obscene,'' I assumed that you
  | were thinking of some serious cost for system administrators.

The admins of the big zones (COM etc) were the ones I meant - plus those
who don't just happen to have servers that do the necessary automatic updates
of the glue records, and the necessary automatic requests to update the glue
in the parent zone.   That is, today at least, essentially all of them.

  | No. The whole point here is to provide glue. You don't want out-of-zone
  | glue: as you know, if other nameservers provide incorrect glue for
  | munnari.oz.au, they cause problems for you, not just for themselves.

Glue is glue.    Any glue is an unnecessary cost.   It can't all be avoided, 
but we can minimise the amount of it that needs to appear (and hence be
updated).   It makes no difference at all whether it is "in-zone" or
"out-of-zone" (glue is by definition out of zone, but I know your meaning)
glue if it is correctly updated, and is just as evil if it isn't.

kre

ps: since nothing substantive seems to be being suggested here, I shall not
be sending any more messages on this thread.

---

From: Paul Vixie <[email-masked]>
Date: 31 Jan 1999 00:04:06 -0800

> I concur with these sentiments.  As a new server developer and, as far
> as I can tell, the only in a long time (if ever) to attempt a clean-room
> implementation[1] of the server half RFC-1035,

microsoft's current DNS product is not BIND-based.
likewise software.com's.  and american internet's (now cisco's).
there have also been at least two internal-only implementations
written and used by DNS service providers but not productized.

every one of these people has come forward with a laundry list of
the things 1035 doesn't say and that BIND blazed its own trail for.
(BIND had to be compatible with Jeeves, in the same way and for the
same reasons.)

some of these concerns were addressed in the "DNS Clarifications"
RFC.  more such RFC's are in the works, and more are needed.

> Rather than argue the subject, our plan has always been to establish
> "facts on the ground", as they are called in diplomacy, and once we have a
> significant share of the server market, then to bring our concerns to the
> IETF for resolution, perhaps forcing the issue if need be by implementing
> standards-conformant features which have interesting side effects.

that's what those others (see above) have done.
-- 
Paul Vixie <[email-masked]>

---

From: Andras Salamon <[email-masked]>
Date: Sun, 31 Jan 1999 17:02:59 +0200

On Sun, Jan 31, 1999 at 12:04:06AM -0800, Paul Vixie wrote:
[attribution missing]
>> implementation[1] of the server half RFC-1035,

I would suggest a quick peek at the Name Server Software Summary (NSSS)
    http://www.dns.net/dnsrd/docs/exotic.html
for a fairly long list of DNS server implementations, many of which have
little relation to BIND (though I can't guarantee clean-room coding).
The NSSS has been around since 1994, and even then there were several
non-BIND DNS servers.

> likewise software.com's.  and american internet's (now cisco's).

According to software.com's website their server is a free BIND port,
and there are some indications that it is based on 4.9.5.
    http://www.software.com/products/bindnt.html
Paul, is this incorrect?

> written and used by DNS service providers but not productized.

Contact details would be appreciated so I can include some information
about this in the NSSS.

-- Andras Salamon                   [email-masked]

---

From: Paul A Vixie <[email-masked]>
Date: Sun, 31 Jan 1999 10:22:13 -0800

>     http://www.software.com/products/bindnt.html

I havn't read the web page.  But I've talked to their CTO at the time they
did their rewrite, and since then have talked to one of the lead developers
who did the work.  They may still be shipping something that's BIND-derived
but they absolutely did do a from-scratch non-BIND implementation later on.