Subject: Re: v6ops-v6onbydefault: link-locals and AI_ADDRCONFIG

From: Keith Moore <[email-masked]>
Date: Mon, 3 Nov 2003 08:36:50 -0500

Having looked at this again, I'm firmly convinced that:

- if address family is set to AF_UNSPEC, getaddrinfo() should by default 
  attempt both IPv4 and IPv6 lookups - even if the local system is not 
  capable of communicating on both IPv4 and IPv6.  it can be useful to do A 
  lookups even if the local system doesn't support IPv4, and it can be useful 
  to do AAAA lookups even if the local system doesn't support IPv6.  the job 
  of an API is to do what the application tells it to do, not to guess what 
  the application needs.  when getaddrinfo() tries to be too clever (as it 
  does on some platforms) it gets in the way and makes it difficult just to
  do simple DNS lookups.

  this implies that a feature to avoid A or AAAA lookups on systems that
  cannot communicate on v4 or v6 (respectively) should be optional, as
  AI_ADDRCONFIG is.

- It's highly desirable to avoid unnecessary DNS queries.  DNS is too
  slow and unreliable for single queries; multiple queries only makes this
  worse.  There are various ways that DNS could be updated to return the same
  information in fewer round-trips, but none of them would obviate the need
  for hosts to do multiple concurrent queries in the near term.

- AI_ADDRCONFIG is an imperfect heuristic, not a reliable indicator of whether
  an address can be used.  that doesn't mean that it's useless, but it does
  mean that people need to be aware of its limitations and not rely on it to
  do magic - in particular, it will not do a perfect job of either avoiding
  unnecessary lookups and connection attempts, or (due to race conditions)
  of finding all usable addresses.

- getaddrinfo SHOULD ONLY DO DNS QUERIES.  LLMNR/mDNS, NIS/NIS+, netinfo, WINS
  and local file lookups only serve to cause divergence between platforms
  and problems for apps that need a consistent view of DNS.

  (Note: I don't expect to get consensus for this view, so I'll offer a 
  less radical view:   Apps whose behavior is defined based on "what DNS
  says",  rather than some other name lookup service, need a portable and
  reliable way to find out "what DNS says" without having to supply their own
  DNS lookup routines.  e.g. maybe there needs to be a AI_DNS_ONLY flag for 
  getaddrinfo?)

- if AI_ADDRCONFIG is set, the decision of whether to do DNS lookups for a
  particular address family should ignore link-local addresses as well as 
  loopback addresses.  This is in keeping with the notion that AI_ADDRCONFIG 
  is an imperfect heuristic.  The configuration of only link-local addresses 
  for a particular address family is a good (though imperfect) indicator 
  that DNS queries for that address family will not yield usable results.

  OTOH, if getaddrinfo() is set up (against my better judgement) to do 
  queries from services other than DNS, the choice of whether to query other
  services might not want to ignore link-local addresses.  For example,
  on a host with only a v4 link-local address configured it might make
  sense to query LLMNR for A records.

- IMHO, getaddrinfo()'s job should be to report what is in DNS, not to try to
  coerce apps into behaving well.  So even though apps shouldn't be using
  link-local addresses, and even though people shouldn't list link-local
  addresses in DNS, getaddrinfo() should not try to hide such addresses from
  apps if they happen to appear in DNS.

Keith

---

From: Stig Venaas <[email-masked]>
Date: Mon, 3 Nov 2003 16:16:26 +0100

[...]
>   apps if they happen to appear in DNS.

I can sympathize with your opinions. I think two things are needed:

o A standard call that most applications can use in a simple way. IMHO it
  should query DNS, other name services, /etc/hosts etc.

o A standard call that applications can use to check what is in DNS

The first should IMHO be something like getaddrinfo() where applications
simply try the addresses in the order they are returned. I also think it
should be possible for an administrator to define which name services
should be used, which type of addresses should be returned, and the order.

Another matter is what the API should be. Most IPv6 enabled applications
today use getaddrinfo(), while specialized DNS applications like "host"
use their own resolver routines. So I would say that getaddrinfo()
without special arguments would be good for the general applications.
For the DNS applications one could use getaddrinfo() with a new option,
or a completely new call. It's easier to change the few DNS applications
that are out there.

Stig

---

From: Keith Moore <[email-masked]>
Date: Mon, 3 Nov 2003 10:42:02 -0500

> >   coerce apps into behaving well.  So even though apps shouldn't be
> >   using link-local addresses, and even though people shouldn't list
> >   link-local addresses in DNS, getaddrinfo() should not try to hide
> >   such addresses from apps if they happen to appear in DNS.
> 
> o A standard call that most applications can use in a simple way. IMHO
>   it should query DNS, other name services, /etc/hosts etc.
> 
> The first should IMHO be something like getaddrinfo() where
> applications simply try the addresses in the order they are returned. 

I don't think that we should be defining getaddrinfo() in terms of
"whatever lookup service happens to be around" because it's very
difficult to get reliable and repeatable behavior that way.  

> I also think it

I disagree.  Different apps have different needs, and no single
host-wide or site-wide policy can accomodate the needs of the variety of
apps in use. Also, apps often cross administrative boundaries, which
creates problems when different nodes of those apps are subjected to the
whims of different administrators.

> Another matter is what the API should be. Most IPv6 enabled
> applications today use getaddrinfo(), while specialized DNS
> applications like "host" use their own resolver routines.

It's not just specalized DNS applications that need to know what really
is in DNS.  Many apps need consistent views of DNS without having to
second-guess the local host API implementation, brain-damaged sysadmins,
etc.

Keith

---

From: Colm MacCarthaigh <[email-masked]>
Date: Mon, 3 Nov 2003 17:06:05 +0000

> difficult to get reliable and repeatable behavior that way.  

Isn't the DNS a lookup service that happens to be around ?

The purpose of getaddrinfo is to perform network address and service
translation. Whilst a portable, granular, interface to DNS would be 
very much welcome - getaddrinfo should not be it.

> apps in use.

A large ammount of apps have common needs; a very common need is to
translate network and service names into numbers. The vast majority
of these apps should not care whether it comes from lookupd/nscd, a 
hosts file, dns, WINS or another source.  

> Also, apps often cross administrative boundaries, which

Indeed, but to remove responsibility from administrators is to remove
power from administrators. Many administrators cherish and understand
their ability to define a search-order, over-ride domains, cache lookups.
Others misunderstand and abuse these facilities, but the primary fault
lies with those administrators - not elsewhere.

Resolving only DNS would even hinder an administrators ability to fix
these problems. Many sites have RFC1918 bogons in their zones (forward
and reverse), some even listed as MX. It can ocasionally be useful to
over-ride such sillyness locally. I'm not saying administrators should
have to work around other's problems, de-incentivising the need to really
fix them, but ocasionally we do.

> is in DNS.

I would argue that really only an application with specialised DNS
functionality (for instance an SMTP implementation) needs to know what
really is in DNS. Certainly more applications than host/dig and so on 
need to know what's really is in DNS, but I don't think the term
"specialised" is unsuited.

A the issue of link-locals in DNS, I agree that they should be returned
by getaddrinfo, for the same reason I cite below. 

> Many apps need consistent views of DNS without having to

Whilst it's extremely desirable that apps not have to deal with API
implementation inconsistency, the application writers sense of priorities
should come much lower than the local administrators IMO.

-- 
Colm MacCárthaigh                        Public Key: [email-masked]

---

From: Colm MacCarthaigh <[email-masked]>
Date: Mon, 3 Nov 2003 22:41:31 +0000

On Mon, Nov 03, 2003 at 04:22:55PM -0500, Keith Moore wrote:
> DNS is a specific lookup service.  not a random one that just might happen
> to be around.

NIS is a specific lookup service, as is WINS, and well I could go on ...
DNS may be more standardised, more widely-deployed and frankly - more
tasteful, but it is no more specific. It's not the only show in town :)

I don't mean to be facetious here, of course I regard DNS as the "standard"
name to number resolver, but I'm a long long way from thinking it should 
be the only means allowed for applications to resolve names.

> Perhaps.  But if getaddrinfo isn't the service that does DNS lookups,
> then at least some apps should not be using it.

Could you give some specific examples of the type of apps you're talking
about?

> That's simply insane.
> Apps (and users) often care if the results from whatever lookup service
> getaddrinfo/gethostname happens to use are not consistent with reality.

When you say "reality", do you really mean - "DNS"? If so, what makes
DNS more authorative than the decisions of the local administrator ? 
What about site to site inconsistencies in DNS ?  

I don't think many people would argue that Global DNS names should be
broken by local administrators. But is making that slightly harder
worth heavily impairing their ability to have site-local name resolution?

> Apps sometimes care if the results of those lookups at different points 
> in the network are not consistent with one another.

Certainly, but since this can (and frequently does) happen with DNS on its 
own, surely the problem should be resolved more generally? I don't see how
the problem of distributed applications needing to agree on a network number
is solved by only using DNS to resolve network names. 

> People are often claiming that apps should not use IP addresses for
> referrals - that they should use DNS names instead.  Indeed, it would 
> be nice for apps to be able to do that, but that would imply (among
> other things) that a name means the same thing from all points of 
> the application - NOT whatever some random lookup service or stale
> host file happens to return to a particular host.

I disagree. And there are many examples of when people consider it useful
for domains to return inconsistent results. Many end-users wish to blackhole
the domains of web-banner advertiser; integral to many testing and staging 
procedures is the ability to spoof production domains; users of RFC1918
space frequently wish to resolve to different addresses depending on scope.  

Whilst all of this is possible with control over a resolving name-server
(assuming there is one), it very much removes power from the administrator
which they find useful.

> Every additional oracle that attempts to provide name to address translation
> is another potential source of variation and error.  Every additional 
> configuration knob that attempts to choose which source of name lookup should
> be used (whether it be an API, a config file, or a DHCP option) is yet
> another knob that can be set wrong.  All of this garbage makes applications
> that ought to be portable from one host to another and one network to another,
> overly sensitive to host and site configuration.  It causes bugs that are often 
> difficult to troubleshoot.

Agreed, however I view the alternative as worse. I've been involved in the
IPv6 development of widely-deployed applications, and have experience
dealing with obsure user problems, extremely tenous getaddrinfo bugs and
other such madness, and I can't think of many such problems occuring. On
the other hand, I've been saved by the ability to quickly over-ride
local name resolution more than once.

> > power from administrators.
> 
> It removes the power to screw things up.  I don't see this as a bad thing.
> 
> Administrators can already set the meanings of DNS names in the zones that
> they control.  Why do they need multiple ways to screw things up ?

Because local administrators arn't always in control of their DNS names.
All that would happen is that more local administrators would run local
DNS resolvers. It just gets you right back to square one :)

> > really is in DNS. 
> You're wrong, of course.

Could you provide an example ? Personally I consider lookup-agnosticism
as a good feature of getaddrinfo. It will take a lot to persuade me 
otherwise.

"really in DNS" and "globally consistent" are not equatable statements, 
because well ... they're not. Unless you're arguing for a global 
short-list of resolvers ?

The problems seen with name-resolsution inconsistency are the result
of stupid operators - they arn't going to go away. They're just going
to be stupid with DNS instead :)

> Quite often local administrators forget that their purpose is to support
> users, that those users need to run apps, and that for the apps to run
> reliably they need a reliable and predictable environment. 

Totally, but it is not for an API to dictate to local administrators how
best to achieve this. Extra lookup methods can be part of providing a
more reliable and predictable environment.

---

From: Keith Moore <[email-masked]>
Date: Mon, 3 Nov 2003 16:22:55 -0500

That's simply insane.

> Many administrators cherish and understand

And many administrators should be terminated with extreme prejudice, too.  

> fix them, but ocasionally we do.

Okay, fine.  Maybe admins need a way to work around occasional problems
that occur.  But we don't need a hodgepodge of different lookup methods
and APIs and configuration mechanisms that interact with each other in 
arbitrary ways that vary from one host or site to another.

You're wrong, of course.

> by getaddrinfo, for the same reason I cite below. 

Well, link-local addresses should never be in DNS in the first place.  
My argument is not that link-local addresses should be returned by 
getaddrinfo() , it is that getaddrinfo() should be transparent.  It 
shouldn't be yet another layer (in addition to all of the ones 
mentioned above) that is getting in the way of the app finding out 
what the DNS name means.

Keith

---

From: Pekka Savola <[email-masked]>
Date: Tue, 4 Nov 2003 08:52:24 +0200 (EET)

> be the only means allowed for applications to resolve names.

My personal experience is that the lookup mechanisms are rarely fully
equivalent/replaceable.  They give different kinds of results.  
/etc/hosts and DNS are (typically) pretty close to each other, but if you
pluck in e.g. WINS, the names it'll give you are typically entirely
different (starting from the format) from those learned from DNS.

I'd expect similar inconsistancies if queries were done using LLMNR or 
ICMP Node Information Queries.

The important point here is who configures the names to be looked up.  
Looking them up e.g. in LDAP would probably yield similar results than
lookup up from the DNS (as they're very probably configured by the same
entity) -- but asking from the node itself what it thinks its name is is
*very* likely to yield different results than one would get asking his
network administrator..

I think typically we have been interested of the latter, that is, what the 
network administrator of a node thinks a node's name is (and NOT what the 
node or its user thinks!)

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Mohacsi Janos <[email-masked]>
Date: Tue, 4 Nov 2003 09:28:17 +0100 (CET)

> then at least some apps should not be using it.

getaddrinfo() and getnameinfo() was developed as a generalisation of
gethostbyname() and gethostbyaddr(). Thus, from pragmatic point of view
should follow the behaviour of the extended functions (gethostbyname etc.)

Basically if you call them you are rely on the several different name
service functions controlled by local configuration (e.g nsswitch.conf
resolv.conf etc.)

> Well, link-local addresses should never be in DNS in the first place.

Yes this is advisable!

> getaddrinfo() , it is that getaddrinfo() should be transparent.

Completely agree. The getaddrinfo() should be completely transparent from
programmers' point of view. The local setup should be system/environment
specific. I think the cleanest solution would be to setup the DNS
resolution order locally by administrator: IPv6 address-resolution enable,
and order of IPv4/IPv6 address-resolution. In my opinion, this should be
included the resolver code and configurable from resolv.conf.

We already have sortlist: IPv6 should be added, and then we could
configure whether we want IPv6 address or IPv4 in the first place. To
disable IPv6 DNS query should be configurable one the _res.options e.g
RES_NO_INET6_QUERY.  The current default is to do IPv6 RR query, so that
one could be disabled.

I strongly against the idea, to change the current default behavior of
IPv6 applications without setting up new things. The compatibility is one
of the most important feature that we have to keep!

Regards,
	Janos Mohacsi

---

From: Keith Moore <[email-masked]>
Date: Tue, 4 Nov 2003 07:35:14 -0500

> included the resolver code and configurable from resolv.conf.

what you are describing is not transparent to applications that actually
need to know what is in DNS.  

Keith