Subject: Re: Issue: Add a new QTYPE

From: Kevin Darcy <[email-masked]>
Date: Mon, 24 Nov 2003 22:36:32 -0500

Paul Vixie wrote:

>>... IMO this raises the unreliability of the QTYPE to a whole new
>>level, and I think it's why QTYPE=* has been abandoned for the most
>>part (e.g. by sendmail), even in situations where it could be a useful
>>optimization.
>>    
>>
>
>sendmail was incorrect in its usage of QTYPE=* and when they "abandoned"
>it, it was to fix the bug in their code where they had used it at all.
>
>  
>
>>>... it wasn't resource/capacity, but rather complexity, which took
>>>this functionality off the table.  see http://sa.vix.com/~vixie/edns1.txt
>>>and the working group minutes from DNSIND.
>>>      
>>>
>>I did dig back through the archives, and saw a message from you
>><http://ops.ietf.org/lists/namedroppers/namedroppers.2002/msg01448.html>
>>admitting that QTYPE=* wasn't really germane ("it was a debugging wish
>>rather than an operational expectation") to the RRD-flag part of the EDNS1
>>proposal, and a statement of intention that you were going to remove that
>>reference from the draft. There was, however, no subsequent published
>>revision of the draft, so the verbiage still remains.
>>    
>>
>
>the draft is expired.  the DNSIND working group decided to scrap everything
>therein.  my offer to pull out RRD was insufficient; the wg wanted the whole
>thing killed.
>
>  
>
>>As far as I can tell, there has been no attempt to clarify QTYPE=*
>>recursivity independently of the EDNS1 proposal.
>>    
>>
>
>there is no way to "clarify" QTYPE=* short of a protocol extension.  the
>current behaviour is perfectly well understood and there are no incorrect
>implementations of it.  there's no document quality issue around QTYPE=*,
>and all extant implementations are fully interoperable regarding QTYPE=*.
>
Paul,
         The current behavior is fine as long as one puts on blinders 
with respect to correctness and completeness of answer content. In my 
opinion, giving out a known-incomplete answer to *any* query, when a 
complete answer is available via recursion and where recursion was both 
requested and its availability advertised, is a violation of the spirit 
if not the precise wording of the standards (and let's face it, RFC 
1034&1035 leave a lot to the imagination wrt how QTYPE=* queries are 
actually supposed to work). Just as partial-RRset answers were outlawed 
in in RFC 2181, partial answers at owner-name level (i.e. with RRsets 
missing) should never IMO be returned to recursive QTYPE=* queries either.

The fact that all extant implementations (arguably) violate the standard 
in the same, interoperable way does not mean the behavior is correct -- 
that is, after all, the defining difference between _de_facto_ standards 
and _de_jure_ ones.

More importantly, a faithful implementation of QTYPE=* recursivity might 
rescue that QTYPE from its current state of dubious usefulness (at best) 
and/or near-obsolescence (at worst), and put an end to all of these 
new-metatype proposals without the overreaching that EDNS1 represented 
or was perceived to represent. I can understand why you'd be bitter 
about how EDNS1 turned out, but that doesn't mean a more modest 
approach, one that stays within the existing RFC 1034/1035 framework 
without requiring extensions, can't or shouldn't work.

                           - Kevin

--
to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/namedroppers/>

---

From: Ólafur Guðmundsson<[email-masked]>
Date: Tue, 25 Nov 2003 13:27:26 -0500

</co-chair>
>that is, after all, the defining difference between _de_facto_ standards 

None of the implementations IMHO violate the standard.
QTYPE=* is a debugging tool only.

>new-metatype proposals without the overreaching that EDNS1 represented or 
>was perceived to represent. I can understand why you'd be bitter about how 
>EDNS1 turned out, but that doesn't mean a more modest approach, one that 
>stays within the existing RFC 1034/1035 framework without requiring 
>extensions, can't or shouldn't work.

If you want all the types just ask an authoritative server directly.
There is no need to complicate caches.

(If you are stuck behind a firewall that does not allow you to bypass the
cache, you are out of luck, but that is a well known problem.

         Olafur

---

From: Kevin Darcy <[email-masked]>
Date: Tue, 25 Nov 2003 22:59:19 -0500

O'lafur Gu?mundsson wrote:

>> The fact that all extant implementations (arguably) violate the 
>> standard in the same, interoperable way does not mean the behavior is 
>> correct -- that is, after all, the defining difference between 
>> _de_facto_ standards and _de_jure_ ones.
>
>

The standard describes the basic resolver algorithm as a) answer from 
your cache if you can, or b) if recursion was requested and you're 
willing to provide it, go get the answer from an authoritative server or 
upstream resolver. Now, it's hard for me to read into that that a known 
*incomplete* answer from cache is acceptable to pass back to the client 
even though a complete answer is available via recursion. As was 
clarified later, dropping individual RRs from an answer RRset is illegal 
(except for properly-flagged truncation scenarios), why should it be any 
more legal to drop whole RRsets from a QTYPE=* response for any reason 
other than properly-flagged truncation or the fact that the only way to 
provide the missing RRset(s) would be to recurse, and recursion was 
either not requested or not available?

> QTYPE=* is a debugging tool only. 

That is arguably what it has become. I still don't agree that that is 
what was originally intended or specified.

>> More importantly, a faithful implementation of QTYPE=* recursivity 
>> might rescue that QTYPE from its current state of dubious usefulness 
>> (at best) and/or near-obsolescence (at worst), and put an end to all 
>> of these new-metatype proposals without the overreaching that EDNS1 
>> represented or was perceived to represent. I can understand why you'd 
>> be bitter about how EDNS1 turned out, but that doesn't mean a more 
>> modest approach, one that stays within the existing RFC 1034/1035 
>> framework without requiring extensions, can't or shouldn't work.
>
>

> cache, you are out of luck, but that is a well known problem.

It's not just a matter of connectivity, it's also a matter of resources 
and capacity. Bypassing one's upstream resolver may consume resources 
that one's network administrator never expected or provisioned for. Is 
conserving those resources worth the caching complexity that a robust 
implementation of QTYPE=* would require? Hard to generalize: in some 
cases, yes, some cases no. But this is something IMO that each 
administrator should decide for himself/herself (it's legal for them to 
simply REFUSE all QTYPE=* queries after all); it shouldn't be decided 
pre-emptively for them by software-implementation decisions based on 
questionable interpretations of the RFC.

Also, just because a problem is well known doesn't mean it cannot or 
should not be solved.

- Kevin

---

From: Robert Elz <[email-masked]>
Date: Wed, 26 Nov 2003 16:20:36 +0700

Date:        Tue, 25 Nov 2003 22:59:19 -0500
    From:        Kevin Darcy <[email-masked]>
    Message-ID:  <[email-masked]>

  | The standard describes the basic resolver algorithm as a) answer from 
  | your cache if you can, or b) if recursion was requested and you're 
  | willing to provide it, go get the answer from an authoritative server or 
  | upstream resolver. Now, it's hard for me to read into that that a known 
  | *incomplete* answer from cache is acceptable to pass back to the client 
  | even though a complete answer is available via recursion.

What is a "known incomplete answer"?   How can the cache possibly know
whether what is cached is complete or not?   The effect of enforcing the
protocol that you're suggesting should be enforced, is that any QTYPE=* RD=1
query must be sent to an auth server, and the reply forwarded back to the
client - a cache could cache the results, but not for use in any later
QTYPE=* query, only specific queries.

Since caches are a primary object in the DNS, and everything is designed
around making caches effective, it is almost impossible to conceive that
a query that explicitly avoids the cache could possibly have been intended.

If the client wants to defeat caching, let the client talk directly to an
auth server for the name (and do the extra work to find that server).
Explicitly requiring caches to defeat their reason for existing would be
absurd.

  | As was 
  | clarified later, dropping individual RRs from an answer RRset is illegal 
  | (except for properly-flagged truncation scenarios), why should it be any 
  | more legal to drop whole RRsets from a QTYPE=* response for any reason 
  | other than properly-flagged truncation or the fact that the only way to 
  | provide the missing RRset(s) would be to recurse, and recursion was 
  | either not requested or not available?

They're not being dropped, they simply aren't available at the server
queried to ask.   I'm actually somewhat skeptical of having a cache to
a QTYPE=* request of the auth server if it has no data about the name,
I think I'd prefer to have it simply return NODATA in that case (or
NXDOMAIN if it knows the name doesn't exist) - the only real problem
with that would be the possibility of getting a NODATA for a name which
really doesn't exist (or perhaps a NXDOMAIN for a name that does if
that was the selected response to an ANY query for a name not in the cache).

  | That is arguably what it has become. I still don't agree that that is 
  | what was originally intended or specified.

Nothing says why it was originally specified, in fact, it isn't clear
that there was any reason at all, other than "this looks nice and easy"
(which turned out to be wrong on both counts - if t hat was the explanation).

I support leaving QTYPE=* defined as it has been implemented everywhere.
Return any data in the server receiving the query (ignoring RD if there
is any such data) - if there is none, then send a non-auth NODATA (or a
referral, I guess, that would be better) (again regardless of RD).
Implementations that choose to honour RD when there is no data are OK,
as from the outside no-one can tell if the data existed before the
query or only after it was received - so returning the answer from a
query is OK (as I guess is an implementation which actually always does
a query of the auth server when RD=1 - though I'd never deploy such
a thing).

kre

---

From: "D. J. Bernstein" <[email-masked]>
Date: 26 Nov 2003 23:29:27 -0000

> QTYPE=* is a debugging tool only.

QTYPE=* can be used to check for CNAME records. RFC 1034 prohibits
non-CNAME records at a name that has a CNAME record, so if QTYPE=*
returns any non-CNAME records then there are no CNAME records.

This is widely used in Internet mail: clients check for CNAME records
and use them to rewrite SMTP MAIL/RCPT commands. Clients that fail to do
this are in violation of RFC 1123.

(In theory, QTYPE=CNAME would also work. In practice, QTYPE=CNAME
triggers an old BIND bug---see http://cr.yp.to/im/cname.html---so
QTYPE=* is the de-facto standard solution.)

All of this is changing. We mail implementors have agreed that this SMTP
rewriting is a bad thing; meanwhile, many DNS proposals violate the RFC
1034 non-CNAME prohibition. So a three-step transition is in progress:

   (1) mail administrators will stop relying on SMTP rewriting; and then
   (2) clients will stop looking specially for CNAME records; and then
   (3) the RFC 1034 non-CNAME prohibition can be safely violated.

In the meantime, however, the statement that ``QTYPE=* is a debugging
tool only'' is simply wrong.

---D. J. Bernstein, Associate Professor, Department of Mathematics,
Statistics, and Computer Science, University of Illinois at Chicago

---

From: "D. J. Bernstein" <[email-masked]>
Date: 26 Nov 2003 23:35:40 -0000

Kevin, could you please explain _exactly_ how you think a cache should
handle recursive QTYPE=* queries?

Suppose, for example, that a name has a 3600-second A record and a
60-second MX record. The cache receives a QTYPE=* query for that name.
Two minutes later, it receives another. What happens?

If you're going to use words such as ``incomplete'' in your answer,
please define those words first.

---

From: Robert Elz <[email-masked]>
Date: Thu, 27 Nov 2003 15:59:19 +0700

Date:        26 Nov 2003 23:35:40 -0000
    From:        "D. J. Bernstein" <[email-masked]>
    Message-ID:  <[email-masked]>

  | Suppose, for example, that a name has a 3600-second A record and a
  | 60-second MX record. The cache receives a QTYPE=* query for that name.
  | Two minutes later, it receives another. What happens?

Kevin, if you respond to this - also explain what happens when the
cache receives a QTYPE=* query 30 seconds after the initial query
(with all of the same config as above).

kre

---

From: "Eric A. Hall" <[email-masked]>
Date: Thu, 27 Nov 2003 09:34:48 -0600

> (with all of the same config as above).

Easiest approach to making these queries cachable is to synchronize the
TTLs of all the RRs in the response, and set a flag somewhere that a
qtype=* query was made for the owner name. If there's any RRs for that
name in memory, then you've got all of the RRs.

This wouldn't be the first qtype to require extra processing. Being a
little different doesn't mean impossible.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

---

From: Robert Elz <[email-masked]>
Date: Fri, 28 Nov 2003 18:16:11 +0700

Date:        Thu, 27 Nov 2003 09:34:48 -0600
    From:        "Eric A. Hall" <[email-masked]>
    Message-ID:  <[email-masked]>

  | Easiest approach to making these queries cachable is to synchronize the
  | TTLs of all the RRs in the response, and set a flag somewhere that a
  | qtype=* query was made for the owner name. If there's any RRs for that
  | name in memory, then you've got all of the RRs.

Aside from all the extra mechanism you'd have to assume was existing
in the cache, before you could safely use this, this simply does not
work.

Assuming that all TTLs were already the same (the way people used to make 
master files, with the only TTL setting being the copy out of the SOA.minimum
field), so the cache doesn't have to force it, this still doesn't work.

Let that common TTL be 3600.   Do a query at time X, get all the RR's.
At time X+200 the auth servers all add a new RR to the name queried.
Now, at X+500, send a QTYPE=* query (RD=1) (again) to the cache for
the same name.   Is the new RR included in the reply or not?   If not,
does the answer include all the RR's, as has been demanded that it
include?   If it does, how does it get there?

This problem doesn't exist for normal (single type) queries, as the
TTL is a promise (mostly ignored by admins, but that's not the
protocol's fault) that the data will returned will remain valid for
TTL seconds.   It isn't even a problem for negative answers, as those
include the SOA, and the SOA.minimum is supposed to indicate just how
long you can cache a non-existence report.

But here, we get neither - we have no TTL for the RR (RRset) of the
new RR, so nothing to rely upon.   We have no SOA so no information
on how long we can hand out "doesn't exist" information.

kre

---

From: Bruce Campbell <[email-masked]>
Date: Fri, 28 Nov 2003 17:04:49 +0100 (CET)

On Fri, 28 Nov 2003, Robert Elz wrote:

> the same name.   Is the new RR included in the reply or not?

Assuming the cache has not requeried between X and X+500, the X+200 RR
will not be in the cache or in the answer supplied.

> If not, does the answer include all the RR's, as has been demanded that
> it include?

The answer contains all of the RR's as of the previous time that the cache
queried the auth nameservers (time X), as expected.  For the cache to have
knowledge of RRs added since the previous time, it must have requeried for
the information, thus defeating the purpose of having a cache.

So any answers for QTYPE=* that are fulfilled by a cache are only valid
for when the cache last made that query, and may bear little, if any,
relationship to the current RR set for QTYPE=* returned by the auth
servers.

For situations where there isn't a common TTL, after the expiration of the
shortest TTL, the cache should forget that all those RRs were the result
of QTYPE=* (ie, remove the association between them) and then let them
expire in their own individual time as per normal.

--==--
Bruce.

---

From: Greg Hudson <[email-masked]>
Date: Sat, 29 Nov 2003 01:02:34 -0500

> returns any non-CNAME records then there are no CNAME records.

Pardon my confusion here.  Let's say that we have

  foo.example.com CNAME bar.example.com
  bar.example.com MX 0 baz.example.com
  baz.example.com A 1.2.3.4

If I make a query to a recursive nameserver for foo.example.com MX, I
should see the CNAME and MX records in the answer section, as far as I
know.  I can do my SMTP rewriting by looking at the value of the CNAME
record.  Why do I have to do a * query?

---

From: "D. J. Bernstein" <[email-masked]>
Date: 30 Nov 2003 19:05:00 -0000

Greg Hudson writes:
> I make a query to a recursive nameserver for foo.example.com MX
  [ then the response shows whether foo.example.com has a CNAME record ]

That's just as hard to parse as *, sometimes slower, and never faster.
What point are you trying to make?

If you're thinking ``I get the CNAME information for free because I'm
doing an MX query anyway,'' you're correct for RCPT commands---but the
RFC 1123 non-CNAME requirement also applies to MAIL commands.