Subject: [DNSOP] Batch Multiple Query Packet

From: Hector Santos <[email-masked]>
Date: Mon, 27 Feb 2012 13:35:45 -0500

I am interesting to find information about past or possible current 
interest regarding the support of a Batch single call of multiple 
query packets.

If it doesn't already exist or not considered in the past as an 
unfeasible concept, I am interest in seeing if this is something worth 
pursuing.

Technical Background reasoning.

With the advent of new protocols, especially those offering a domain 
policy construct, the TXT record is used as the fastest entry point 
with the widest support for query resolution.

Yet there are (at least were) technical concerns that this not serve 
to benefit DNS for large scale usage, therefore considerations are 
giving to include a migration path to use new registration of RR types.

This migration path comes with the recognition for a short term 
overhead of using a dual type query concept and long term hope for the 
new RR type to become the non-overhead single query satisfying result.

The overall issue is two folds:

1) one where publishers may have to redundantly create two records and 
the DNS resolvers will always have to do a two queries to maximize 
widest support, and

2) The continued possible dearth of DNS servers (see RFC 3597 
"Handling of Unknown DNS Resource Record (RR) Types") that do not 
support unnamed RR types and/or the recursion requirement.

If DNS server offered support for a Batch Query of multiple packets 
under a single call, this may help with the above migration overhead 
concerns.

Is this something worth pursuing as a new I-D for DNS servers?

Thanks

-- 
Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

---

From: Masataka Ohta <[email-masked]>
Date: Tue, 28 Feb 2012 08:57:28 +0900

Hector Santos wrote:

> interest regarding the support of a Batch single call of multiple query 
> packets.
> 

Having a query requesting multiple RRTYPEs is a bad idea,
because only some of the RRTYPEs may be found in cache.

But, it's OK to have separate queries for each RRTYPEs in a
single packet.

Use TCP or try to extend UDP.

However,

> this may help with the above migration overhead concerns.

I don't think so.

						Masataka Ohta

---

From: Edward Lewis <[email-masked]>
Date: Tue, 28 Feb 2012 05:57:44 +0530

At 13:35 -0500 2/27/12, Hector Santos wrote:
>unfeasible concept, I am interest in seeing if this is something
>worth pursuing.

One (not the only, Ohta replied with another) of the oft-cited 
obstacles is the presence of only one RCODE field in the packet. 
(What if one query would be NXDOMAIN and the other has an answer?)

-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis
NeuStar                    You can leave a voice message at +1-571-434-5468

2012...time to reuse those 1984 calendars!

---

From: Paul Vixie <[email-masked]>
Date: Tue, 28 Feb 2012 00:49:36 +0000

On 2012-02-28 12:27 AM, Edward Lewis wrote:
> obstacles is the presence of only one RCODE field in the packet. (What
> if one query would be NXDOMAIN and the other has an answer?)
>

indeed, this is why multiple queries were not supported in the original
DNS, and it's why EDNS doesn't have it either. the number of signalling
bits needed to explain what went on with the multiple queries made
folks' heads explode. the logic is still online if you want to see it:
<http://nsa.vix.com/~vixie/edns1.txt>.

---

From: Joe Abley <[email-masked]>
Date: Mon, 27 Feb 2012 16:55:45 -0800

On 2012-02-27, at 19:49, Paul Vixie wrote:

> DNS, and it's why EDNS doesn't have it either.

Isn't that really an argument against multiple responses being carried in the same packet, rather than multiple queries?

I'll observe that there are exciting amplification opportunities in a world where a single packet could trigger multiple large responses, though :-)

Joe

---

From: Edward Lewis <[email-masked]>
Date: Tue, 28 Feb 2012 06:32:53 +0530

At 16:55 -0800 2/27/12, Joe Abley wrote:

>I'll observe that there are exciting amplification opportunities in a
>world where a single packet could trigger multiple large responses, though :-)

Yes, please, let's not. ;)  Size amplification is already a problem. 
And, this would  make monitoring extra hard ("how many responses were 
supposed to be sent?) and, well, just try to write the "re-assembly" 
code, trying to collect the responses to the original API call.

An especially unpleasant place in the afterlife is reserved for 
whomever enables this.

---

From: Tony Finch <[email-masked]>
Date: Tue, 28 Feb 2012 01:56:33 +0000

On 27 Feb 2012, at 18:35, Hector Santos <[email-masked]> wrote:

> I am interesting to find information about past or possible current interest regarding the support of a Batch single call of multiple query packets.

It isn't necessary to add protocol support, since you can already send multiple concurrent queries on a single socket.

Tony.
--
f.anthony.n.finch  <[email-masked]>  http://dotat.at/

---

From: Hector Santos <[email-masked]>
Date: Mon, 27 Feb 2012 22:28:19 -0500

> <http://nsa.vix.com/~vixie/edns1.txt>.

Thanks Paul. Great material.

I'm just winging it at this point.

First, I was focusing on the batching of related types, i.e. a 
protocol with new RR type but has an initial default intro record and 
fallback to TXT.  The goal is to have a single call that will yield a 
managed result to assist with the current concerns and waste 
associated with the migration of TXT to the new RR type usage.

Second, I considered there is no room for a packet count, but I was 
thinking of simply bundling two separate packets, i.e. 2*RR for the 
UPD send and how would the servers read this.

If DNS servers will barf, then never mind. :)

But if its offers a way to perform this concept with no breakage, 
perhaps the server will just read the first one and act on it or it 
will process the residual packet as well.  Of course, the client will 
still need to manage the responses with all the potential delays.

Again, just winging it. I don't like kludges. :)

---

From: "Marc Lampo" <[email-masked]>
Date: Tue, 28 Feb 2012 08:34:57 +0100 (CET)

Hello,

By "single call", do you mean :
? "single query packet" (holding multiple questions) ?
   (I think this was part of EDNS1)
? "single TCP session" ?
   Where I believe there is no statement that forbids sending
   multiple, subsequent, queries over a single TCP connection.
   --> "batching multiple queries over a single TCP session"
        is already allowed.

   Beware !  Some name server implementations have implicit,
    sometimes configurable, maximum number of queries they accept
    over a single TCP session.

Kind regards,

Marc Lampo
Security Officer
EURid (for .eu)

-----Original Message-----
From: Hector Santos [mailto:[email-masked]] 
Sent: 27 February 2012 07:36 PM
To: [email-masked]
Subject: [DNSOP] Batch Multiple Query Packet

Thanks

---

From: Shane Kerr <[email-masked]>
Date: Tue, 28 Feb 2012 10:02:27 +0100

Paul,

(Possibly my answer belongs on dnsex not dnsop, but oh well.)

On Tuesday, 2012-02-28 00:49:36 +0000, 
Paul Vixie <[email-masked]> wrote:
> indeed, this is why multiple queries were not supported in the
> original DNS, and it's why EDNS doesn't have it either. the number of
> signalling bits needed to explain what went on with the multiple
> queries made folks' heads explode. the logic is still online if you
> want to see it: <http://nsa.vix.com/~vixie/edns1.txt>.

I think we're missing a potential great opportunity here by thinking
too small.

Rather than mucking about with QDCOUNT and breaking old servers, why
not just move additional queries into EDNS(n)? So, the idea is to
support additional queries by effectively encapsulating them safely
into the EDNS space.

Basically, you say "here is my question, and if you know the
multiple-query DNS extension then please give me those answers too". So
for old servers they treat it as an old-style question, and get
old-style answers. New servers will give additional information.

So on the query side you'd need a field with the "real" QCOUNT, and
then a set of additional QNAME/QTYPE/QCLASS entries, possibly like the
normal question section, although we can also consider optimizing this
a bit by requiring the same QCLASS for all questions for example.

Here's the big win I think...

On the answer side, you actually don't need to preserve existing DNS
format at all. This is because the server knows that the client
supports this wacky new multi-question format.

That means that we can design a packet format that ACTUALLY MAKES
SENSE!!! Consider the possibilities there! We can eliminate the
ridiculous waste of 16-bit fields that only ever use one value; we can
align the packet so that the variable length data always comes at the
end; we can put a massive nonce in; and so on.

I don't think this solves with my personal pet peeve (the need to send
concurrent A and AAAA queries), but it does provide a necessary piece
of the solution.

--
Shane

---

From: Masataka Ohta <[email-masked]>
Date: Tue, 28 Feb 2012 19:32:01 +0900

Hector Santos wrote:

> First, I was focusing on the batching of related types, i.e. a protocol 
> with new RR type but has an initial default intro record and fallback to 
> TXT. The goal is to have a single call that will yield a managed result 
> to assist with the current concerns and waste associated with the 
> migration of TXT to the new RR type usage.

That's doable without multiple queries with a QTYPE=NEWTYPE,
which matches both NEWTYPE and TXT.

However, there is a caching problem as is documented in RFC1035.

					Masataka Ohta

---

From: Paul Vixie <[email-masked]>
Date: Tue, 28 Feb 2012 17:43:47 +0000

On 2/28/2012 3:28 AM, Hector Santos wrote:
> Paul Vixie wrote:
>> ... <http://nsa.vix.com/~vixie/edns1.txt>.
>

the way this was handled for MX was to have A (and later AAAA) as
additional data for the MX QTYPE.

we should have made QTYPE AAAA include type A as additional data. we
still can.

we should have amended QTYPE A to include type AAAA as additional data.
we still can.

using additional data for this is great since if it's missing you query
for it but it won't always be missing.

> UPD send and how would the servers read this.

RCODE FORMERR.

this means the initiator would have to be prepared to try again with
discrete queries. this is a form of protocol negotiation -- the fact of
a responder's nonsupport would have to be discovered (and cached "for a
while") by each initiator. this is already happening for EDNS. i would
recommend against adding more responder state to initiators.

this would also mean that when it works you've got one round trip and
when it doesn't work you've got either three (assuming you don't blast
#2 and #3 without inter-record gap) or two round trips (if you use blast
on #2 and #3.)

> If DNS servers will barf, then never mind. :)

yes.

> Again, just winging it. I don't like kludges. :)

please consider additional data, or blast.

paul

---

From: Nicholas Weaver <[email-masked]>
Date: Tue, 28 Feb 2012 09:57:47 -0800

Just some BotE:  Overhead for each DNS datagram (on an Ethernet) is 8 B for the Ethernet Preamble, 14B for the Ethernet header, 20B for the IP header, and 8 bytes for the UDP header, for a total of 50B.

Assuming the question is 50B, and the answer is 150B, the overhead saved is Not That Much by coalesing two requests into one packet: you only save 100B of overhead for communicating 400B of data, which IS non zero, but not enough to really worry about.

But since if you could coalesce you could do parallel, this savings doesn't help latency much at all: just the transit time for 50B out and 50B back.  If anything, parallel will be BETTER on the latency since batching would probably require a coalesced response, while parallel is just that, parallel, so if the first record is more useful, it can be acted on immediately.

---

From: Paul Vixie <[email-masked]>
Date: Tue, 28 Feb 2012 18:04:12 +0000

> But since if you could coalesce you could do parallel, this savings doesn't help latency much at all: just the transit time for 50B out and 50B back.  If anything, parallel will be BETTER on the la…

parallel (what i called "blast") is great solution for things that don't
run at scale. but the lock-step serialization that we get from the MX
approach (where the A/AAAA RRset may be in the additional data section
and so we have to wait for the first answer before we ask the second or
third question) has a feedback loop effect on rate limiting. it's not as
good as tcp windowing but it does tend to avoid WRED in core and edge
router egress buffers.

all i'm saying is, we have to be careful about too much parallelism
since UDP unlike TCP has no windowing of its own.

---

From: Nicholas Weaver <[email-masked]>
Date: Tue, 28 Feb 2012 10:08:33 -0800

> since UDP unlike TCP has no windowing of its own.

We don't need to be "careful" on this until you are talking about ~10 KB of data or more in a single transaction with no interactions, because before then TCP has the same dynamics due to the initial window size (with browsers opening 4+ connections!), and this doesn't seem to bother people.

---

From: Paul Vixie <[email-masked]>
Date: Tue, 28 Feb 2012 18:40:30 +0000

> (Possibly my answer belongs on dnsex not dnsop, but oh well.)

i've heard recently that dns is exy.

> support additional queries by effectively encapsulating them safely

i'd start over with a new port number first. dns wire encoding is
already wildly complicated.

> old-style answers. New servers will give additional information.

if the EDNS option for this just had an array of additional QTYPE, such
that the real QNAME and QCLASS pertained to all of these additional
questions, then i could definitely see value in this. the response's OPT
option would include the list of qtypes which were found to be
non-empty. all of the rrsets would be in the answer section.

> ...
> ...

i think we'd slab the rrsets, put the signatures on as metadata to the
slabs (having no names of their own), and a lot of other things. it's
worth scribbling down what we'd do in the packet format if we could
start over, even though i don't think we can start over.

paul

---

From: Shane Kerr <[email-masked]>
Date: Wed, 29 Feb 2012 10:22:55 +0100

Paul,

On Tuesday, 2012-02-28 18:40:30 +0000, 
> already wildly complicated.

Ya think? ;)

The main (only?) advantage of doing it with EDNS is that you can work
with existing name servers. It means adding more logic to our already
fabulously complicated resolvers to get full benefit, but nobody ever
said DNS was easy.

So possibly the logic would work like this:

1. Send query with the magic EDNS to a server.
2. If the response is in the new format, then remember that.
3. Subsequent queries go to a different port, using a new format.
 
> > multiple-query DNS extension then please give me those answers
> > too". So for old servers they treat it as an old-style question,
> > and get old-style answers. New servers will give additional
> > information.
> 
> if the EDNS option for this just had an array of additional QTYPE,
> such that the real QNAME and QCLASS pertained to all of these
> additional questions, then i could definitely see value in this. the
> response's OPT option would include the list of qtypes which were
> found to be non-empty. all of the rrsets would be in the answer
> section.

I'm curious what use cases you are thinking of, so that you want to
restrict the entire collection of questions to a single QNAME/QCLASS.

One use case which I can think of which fits in here is the A/AAAA
case. You'd need to cache server support for efficient operations, but
I think it could be a win.

Another use case which fits in is getting both SPF records and legacy
TXT records when checking mail from a domain.

One use case which does NOT fit in here is to translate all of the
servers for an NS RRSET at once. I'd like to be able to say "here are a
list of names, give me all of the A and AAAA records for all of them".
 
Anyway, I'm curious. :)

> > ridiculous waste of 16-bit fields that only ever use one value; we
> > can align the packet so that the variable length data always comes
> > at the end; we can put a massive nonce in; and so on.
> 

If we can make the equivalent of name compression more efficient, I
think it could even be something that has enough operational benefit to
make it worthwhile for operators.

--
Shane

---

From: Ray Bellis <[email-masked]>
Date: Wed, 29 Feb 2012 10:17:26 +0000

On 28 Feb 2012, at 18:40, Paul Vixie wrote:

Yes, that's pretty much what was in the draft I wrote (but didn't publish) and discussed with you during the last Prague IETF.

My rationale likewise was that having constant QNAME and QCLASS but multiple QTYPEs should eliminate the need for multiple RCODEs.

However further discussion with other folks revealed issues (ISTR around DNSSEC?) where it would be possible to get inconsistent RCODEs for different QTYPEs.

I've attached a copy of that document.  It could be resurrected if there's sufficient interest.

Ray

---

From: "John Levine" <[email-masked]>
Date: 29 Feb 2012 16:47:16 -0000

>fabulously complicated resolvers to get full benefit, but nobody ever

If you're adding logic to servers and clients, why couldn't some of
that logic listen on a different port?

But honestly, I don't see what problem is being solved here.  The
original motivation was to ask for SPF and TXT records at the same
time, rather than sending two queries.  You don't need a new version
of DNS to handle that, all you need is a kludge in your server that
knows that when it gets a query for one, it can return the other in
the additional section. MX queries already have their kludge,
returning A and AAAA records.

The meta-reason for doing two queries is that it's still so hard to
provision new RR types that people fake them with TXT.  If we're going
to hack on our DNS software, I'd rather work on that.

-- 
Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

---

From: Phil Regnauld <[email-masked]>
Date: Wed, 29 Feb 2012 02:41:58 -0800

> 3. Subsequent queries go to a different port, using a new format.

	Would it be nuts to sugggest that the different port could be specified
	using SRV ? _dns._udp isn't that far off once you've primed the pump
	(and you've got dynamic port numbers to boot, yay!).

	Oh, that means you'd need to know the new port number, if you don't
	do an initial port 53 lookup. Scratch that :)

	Cheers,
	Phil