Subject: weighting transports

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 01 Apr 2010 12:59:09 -0600

Sometimes a client needs to determine which transport or technology an
application server prefers for a given service type. For examples, an
email service might be provided via both POP and IMAP or a website might
be available via both TCP and SCTP.

The approach taken in draft-daboo-srv-email is to check multiple SRV
records (here IMAP would be preferred):

       _imap._tcp     SRV  0 1 143 imap.example.com.
       _pop3._tcp     SRV 10 1 110 pop3.example.com.

The approach taken in draft-yourtchenko-tran-announce-dns is to define a
new DNS RR (here SCTP would be preferred):

       _http.www      XPORT    5 _sctp
                      XPORT    9 _tcp

I think  it would be good to come to rough consensus (if possible) on a
preferred approach in the Applications Area.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Eran Hammer-Lahav <[email-masked]>
Date: Thu, 1 Apr 2010 16:23:50 -0700

The discussion should not be limited to DNS. I'm sure there will soon  
be other protocol to access mail that are HTTP based.

EHL

On Apr 1, 2010, at 14:59, "Peter Saint-Andre" <[email-masked]>  
wrote:

> <smime.p7s>
> <ATT00001..txt>

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 1 Apr 2010 18:45:52 -0500

> be other protocol to access mail that are HTTP based.

DNS is the discovery protocol.

I think the idea is that you'd do some set of DNS queries and then pick
a result.  With a new RR you might do a single query.  With SRV RRs
you'd have to do one query per-transport.

So if you have IMAP, POP3, MailDAV (just made that up), and even
mbox-over-NFS (even more contrived), some of these over TCP and SCTP,
then you might have to do this many queries before you give up or pick
one:

_maildav._tcp
_imap._tcp
_imap._sctp
_pop3._tcp
_nfsmbox._tcp  (would we want to have _nfsv4/_nfsv3 labels here?)

That's a lot of queries.  In practice we all just configure our MUAs and
that's that.  And even if we wanted to do discovery our MUAs would do at
most two or three queries (to heck with SCTP, and search for just two or
three application protocols).  IMO three queries is tolerable.  Much
more and an "XPORT" RR begins to sound highly desirable.

Nico
--

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 02 Apr 2010 03:18:49 +0100

> more and an "XPORT" RR begins to sound highly desirable.
Right.
I think we need to balance deployability of new RRs versa use of 
existing mechanisms. I would rather have a good enough solution now, 
than a better solution in 2 years and do nothing in short term.

---

From: Nicolas Williams <[email-masked]>
Date: Thu, 1 Apr 2010 22:05:47 -0500

> than a better solution in 2 years and do nothing in short term.

Agreed.  SRV may not be as cool, but it does the job.  If the new thing
will become painfully necessary then we should start now on getting it
done.  But I suspect that as new protocols come along they'll mostly
obsolete older ones.  MailDAV, done right, should displace POP3 for
good, and eventually IMAP too, leaving us with a simpler discovery
problem.

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 02 Apr 2010 04:29:12 +0100

Nicolas Williams wrote:
> obsolete older ones.
Or the new discovery protocol may obsolete the old one.
> MailDAV, done right, should displace POP3 for
> good, and eventually IMAP too,
We shall see about that ;-).
> leaving us with a simpler discovery
> problem.
>

---

From: Andrew Newton <[email-masked]>
Date: Fri, 2 Apr 2010 09:30:23 -0400

> preferred approach in the Applications Area.

I glanced at both drafts and did not see a discussion about why neither RFC 3958 nor RFC 4848 are sufficient for the purposes being described.  Perhaps that is a discussion I missed elsewhere.

I believe Philip Hallam-Baker put forward a very similar SRV proposal years ago but don't recall what happened to it.

Personally, I think it is time for the IETF to consider a new record type to simplify this type of DNS discovery if NAPTR and SRV records are considered too cumbersome for this purpose.

-andy

---

From: Dave CROCKER <[email-masked]>
Date: Fri, 02 Apr 2010 09:40:01 -0700

> preferred approach in the Applications Area.

The premise of these mechanisms is that a random client needs to 'discover' the 
set of supported mechanisms at a random server. I add the 'random's in there 
because anything with prior arrangement is likely to include the necessary 
configuration information as part of the arrangement process.

Adoption of a new mechanism like this requires compelling, immediate need.  Do 
we see it now?  Where?  If there is no immediate need, why is anyone going to 
adopt it?

So a mechanism like these needs to be needed by random client/server pairs, at 
Internet scale.

Before deciding which mechanism is preferred, we ought to determine whether it 
is needed.  Otherwise, we'd be standardizing based purely on theoretical, rather 
than actual, need, and that usually does not turn out well.

Using a new RR creates a second adoption barrier.  Although the cleanest 
approach for new types of queries, the track record for new RRs has been 
extremely poor.  On the other hand, at the last IETF, I seem to recall hearing 
assurances that really, truly, finally, new RRs are no longer difficult to 
deploy widely.  I've no idea whether this latest assurance is valid.  If it is, 
that's great.  But we ought to develop a broader sense of this point.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: "Dan Wing" <[email-masked]>
Date: Fri, 2 Apr 2010 10:03:27 -0700

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Dave CROCKER
> Sent: Friday, April 02, 2010 9:40 AM
> To: [email-masked]
> Subject: Re: weighting transports
> 
> 
> 
> we see it now?  Where? 

HTTP over SCTP,
http://tools.ietf.org/html/draft-wing-http-new-tech-00#section-3

-d

> If there is no immediate need, why is 
> anyone going to adopt it?
> 
> _______________________________________________
> Apps-Discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 2 Apr 2010 10:03:51 -0700

At 9:40 AM -0700 4/2/10, Dave CROCKER wrote:
>The premise of these mechanisms is that a random client needs to 'discover' the set of supported mechanisms at a random server. I add the 'random's in there because anything with prior arrangement is likely to include the necessary configuration information as part of the arrangement process.
>
>Adoption of a new mechanism like this requires compelling, immediate need.  Do we see it now?  Where?  If there is no immediate need, why is anyone going to adopt it?
>
>So a mechanism like these needs to be needed by random client/server pairs, at Internet scale.

I'm with Dave on this one. When reading the initial proposal, I could not think of a real reason for it to exist. It does *not* make service deployments easier unless the central party knows for sure that the client has the new capability, but I don't see a way to test that.

What are the actual use cases?

---

From: Dave CROCKER <[email-masked]>
Date: Fri, 02 Apr 2010 11:34:54 -0700

> http://tools.ietf.org/html/draft-wing-http-new-tech-00#section-3

Really?  Is there a pressing, immediate problem in using HTTP over SCTP?  Do we 
have a solid portion of the community currently suffering from the lack of a 
guidance mechanism to resolve this?

And, by the way, the query mechanism has to be cheaper than a simple test of 
alternative transports.  Trying an SCTP-based connection and then trying a 
TCP-based one is pretty cheap, particularly given the relatively long-term 
reality that querying the DNS will provide a negative response, since a random 
server's entries won't support this feature...

And by the way, the proposals seem to be all-or-nothing for a domain name, 
rather than guiding towards different transports according to different 
criteria.  For example, I suspect that choosing SCTP is more important fro some 
content than other?

d/
-- 

   Dave Crocker

---

From: "Dan Wing" <[email-masked]>
Date: Fri, 2 Apr 2010 11:54:19 -0700

> have a solid portion of the community currently suffering 
> from the lack of a guidance mechanism to resolve this?

No, Dave, we don't.  I am trying to get SCTP off the ground,
and I am trying to avoid sending zillions of probes (SCTP INITs)
to do it.

> And, by the way, the query mechanism has to be cheaper than a 
> simple test of alternative transports. 
>
> Trying an SCTP-based connection and 
> then trying a TCP-based one is pretty cheap,

Yes, that is the plan short term and already discussed in 
draft-wing-http-new-tech-00.  However, there can't be any delay
between attempting SCTP versus attempting TCP -- or else we 
get into the exact problem we have today with poor-quality IPv6 
connections (currently being discussed on dnsop).  

draft-yourtchenko-tran-announce-dns-01 is because we are 
worried about the longer term:  when we need SCTP, TCP, 
SCTPoverUDP (for NAT or firewall traversal), SCTPoverTCP 
(for NAT or firewall traversal), and IPv6 and IPv4.  That
is a lot of probes.  Throw in a new, fancy transport 
protocol, or a host trying all of its source addresses, 
and that is a lot of probes.

> particularly given the relatively long-term 
> reality that querying the DNS will provide a negative 
> response, since a random server's entries won't support 
> this feature...

An argument for negative caching in the DNS recursive 
resolvers.

> criteria.  For example, I suspect that choosing SCTP is more 

That can be accomplished -- XPORT is per-hostname, not 
per domainname.

-d

---

From: Dave CROCKER <[email-masked]>
Date: Fri, 02 Apr 2010 13:11:29 -0700

On 4/2/2010 11:54 AM, Dan Wing wrote:
>>>> Adoption of a new mechanism like this requires compelling, immediate
>>>> need.  Do we see it now?  Where?
>>>
>>> HTTP over SCTP,
>> Really?  Is there a pressing, immediate problem in using HTTP over SCTP?
>> Do we have a solid portion of the community currently suffering from the
>> lack of a guidance mechanism to resolve this?
>
> No, Dave, we don't.  I am trying to get SCTP off the ground, and I am trying
> to avoid sending zillions of probes (SCTP INITs) to do it.

Sorry, I thought you were responding to the point I made that you quoted, about
compelling immediate need.  Yours is an example of a double-dependency adoption
barrier.

It's not a question of whether the goal is worthy but of whether
potential adopters will be sufficiently motivated to overcome both hurdles.
Note that achieving utility is not a single-adopter issue.  Adopters among
clients and servers -- in sufficient number -- are needed, to achieve meaningful
utility.

>> And, by the way, the query mechanism has to be cheaper than a simple test
>> of alternative transports.
>>
>> Trying an SCTP-based connection and then trying a TCP-based one is pretty
>> cheap,
>
> draft-wing-http-new-tech-00.  However, there can't be any delay between
> attempting SCTP versus attempting TCP -- or else we get into the exact
> problem we have today with poor-quality IPv6 connections (currently being
> discussed on dnsop).

V6 makes this a three-barrier adoption problem.

> draft-yourtchenko-tran-announce-dns-01 is because we are worried about the
> longer term:  when we need SCTP, TCP, SCTPoverUDP (for NAT or firewall
> traversal), SCTPoverTCP (for NAT or firewall traversal), and IPv6 and IPv4.
> That is a lot of probes.  Throw in a new, fancy transport protocol, or a host
> trying all of its source addresses, and that is a lot of probes.

As we've seen with v6, folks do not deploy and use new mechanisms out of a sense 
of possible loong-term need.

>> particularly given the relatively long-term reality that querying the DNS
>> will provide a negative response, since a random server's entries won't
>> support this feature...
>
> An argument for negative caching in the DNS recursive resolvers.

That helps with re-queries, not new queries.  The rate of new queries by a 
client tends to be significant.

>> criteria.  For example, I suspect that choosing SCTP is more important fro
>> some content than other?
>
> That can be accomplished -- XPORT is per-hostname, not per domainname.

Except that I describe a per-service differentiation of need, not per-host.

Maybe that's not the right chacterization, but it highlights the need to be very 
careful about making sure that a proposed facility really does satisfy 
real-world usage scenarios, including proper granularity.

d/
-- 

   Dave Crocker

---

From: "Dan Wing" <[email-masked]>
Date: Fri, 2 Apr 2010 14:12:14 -0700

> clients and servers -- in sufficient number -- are needed, to 
> achieve meaningful utility.

An already-raised point, on the TAE mailing list and our
Bar-BoFs, has been how we might avoid/reduce the combinatorial
explosion of lots of probes.  draft-yourtchenko-tran-announce-dns
is our response to that concern.

You are saying the concern is not legitimate until there is
an explosion, and that when the explosion occurs we (the IETF)
should then worry about that combinatorial explosion and how
to mitigate it.

> worried about the
> As we've seen with v6, folks do not deploy and use new 
> mechanisms out of a sense of possible loong-term need.

I also do not expect anyone to deploy SCTP or DCCP from a 
sense of community-good.  Rather, there are other reasons new
transports provide value to applications (e.g., quicker
response times) or to application developers (e.g., record
oriented versus stream oriented).

> That helps with re-queries, not new queries. 

Negative caching does help with new queries, because many people 
use the same DNS recursive resolver (belonging to their ISP, 
or belonging to opendns.org or Google or whatever) and many 
people access the same popular websites (facebook, cnn, 
etc.).

> The rate of new queries by a client tends to be significant.

By that argument, probes would be costly.

> Except that I describe a per-service differentiation of need, 

I don't know what you mean by 'per-service', sorry.  

> real-world usage scenarios, including proper granularity.

An example would be helpful.

-d

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 2 Apr 2010 15:46:40 -0700

This discussion has now gone off in many directions. Peter started off with two questions, only one of which was matched in the subject line:

>be available via both TCP and SCTP.

POP and IMAP are application protocols.

TCP and SCTP are transports.

Dan then spread it to more transports (SCTPoverUDP and SCTPoverTCP), and also the internet layer (IPv4 and IPv6).

We now have combinations of 7, 4, and 3.

I suggest that not one size of solution will fit all three best. Dan pointed out the current discussion in the DNSOP WG, and that is proving that there is not strong consensus on how to use the DNS for differentiating on what to answer for layer 3 addresses for a host name. I can imagine the same problems for layer 4 and 7, much less 3-and-4, 4-and-7, and 3-and-4-and-7.

---

From: Dave CROCKER <[email-masked]>
Date: Fri, 02 Apr 2010 16:01:01 -0700

>>  it highlights
>> the need to be very

S/MIME and PGP vs. DKIM -- that's the only 'granularity' example I can think of.

As for the challenge of getting potential adopters motivated, how about: 
PEM/MOSS; IPv6; IRI; IPSec; DNSSec; PKIX; Diameter; Simple; Enum.

All have been well-intentioned, pursued diligently, and started a long time ago. 
  None has (yet) satisfied their claimed goal, though some have found useful 
niches and a couple of obvious ones might finally be reaching critical mass, 
after 15-18 years.

Since the cost of developing a standards is millions of dollars it seems worth 
validating who will consume the work when it is available.

d/

-- 

   Dave Crocker

---

From: "Dan Wing" <[email-masked]>
Date: Fri, 2 Apr 2010 16:47:36 -0700

> >>  it highlights the need to be very
> niches and a couple of obvious ones might finally be reaching 
> critical mass, after 15-18 years.
>
> Since the cost of developing a standards is millions of 
> dollars it seems worth validating who will consume the 
> work when it is available.

The IETF standardized SCTP a decade ago.  We could just add
it to the list of 'failures' at 
http://trac.tools.ietf.org/misc/outcomes, or sit and wait
and wonder why The Industry has not adopted the technology.

However, I would rather take advantage of the millions of 
dollars that have already been spent on SCTP to help get 
SCTP deployed on the Internet.

Towards that end, yes, we can have SCTP- and TCP-capable
clients probe for both SCTP and TCP.  And, if they think
they are behind a non-SCTP-capable NAT (which is most 
likely true), they can also probe for SCTP-over-UDP.  I 
believe that is necessary, and I co-authored an Internet 
Draft describing exactly that: draft-wing-http-new-tech.
Such probing is what the SCTP-enabled Firefox build at
http://pel.cis.udel.edu does.

But at the Dublin IETF meeting during the TAE Bar-BoF,
I received feedback that probing will explode with
more transports (including SCTP over UDP), and was 
asked to look at how to reduce probing.  Mailing list
archives are at https://www.ietf.org/mailman/listinfo/tae
or you can talk with any of the principals that were
there, such as for example Bryan Ford who was worried
about this specific problem.

The consumers of the work will be SCTP-enabled clients,
which are using TCP today, that want or need to reduce
their SCTP (and SCTP-over-UDP) probes across the 
Internet.  So far there is an SCTP-enabled fork of
the Firefox browser http://pel.cis.udel.edu

So far we have received guidance that SRV is not the 
direction to go from Ted Hardie [1] and John Klensin [2],
[1] http://www.ietf.org/mail-archive/web/apps-discuss/current/msg00095.html
[2] http://www.ietf.org/mail-archive/web/apps-discuss/current/msg00096.html

Your guidance appears to be "don't bother us, we have 
standardized things that have received little use.  Just 
send probes."

-d

---

From: "Dan Wing" <[email-masked]>
Date: Fri, 2 Apr 2010 17:11:40 -0700

> [mailto:[email-masked]] On Behalf Of Paul Hoffman
> Sent: Friday, April 02, 2010 3:47 PM
> Subject: Layer explosion (was: Re: weighting transports)
> 
> This discussion has now gone off in many directions. Peter 
> started off with two questions, only one of which was matched 
> in the subject line:
> 
> Dan then spread it to more transports (SCTPoverUDP and 
> SCTPoverTCP), and also the internet layer (IPv4 and IPv6).
> 

Actually, I only pointed out the IPv4/IPv6 problem to show
that probing can be involved on IPv6, too.

Today, we have content providers that are reluctant to put
AAAA records on their servers.  "Reluctant" is putting it
mildly.  AAAA is how we know a server is running IPv6.  But
we are not probing for IPv6; instead, connections are 
attempted and time out (after multiple seconds) and then
fall back to IPv4.  Over and over and over, taking many
seconds to display a webpage.

I would like to get SCTP deployed (to replace TCP where it
makes sense), but we need to do that without the same issues
currently suffered with IPv6.  Active probing is a reasonable
answer to meet that desire.  But there have been concerns
about that additional network traffic.

-d

> I suggest that not one size of solution will fit all three 
> best. Dan pointed out the current discussion in the DNSOP WG, 
> and that is proving that there is not strong consensus on how 
> to use the DNS for differentiating on what to answer for 
> layer 3 addresses for a host name. I can imagine the same 
> problems for layer 4 and 7, much less 3-and-4, 4-and-7, and 
> 3-and-4-and-7.

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 2 Apr 2010 17:42:56 -0700

>that probing can be involved on IPv6, too.

Sure can. And if any of the application protocols that will be available over TCP and SCTP on IPv4, will be available on both IPv4 and IPv6 within a few years.

>seconds to display a webpage.

Yep. That's the difference between "probe" and "try/timeout/fallback".

>about that additional network traffic.

There are tradeoffs to probing, but for picking between different transports, I believe that probing is a better tradeoff than "devise some DNS scheme and wait for it to be deployed and hope we got it right" or "new transports are only for new application protocols". Probing is even more attractive if the client caches the results (maybe even with TTLs coming from extensions the transport). The increased traffic of occasional cached probes will be way less than the malicious probing done all the time today.

---

From: Nicolas Williams <[email-masked]>
Date: Fri, 2 Apr 2010 20:47:22 -0500

> seconds to display a webpage.

The obvious thing to do on the client side is to parallelize.  Try iPv4
and IPv6 both (and SCTP and TCP and SCTP-over-UDP).  Whichever finishes
first wins the first time, and the others, when they complete or
timeout, go in a cache.  Next time you pick the best that you remember
being available.

Of course, that just consumes more server resources on the first try.
And if your cache becomes invalid then you get timeouts.  Also, whether
SCTP works or not will depend on where the client is and can't really be
cached reliably on a mobile device, nor can it be discovered via DNS or
anything other than probing.  But aside from that parallelized probing
should work, so much so that I can't see why bother with new discovery
schemes.

> about that additional network traffic.

If you can cache then the extra network traffic will be in the noise.
Actually, even if you don't cache the extra network traffic will be in
the noise (for protocols like IMAP we're talking bulk data, which
overwhelms any SCTP INIT or TCP SYN packets).  I'd worry more about the
server, but only if you can't cache.

Nico
--

---

From: "Dan Wing" <[email-masked]>
Date: Fri, 2 Apr 2010 19:11:49 -0700

> From: Paul Hoffman [mailto:[email-masked]] 
> Sent: Friday, April 02, 2010 5:43 PM
> To: Dan Wing; [email-masked]
> Subject: RE: Layer explosion (was: Re: weighting transports)
> 
> Sure can. And if any of the application protocols that will 
> be available over TCP and SCTP on IPv4, will be available on 
> both IPv4 and IPv6 within a few years.
> 
> There are tradeoffs to probing, but for picking between 
> different transports, I believe that probing is a better 
> tradeoff than "devise some DNS scheme and wait for it to be 
> deployed and hope we got it right" or "new transports are 
> only for new application protocols". Probing is even more 
> attractive if the client caches the results (maybe even with 
> TTLs coming from extensions the transport). The increased 
> traffic of occasional cached probes will be way less than the 
> malicious probing done all the time today.

Great.  We will probe.  As I said, that is what is already
described in draft-wing-http-new-tech.

The DNS proposal is an OPTIMIZATION, to reduce the need to
probe so much.  It is not, as you summarize, "devise some DNS 
scheme and wait for it to be deployed and hope we got it 
right".

-d

---

From: Scott Brim <[email-masked]>
Date: Fri, 02 Apr 2010 22:15:44 -0400

> Of course, that just consumes more server resources on the first try.

and "first tries" will be relatively rare compared to "later tries".

> And if your cache becomes invalid then you get timeouts.  

Only if your cache is absolute, yes-or-no.  http-new-tech suggests
weighted preferences.

> Also, whether
> anything other than probing.  

If you are roaming within cellular operators, there is a high likelihood
that if SCTP works in one cell it will work in another.  There is also
reasonable likelihood that if it works in one WiFi hotspot you visit
that it will work in the next ... but again, what you cache is not an
absolute choice but a behavior modifier that learns with use.

> overwhelms any SCTP INIT or TCP SYN packets).  

:-)

Scott

---

From: "Dan Wing" <[email-masked]>
Date: Fri, 2 Apr 2010 19:23:37 -0700

> timeout, go in a cache.  Next time you pick the best that you remember

Yes, that's what we describe in draft-wing-http-new-tech
already.

> Of course, that just consumes more server resources on the first try.

Yes, exactly.  Especially if the SCTP connection finishes.  The TCP
connection now has to be abandoned by the client (RST).

> anything other than probing. 

Completely agreed we will always need to do some probing.  That is 
where we (the industry) went wrong with IPv6-versus-IPv4 for 
time-critical applications such as web browsing.  I am trying to
avoid that mistake with SCTP.

> But aside from that parallelized probing

It is an optimization.  An optimization to avoid probing when the 
probing will not be successful.

> overwhelms any SCTP INIT or TCP SYN packets).  I'd worry more 
> about the server, but only if you can't cache.

HTTP and the many servers used by today's modern web page is my
concern, far more than POP or IMAP.  HTTP retrieves objects from 
many servers, in order to get around the client's server connection 
limit, and each of those would suffer a probe from every client.  
Successful probes (where both SCTP and TCP work) would incur 
a "wasted" TCP connection on the server that is abandoned by the
client.

-d

---

From: Nicolas Williams <[email-masked]>
Date: Sat, 3 Apr 2010 21:01:13 -0500

On Fri, Apr 02, 2010 at 07:23:37PM -0700, Dan Wing wrote:
> > And if your cache becomes invalid then you get timeouts.  Also,
> > whether SCTP works or not will depend on where the client is and
> > can't really be cached reliably on a mobile device, nor can it be
> > discovered via DNS or anything other than probing. 
> 

You have a very different view of the world than I do then.  IMO the
problem with IPv6 was most definitely not about discovery, but about the
difficulty of getting any sort of incremental deployment going comined
with the rise of NAT, which pushed off the addressing problems with
IPv4, and the lack of other sufficiently compelling reasons to move off
IPv4.

> probing will not be successful.

Is it really?  Consider: 1 DNS lookup + N parallel connects vs 2 serial
DNS lookups and 1 serial connect.  The parallel case ought to win.
OTOH, if you can reduce the cost of the DNS lookups for discovery and
name resolution so that one query will do then discovery could at least
match the performance of probing, but at probably only at significant
architectural cost.

> a "wasted" TCP connection on the server that is abandoned by the

That HTTP isn't usually used for bulk isn't really important, since when
it isn't used for bulk it's used in many serial connections in a short
space of time... which means that you can cache effectively for long
enough that the overhead of probing will still be in the noise.

The more I think about this the more I think that if there was a simple
API for asynchronously and concurrently connecting to one service in N
ways then probing would win outright and that would be that.  Servers
would have to cope, but then, they already have to cope with SYN DoS
attacks, so I'm not terribly concerned, and besides, probing overhead
will be in the noise for well-behaved clients.

Nico
--

---

From: Dave CROCKER <[email-masked]>
Date: Sun, 04 Apr 2010 10:51:36 -0700

Dan,

On 4/2/2010 4:47 PM, Dan Wing wrote:
> The IETF standardized SCTP a decade ago.  We could just add it to the list of
> 'failures'... or sit and wait and wonder why The Industry has not adopted the
> technology.

This is what's called a false dichotomy.  It asserts only two alternatives --
which can be as arbitrary as these two -- and ignores all the others that are
available.  It is an expression of frustration used when attempting to terminate 
debate and prematurely force adoption of one of the alternatives, without 
consideration of the underlying issues or tradeoffs.

I now believe the deeper question is persistent validity of URLs, across changes
to the infrastructure.  For example, should a URL become invalid because a
different transport is used?

The answer needs to be no.  Yet the view that the URL scheme would change when a
new transport is used will produce that unfortunate result.

It's not that Ted's concern about the possibility of divergent results based on
using different library packages is wrong, it's that the proposed solution with
different URIs is clearly the wrong result.  Having an application protocol run 
over a different transport does not create a different service; it should not 
render the URI invalid.

I've pushed back on your proposal not because I think you are pursuing an
unattainable or unproductive goal but because its success has too many critical
dependencies and too little compelling near-term value for adopters.

The hard work is in reducing dependencies and increasing perceived value.
Ignoring barriers to adoption seems to account for the failure of most
otherwise-appealing efforts.

The idea that a URI scheme couples application protocol with transport is really
an architectural form of source-routing.  One of the architectural wins for
Internet design has been hiding details from upper layers.

(For that matter, "http://" has this problem for the application layer, whereas
"mailtto://" does not.  It's the difference between using protocol names versus
service names. We need to be careful about forcing the wrong set of changes on 
existing behaviors.)

As of right now, I'd be interested in a review of the reason SRV records aren't
perfect dealing with the choice between TCP and SCTP.  I thought that that was 
the one of the reasons SRVs were created.

d/

-- 

   Dave Crocker

---

From: Nicolas Williams <[email-masked]>
Date: Sun, 4 Apr 2010 15:06:47 -0500

> scheme and wait for it to be deployed and hope we got it 

I don't believe it'd actually optimize anything, except, _pehaps_
implementation complexity.

Nico
--

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 5 Apr 2010 13:56:37 -0400

On Fri, Apr 02, 2010 at 09:40:01AM -0700, Dave CROCKER wrote:
> extremely poor.  On the other hand, at the last IETF, I seem to recall 
> hearing assurances that really, truly, finally, new RRs are no longer 
> difficult to deploy widely.  I've no idea whether this latest assurance 
> is valid.  If it is, that's great.  But we ought to develop a broader 
> sense of this point.

I'm not sure from whom you heard this, but here's approximately
everything I know about the getting of and deploying new DNS RRTYPEs.

1.  Getting a new RRTYPE is done by expert review.  Some of the
experts on the panel have not been as fast as we'd like about
turnaround, but we've had more than one successful assignment of
RRTYPE numbers under this process over the past little while.  In at
least one case, and maybe two, there were many people who thought the
RRTYPE in question was a Bad Idea.  Didn't matter.  They met the
criteria, so they're in.

2.  Practically every deployed name server in the world can deal with
unknown RRTYPEs any more.  There are some exceptions, of course.  But
I do not believe that anyone who seriously needs a new RRTYPE is going
to be prevented from using it because of the lack of support in an
authority or infrastructure-grade recursive nameserver.

3.  There is a distressing number of middleboxes on the Internet that
do hideous things to DNS packets.  This includes brilliant behaviour
(I have actually seen this) such as treating every single DNS request
as a lookup for A records, which means that if you ask for anything
else at a name you either get nothing or an A record back.  These are
in decline, but their population is not negligible.  You should assume
users will sometimes have such a thing in the way.  As DNSSEC
deployment becomes more widespread, we may find some of these getting
replaced because of how they break DNSSEC.

4.  The OS APIs to get anything meaningful out of the DNS are not
there, and when they are they're often not used much.  That is, if you
want to look up your favourite RRTYPE on many OSes, you're stuck doing
all the resolution yourself either because the underlying system
doesn't do it right, or else getting the system to do it right is
harder than just implementing some minimal resolver yourself.

5.  Application people shouldn't have to care about the DNS anyway:
it's too far down the stack.  So the pressure to change (4) isn't
there.  (I don't speak authoritatively on this one, but someone gave
me a compelling argument to this effect over lunch in Anaheim, and I
think I agree.)

So, depending on what you want your new RRTYPE for, it's either
trivial to get and deploy or else just short of impossible.

Best,

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Dave CROCKER <[email-masked]>
Date: Mon, 05 Apr 2010 11:10:38 -0700

Andrew,

Thanks for the followup.

On 4/5/2010 10:56 AM, Andrew Sullivan wrote:
> do hideous things to DNS packets.  ... You should assume
> users will sometimes have such a thing in the way.

That would seem to be a rather fundamental, current barrier to the utility of 
new RRs.  No?

> there, and when they are they're often not used much.

Seems like another fundamental, current barrier.

> it's too far down the stack.

Not really.  Beyond mapping name to IP Address, the DNS is of increasing use for 
assorted other application-related mapping services.  (It won't be a surprise 
that I'll cite DKIM.)

> trivial to get and deploy or else just short of impossible.

Ack.  Might be useful to to characterize each, as guidance for work that expects 
to use the DNS.

d/
-- 

   Dave Crocker

---

From: Eliot Lear <[email-masked]>
Date: Mon, 05 Apr 2010 20:18:25 +0200

Andrew,

You left off a huge additional reason:

    * Nobody's provisioning software supports a new RRtype and the time
      it takes to deploy such provisioning tools is measured in eons. 
      Wasn't SRV done in the Paleolithic Era?  ;-)

This is why we have any number of uses of TXT...

Eliot

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 5 Apr 2010 14:23:40 -0400

On Mon, Apr 05, 2010 at 11:10:38AM -0700, Dave CROCKER wrote:
> That would seem to be a rather fundamental, current barrier to the 
> utility of new RRs.  No?

It depends on what you want to do with them.  If you know your
application is going to be able to work around these things, then
maybe not.  If you know your application just can't be deployed in
such environments (it's not like such environments only screw with DNS
packets, and have no other ideas about what is "allowed"), then
probably not.  But you need to take it into account, for sure.  

> Seems like another fundamental, current barrier.

It would appear that the consumers of OS APIs are the needed source of
pressure on this topic.  OS vendors aren't going to listen to DNS
weenies telling them, "We have all these neato things you could
expose.  Why not do so?"  They're maybe going to listen to application
authors saying, "We want to be able to do _x_."  And I bet that
they'll really pay attention if a number of applications start
installing alt_resolver_lib so that they can get the functionality
they want.  Anyway, the community on this list is in a better position
to push for this than I am.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 5 Apr 2010 14:26:03 -0400

> Wasn't SRV done in the Paleolithic Era?  ;-)

Really?  What provisioning software is this?  Most of the DNS
libraries I'm familiar with are perfectly capable of coping with
unknown RRTYPEs in exactly the same way that the DNS servers are.  The
unknown type was in fact also done in the Paleolithic Era, and if
people just can't handle it these days I think they ought to be
embarrassed to call themselves capable of "provisioning".

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Eliot Lear <[email-masked]>
Date: Mon, 05 Apr 2010 20:34:10 +0200

On 4/5/10 8:26 PM, Andrew Sullivan wrote:
> Really?

Really.

> What provisioning software is this?

In the enterprise space this varies from commercial tools to shell 
scripts to perl.  Quite all over the map.  Once up and running IT 
departments are loathe to change it because of the critical purpose it 
serves in the network, and the fact that you have to hold a voodoo 
ceremony involving puppets and pins after you make any changes because 
of the nature of the beast.

>    Most of the DNS
> unknown RRTYPEs in exactly the same way that the DNS servers are.

You've missed the key word "provisioning".  It's great that the server 
can handle an unknown type.  But most tools that call it can't.

Eliot

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 5 Apr 2010 14:44:01 -0400

> of the nature of the beast.

So what you're really saying is that the existing tool doesn't handle
the unknown RRTYPE or your new special RRTYPE; they won't modify it
for the general case because, well, they won't; and the new RRTYPE
isn't valuable enough to them to make the feature worth the cost.
That's ok: that's a trade-off they're making.  I've worked in plenty
of shops like this, too, and the usual answer is that everyone gets
crappier behaviour than they might otherwise.  They either live with
it or leave to work in a company where they can get their work done. 

Handling of unknown RRTYPEs is not quite as old as I recalled.  Only
2003.  To me, that says that if it's actually useful in your case,
then you should use it & be prepared for what happens when it fails.

> can handle an unknown type.  But most tools that call it can't.

No, I didn't miss it.  Handing data _in_ to a DNS server requires DNS
libraries, too.  And that's provisioning, where I come from.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Joe Hildebrand <[email-masked]>
Date: Mon, 05 Apr 2010 13:01:30 -0600

> to push for this than I am.

I'd settle for being able to do SRV lookups, as a start.  For example, the
.Net framework still only allows A (and maybe AAAA) lookups, without
resorting to hand-implementation of DNS or P/Invoke to get to the Win32 API,
where the call in question isn't supported on all OS versions.

Is there an alt_resolver_lib yet that meets all of the requirements?  I
still have that doc around somewhere that lists them all out.

Regardless, I doubt that it's possible to put enough pressure on some
vendors to get them to support generic/new RRs in their API.  To do it
*well* (i.e. in such a way that a typical VB programmer can use the method),
the library needs to parse the results and give them back in an
easily-digestible form.  That means work for each new RR type, which has to
be scheduled and delivered in a multi-year product cycle.  Having an escape
mechanism for these new RRs is useful and interesting, but hard to document,
and easy for a typical programmer to mess up in use, leading to support
costs.

Therefore, I still fall squarely in the camp of "impossible to deploy" for
application-layer protocols.

-- 
Joe Hildebrand

---

From: Nicolas Williams <[email-masked]>
Date: Mon, 5 Apr 2010 14:11:49 -0500

> isn't valuable enough to them to make the feature worth the cost.

Imagine a tool like UName*It or Moira.  You won't find much on UName*It
online so I'll give you a brief description: an object-oriented RDBMS
specifically designed for naming data (DNS, NIS, LDAP, ...).  It
encourages one to fully normalize one's schema, which means new RR types
require changing the schema.

Q: How would you handle new RR types with such a tool?  A new RR type's
   RDATA might reference a variety of objects, and these relationships
   need to be modelled appropriately if you're to take advanatage of the
   RDBMS' referential integrity features.
A: Modify the schema, and also any scripts that generate/update name
   service data from the DB.

Q: What if you're working with something less advanced?
A: Modify whatever scripts/programs make it up.

Q: What if you get this tool from a vendor?
A: File a bug/RFE, wait.

Q: What if you're getting service from a provider who's getting their
   provisioning system from some other vendor?
A: File a bug/RFE, wait for the provider to... file a bug/RFE, and wait.

Q: What if you're part of a captive audience?
A: Wait, and wait, and wait, then wait some more.  Or find a way to vote
   with your feet/money.

In any case, adoption isn't instantaneous.

Nico
--

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 5 Apr 2010 15:20:39 -0400

>    service data from the DB.

Well, maybe.  Another way would be to implement the unknown type in
the tool, with opaque (i.e. RRTYPE id only) types and just a blob of
data on the right hand side.  (And yes, this bugs me too, having come
to this game as a database geek.  But that's what the unknown type is
like.)

> In any case, adoption isn't instantaneous.

I wasn't trying to suggest it was.  On the other hand, all of these
complaints are the sort of thing we hear about every other technology,
too, yet I don't notice people refusing to implement stuff on the
grounds that there won't be instantaneous worldwide adoption.  It's
only some protocols (the DNS is among them) where we hear that using
new features is "impossible".

I have a feeling that part of the conviction of "impossibility" comes
from years of having to deal with obstreperous DNS weenies who made
everything harder than it had to be.  I'm content to accept that
there's a hangover from that.  It's for the most part gone, however.
That does not mean, in any case, that I believe the world has
magically made new DNS RRTYPEs easy.  It does mean that one barrier
has been removed.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Nicolas Williams <[email-masked]>
Date: Mon, 5 Apr 2010 14:24:55 -0500

> to this game as a database geek.  But that's what the unknown type is

Yes, but this way you lose the referential integrity features of the
RDBMS w.r.t. the new RR's RDATA.  That's not cool, and it's a non-answer
to the question posed (I specifically menaioned referential integrity in
the question for a reason :)

> magically made new DNS RRTYPEs easy.  It does mean that one barrier

In this case the alternative to new RR types is parallelized probing.
And because SCTP might not go through midlle-boxes and DNS cannot
possible know when there are such middle boxes between a client and some
target, there's really little choice but to probe.  The remainder of the
discussion (e.g., this sub-thread), is interesting but tangential.

Nico
--

---

From: Dave CROCKER <[email-masked]>
Date: Mon, 05 Apr 2010 12:34:04 -0700

On 4/5/2010 12:01 PM, Joe Hildebrand wrote:
> I'd settle for being able to do SRV lookups, as a start.

While a new RR is the cleanest approach for extending DNS response data 
semantics, it has been a barrier for 20+ years.

One can bemoan this or one can try to deal with it.

Underscore-based naming such as

    RFC 2782 -- _ldap._tcp.example.com

and

    RFC 4871 -- foo.bar._domainkey.example.com

has fared considerably better, although as Joe notes, there are still limitations.

In effect, an underscore name defines an scoped attribute space for the 
associated parent domain.

An RR record is a type of attribute.  Hence, there seems to be an equivalence 
between using RRs and using underscore-based names with a limited set of RRs, 
such as SRV and TXT.

My question is whether there are semantic or performance problems that render 
use of underscore names meaningfully inferior to use of new RRs?

d/

-- 

   Dave Crocker

---

From: Nicolas Williams <[email-masked]>
Date: Mon, 5 Apr 2010 14:40:23 -0500

On Mon, Apr 05, 2010 at 12:34:04PM -0700, Dave CROCKER wrote:
> My question is whether there are semantic or performance problems
> that render use of underscore names meaningfully inferior to use of
> new RRs?

IMO, not really.  The main issue is that the format of the text in a TXT
RR's RDATA is dependent on the domainname of the RR.  This complicates...
referential integrity in provisioning.  (It's not a barrier to
provisioning though.)  It's also the case, IMO, that specification of a
new RR's RDATA is likely to be more formal than specification of a TXT
RR's payload format, but I've no data to back this up, and my preference
for formal specification is arguably just that: a personal preference.

Nico
--

---

From: Patrik Fältström <[email-masked]>
Date: Mon, 5 Apr 2010 21:43:14 +0200

On 5 apr 2010, at 21.34, Dave CROCKER wrote:

> While a new RR is the cleanest approach for extending DNS response data semantics, it has been a barrier for 20+ years.

I think we should be able to overcome the issues with new RR types. Just because it was hard say before year 2000 does not imply it is (as) hard today. I.e. please be careful with arguments like these.

That said, prefixes of owner names is one way of "expanding" the DNS, although it has some issues.

Se RFC 5507.

5507 Design Choices When Expanding the DNS. IAB, P. Faltstrom, Ed., R.
     Austein, Ed., P. Koch, Ed.. April 2009. (Format: TXT=44045 bytes)
     (Status: INFORMATIONAL)

   Patrik

---

From: Dave CROCKER <[email-masked]>
Date: Mon, 05 Apr 2010 12:53:22 -0700

On 4/5/2010 12:40 PM, Nicolas Williams wrote:
> IMO, not really.  The main issue is that the format of the text in a TXT RR's
> RDATA is dependent on the domainname of the RR.  This complicates...
> referential integrity in provisioning.

Please explain how.

There is a dependence onthe underscore name, not the root name.  Note that the
alternative is dependence on the RR "name".  It's easy to construct software
which makes the difference large and insurmountable, but it's also easy to
construct it in a way that isn't.

So, what makes one worse or the other better?

  (It's not a barrier to
> provisioning though.)  It's also the case, IMO, that specification of a new
> RR's RDATA is likely to be more formal than specification of a TXT RR's
> payload format,

You think the method of labeling affects quality of the specification writing?

I'll disagree.

On 4/5/2010 12:43 PM, Patrik Fältström wrote:
> I think we should be able to overcome the issues with new RR types. Just
> because it was hard say before year 2000 does not imply it is (as) hard
> today. I.e. please be careful with arguments like these.

I think we should too.  But a long track-record of failing to do that cannot 
realistically be ignored.

The point is not that it was hard 20 years ago, it is that it has /continued/ to 
be difficult.

Note that I started by citing the latest assurances that all was now well with 
the use of RRs and an experience DNS-o-phile provided a counter-view.  I'm 
merely trying to take that expert input and look for a way to work with it.

Feel free to counter Andrew's details.

But separately, please note that I asked a question about relative performance 
and functionality, not about personal preference.  I am simply looking for an 
honest technical comparison of two design approaches.

Technical.  Not religious.

> Austein, Ed., P. Koch, Ed.. April 2009. (Format: TXT=44045 bytes) (Status:
> INFORMATIONAL)

Yes.  Thanks.  I should have included that citation in my query.

d/

-- 

   Dave Crocker

---

From: Joe Hildebrand <[email-masked]>
Date: Mon, 05 Apr 2010 13:57:00 -0600

> use of underscore names meaningfully inferior to use of new RRs?

The underscores themselves aren't a problem, but the tight coupling with
transport protocol in the case of SRV can be an issue, as we've been
discussing on another thread.

If I was to do "SRV-lite" for XMPP federation, I might do a TXT record at
_xmpp-server._lite.example.com that returns a list of records each of which
has priority, transport, port, and host IPv4/IPv6.  You SHOULD NOT return
two records with the same priority, but if you do, the recipient can pick
either.  All of the load-balancing stuff in SRV is a pain to implement, and
it's rarely used due to the availability of network-layer load balancing.
I'd recommend skipping an extra A/AAAA lookup, and just return a different
TXT in priority order for that case as well.

Moving out of _tcp has the extra side-benefit that when you're deploying
into a broken Microsoft AD DNS, your results don't have to include all of
the AD servers every time, which means you've got a shot at staying under
one MTU.

If I'm a deployment that implements XMPP over SCTP, TCP, and HTTP (heh.
Only mostly kidding about this), I just have different TXT records, in order
of the deployment's preference.  The client runs through the list in
priority order, using the draft-wing-http-new-tech approach, but skipping
the methods it doesn't implement or can't access at the moment.

-- 
Joe Hildebrand

---

From: Nicolas Williams <[email-masked]>
Date: Mon, 5 Apr 2010 15:02:25 -0500

> >referential integrity in provisioning.

It means that you need to examine the name of the RR to decide how to
decode the RDATA so that you can model the object correctly in your
RDBMS.  Nothing spectacularly hard.

> So, what makes one worse or the other better?

Nothing (remember, I started my reply to that same question with "IMO,
not really").

> You think the method of labeling affects quality of the specification writing?

Possibly.  Some authors will use ABNF, some will use languages much less
formal, or just English to describe the payload.

Nico
--

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 5 Apr 2010 17:35:43 -0400

> the question for a reason :)

Oh, come on.  Now what you're asking for is that old software be
infinitely forward-compatible.  Moreover, the plain fact is that the
DNS is not a relational database, and it is nowise plain to me that
every possible use of every DNS RRTYPE is going to fit properly into a
referential model, even if you really wanted it to.  TXT records
overloaded to handle everything under the sun won't maintain
referential integrity either except in the boring sense that your
tools can claim it's there.  All the important checks that you'd
actually want are going to be hidden inside the semantics of the TXT
record as privately defined by some other trick (like underscore
labels).  I find it passing belief that people who'd be exercised
about missing referential integrity for opaque datatypes would be just
fine with a conflation of data value and data typing.  (On the other
hand, I once consulted for someone whose "database experts" had
written mainline runs-80%-of-the time queries with 14 -- no
exaggeration -- CASE statements in them.  "This fit the data model."
I know there's a hammer around here somewhere . . .)

> In this case the alternative to new RR types is parallelized probing.

Or both, of course.  You try one, see whether it's working for you,
and if not you fall back to the thing you don't like to do.  Why do we
have to chose?

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 5 Apr 2010 17:41:04 -0400

> semantics, it has been a barrier for 20+ years.

[…]

> Underscore-based naming . . . has fared considerably better

Note that in my original remarks that (regrettably, I now think)
revived this discussion, I included an example where underscores
wouldn't help either: the stupid middlebox just thinks that it is only
ever replying to A queries, so it doesn't matter what you ask it.  The
same is, alas, apparently true (I've heard this before) in the VB
APIs, so you just can't get anything useful out of the DNS as far as
the application is concerned.

Arguing about whether to use a new RRTYPE or whether to lard up
existing RRTYPEs with additional semantics using underscore names is,
in my view, a distraction from that more central problem: some
implementers of things that stand between your application and the DNS
have made foolish simplifying assumptions about what you might want. 

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Nicolas Williams <[email-masked]>
Date: Mon, 5 Apr 2010 16:49:30 -0500

> infinitely forward-compatible.  Moreover, the plain fact is that the

No, I'm stating that there exist barriers to deployment of new RRs.

> referential model, even if you really wanted it to.  TXT records

That does not mean that DNS data that one maintains cannot be modelled
as an object-relational database.  I supported (for internal customers)
a product that did just that.  Excluding NAPTR (which always gives me
headaches), probably incorrectly, I can't think of any RR that can't be
generated from an RDBMS that models DNS data as relational.

> fine with a conflation of data value and data typing.  (On the other

I'm Ok with TXT RRs provided there are clear rules for determining the
format of the text payload given the RR name.  (I'm also Ok with
free-form text payloads if they are intended for human consumption.)

> I know there's a hammer around here somewhere . . .)

The database I mentioned had no such problems.  (It's open source now,
under GPL, so you can always go look at it yourself.  It's getting dated
now, but it's still in use.)

> and if not you fall back to the thing you don't like to do.  Why do we

For all the reasons given earlier.

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 5 Apr 2010 18:07:01 -0400

> free-form text payloads if they are intended for human consumption.)

I'm sure you've heard this broken record before, but inside the
protocol, that requirement can't be met.  The very nature of a TXT
record is that there is no restriction on what you might put in it.
So it's now a matter of registration policy what the RDATA of the TXT
record at a given owner name may be.  So now you have two places that
control the allowable values for the RRs you allow in: there are the
protocol-defined rules about what goes in a resource record, and then
there are localized rules (some of them in protocol documents) that
specify special handling of some of those protocol-defined resource
records, if the data in them is just so.

Now, from a practical point of view, of course, we can do that, filthy
hack though it be.  I rather think Mssrs Codd, Date, and the rest of
them would start to get a little annoyed if you told them that, in
order to find out just how you're supposed to handle a given row of
data inside the database you have to look at a particular datum, and
draw some conclusions from that.  In a typed database like the DNS, it
ought to give data people the heebie-jeebies to hear that the solution
to datatype extensibility is to subtype all the data and overload one
datatype.  _Especially_ when overloading that datatype won't allow
everyone to use it (because they can't see anything except A or maybe
AAAA records) and when there is in fact an existing datatype extension
mechanism.  Also, of course, just to make things more entertaining,
the underscore mechanism has been used in two slightly different ways.

> For all the reasons given earlier.

I must have missed the argument where someone explained why an
application couldn't have both mechanisms available, and make
intellingent decisions about what it's going to be able to do
depending on its environment.  (I mean, every single network
application for at least the past 10 years has had to cope with NAT
this way.  No?)

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Nicolas Williams <[email-masked]>
Date: Mon, 5 Apr 2010 17:12:41 -0500

> record is that there is no restriction on what you might put in it.

If a protocol says that _foo._bar.<domainname>. IN TXT must have a text
payload with such and such format, then that is what will be expected by
the clients that look for TXT RRs named _foo._bar...  Sure, they might
get garbage, but how is that not true of any other RRs?  (e.g., I might
get an IP address in an A RR RDATA, and it might be well be a useless
one.)

> the underscore mechanism has been used in two slightly different ways.

I don't think of DNS as a database.  I think of DNS as serving data
generated by a database (and also updated dynamically).

> application for at least the past 10 years has had to cope with NAT

It adds unnecessary complexity to solve... what problem?  Probing seems
like good enough to me.

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 5 Apr 2010 18:55:02 -0400

On Mon, Apr 05, 2010 at 05:12:41PM -0500, Nicolas Williams wrote:

> the clients that look for TXT RRs named _foo._bar.

But it is not what the DNS protocol requires.  That is, it is
syntactically correct for someone to put something else in the RDATA
for that TXT record, and any DNS server will serve it up.  This is a
fantastic vector for attacks as a result.

> get an IP address in an A RR RDATA, and it might be well be a useless

But you won't get random binary data poisoning caches, because
well-built caches will detect a malformed resource record and refuse
to cache it.

> generated by a database (and also updated dynamically).

Hrm.  I suspect you and I have different meanings of the word
"database".  DNS is certainly not a relational database.  But I think
you'll find that the DNS protocol documents and people working on it
have commonly referred to the DNS as a distributed database, or else
"having" a distributed database (this is not the same thing as the
"database" in your usage above, which in my reading is approximately
the zone data or, in old-fashioned terms, the "zone file" at the
"master server").

> It adds unnecessary complexity to solve... what problem?  Probing seems

I thought the problem was that probing was maybe too costly in at
least some circumstances.  That was the argument I thought I heard
from Dan Wing.  Though I get the impression he gave up and will fall
back to probing.  So maybe you're right: this really is a waste of
time.  Sorry.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Nicolas Williams <[email-masked]>
Date: Mon, 5 Apr 2010 18:01:16 -0500

> fantastic vector for attacks as a result.

Sure.  But that's not important.

> well-built caches will detect a malformed resource record and refuse

You should expect random binary data and not let it poison you.

> the zone data or, in old-fashioned terms, the "zone file" at the

Yes, but it's so... basic.

> back to probing.  So maybe you're right: this really is a waste of

I'd agree except that if the middle-boxes-drop-SCTP problem is real (and
I believe it is, but I don't use SCTP much, so I don't know from
first-hand experience) then only probing will help you.  Caching will
more than alleviate the extra probing overhead in most cases, and
servers had better be able to cope anyways.

Nico
--

---

From: Dave CROCKER <[email-masked]>
Date: Mon, 05 Apr 2010 16:13:31 -0700

> fantastic vector for attacks as a result.

First, the uses of TXT records at an underscore leaf are specified by additional 
documents.  The fact that the core DNS engine doesn't care is largely the reason 
that the underscore convention is appealing.

As for being an attack vector, it is the business of the additional documents to 
explore such possibilities.

If one goes wandering into underscore name space without having any idea what 
further specifications apply, well, that would be unfortunate.

d/
-- 

   Dave Crocker

---

From: Patrik Fältström <[email-masked]>
Date: Tue, 6 Apr 2010 07:45:41 +0200

On 6 apr 2010, at 01.13, Dave CROCKER wrote:

> First, the uses of TXT records at an underscore leaf are specified by additional documents.  The fact that the core DNS engine doesn't care is largely the reason that the underscore convention is appealing.

The core DNS engine do not care about new RR types, as it can handle unknown RR Types (as compared to say 10 years ago). This is why I reacted on your comment on "it has been hard for 20 years", as it was hard 10 years ago. It is not hard to add new RR types now.

> As for being an attack vector, it is the business of the additional documents to explore such possibilities.

Too easy to say. Just look at the various attacks we have regarding layering in other protocols.

> If one goes wandering into underscore name space without having any idea what further specifications apply, well, that would be unfortunate.

The main problems with underscores are two:

1. You can not have two records with the same owner and same class. You want that sometimes, as there are RR Types that do by definition have the same owner as already existing records. Such as the DNSSEC records.

2. You can have a zone cut at the periods, and in some cases you explicitly do not want a zone cut between records. See the trouble with the NS record sets for Microsoft AD that create quite some problems at the moment.

The best path forward is to define a new RR Type, and reusing TXT is just not good at all.

Using an underscore name is of course the only selector you have IF records have the same RR Type as SRV, so that is "sort of ok", but then the RR Type is still structured.

One should NOT do neither (no underscore, no special RR Type).

    Patrik

---

From: Patrik Fältström <[email-masked]>
Date: Tue, 6 Apr 2010 08:00:53 +0200

> RRTYPE in question was a Bad Idea.  Didn't matter.  They met the

Correct.

> authority or infrastructure-grade recursive nameserver.

Those nameservers that do not support unknown resource record types (or the new RR Types of course) should be replaced for other reasons. The number of attacks that have been invented the last 10-15 years is quite significant. And various projects exists in the world to ensure that for example Bind version 8 or earlier is removed from the net.

> replaced because of how they break DNSSEC.

This of course has impact on the new RR Type, but also lookup of other RR Types that are not A. The software dnsmasq for example is widely deployed and do not support TXT, SRV etc without changes in the configuration and in some cases patches.

But, those things will not be changed if not IETF do not only say for real that unknown RR Types are needed (IETF has already said so) but also start requiring that for good deployment.

> harder than just implementing some minimal resolver yourself.

If one want to look up anything else than standard address records, you must almost always dive down one layer deeper in the resolver code -- or like Firefox include your own resolver.

This INCLUDE the TXT record.

I.e. the code I have written during the years do not make any difference in the code itself whether I want TXT, SRV or a new RR Type I write myself.

> me a compelling argument to this effect over lunch in Anaheim, and I

Application people do have to care, because as soon as you want to do something more than call libcurl and fetch some blob over HTTP, you have to do quite a lot by yourself.

But, that is not hard, and definitely not the hardest part of any software one is writing. Specifically when using the resolver library that is included in the operating system.

I would add 6. here, and that is the provisioning that many people are talking about. Provisioning of new RR Types might be confusing for some people. *Specifically* people that are active in the IETF and similar organisations because those individuals are the (few) ones that still edit the zone files by hand in a text editor, and understanding the encoding of unknown RR Types is not easy. It is not even easy to find the documentation! I think the people that come up with new RR Types can make it easier for people by giving examples. I.e. the following is not very easy to understand is it:

paf> dig frobbit.se TYPE64999 +short
\# 15 763D73706631202B6D78202D616C6C

For provisioning systems, people have to add support for new RR Types, or at least support unknown RR Types just like I show in the example above.

There is a reason why the DNS world started requiring unknown RR Types say 10-15 years ago, and I do not like seeing people still fighting against when we are so close to get it there.

   Patrik

---

From: "Martin J. Dürst" <[email-masked]>
Date: Tue, 06 Apr 2010 18:58:24 +0900

> new features is "impossible".

I think quite a bit of this may have to do with the fact that in most 
cases, seen from the application involved, DNS is a secondary 
technology. As an example, to deploy DKIM, one needs to deploy the DKIM 
software itself (in mail servers and possibly clients) and one needs to 
decide and deploy some way to put the necessary information into DNS. It 
means pushing up two hills, one that's familiar to the application 
people, and another (DNS) that's not.

Regards,   Martin.

-- 
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp   mailto:[email-masked]

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 6 Apr 2010 09:48:15 -0400

> You should expect random binary data and not let it poison you.

Of course.  And a good datatype structure allows you to do early,
primitive and fast checks without ever having to look at the elements
of the data structure.  The underscore-plus-local-rules trick means
you actually have to parse part of the data before you can even check
the data for syntactic correctness. 

> Yes, but it's so... basic.

It may be basic, but it seems to me it is a leading candidate for
"most successfully deployed distributed database", and I think we
shouldn't take that lightly.  There are _plenty_ of things to hate
about the DNS, but I don't want to make those things worse by refusing
to use the features it offers.

> servers had better be able to cope anyways.

I agree with this, but I think there are really two problems we have
to solve here.  Part of the reason I'm advocating some pressure from
applications actually to use the features already available in the DNS
is because I think they help us solve a long-term problem.

The short-term problem is deployment of new things (such as SCTP).
For each such case, we have two options: probe or lookup.  Now,
because of the existing breakage, we can't rely on lookup.  So we're
stuck probing.

The problem is that if we _only_ ever stick to probing this way, then
we'll never get to the point where lookups will work.  So, we'll keep
adding more probes.  That seems to me to be a bad long-term plan, so I
think it would be nice to start using lookups where we can, in order
to create a virtuous circle in which more lookups become usable as
more people want to use them.

Best,

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: "Dan Wing" <[email-masked]>
Date: Tue, 6 Apr 2010 11:00:34 -0700

> From: Dave CROCKER [mailto:[email-masked]] 
> Sent: Sunday, April 04, 2010 10:52 AM
> To: Dan Wing
> Cc: [email-masked]
> it to the list of
> different URIs is clearly the wrong result. 

It sounds like you were reading a -01 (or later) of
draft-jennings-http-srv.  The later versions do propose
a different URI.

However, that was not what I cited. 

Cullen's original proposal (-00) to which Ted (and John) were
commenting, did _not_ have a different URI.  It uses the same
URI.  Cullen's draft-jennings-http-srv-00 was submitted July 5, 
2008 and Ted and John's comments were on July 7.

> Having an 
> application protocol run 

Sure, that can easily added to the I-D.

Big difference is that:

  * SRV requires a separate DNS query for each transport 
    a client supports.
  * the earlier proposal to use SRV (draft-jennings-http-srv-00) 
    for HTTP was rejected by Ted Hardie and John Klensin.

-d

---

From: Nicolas Williams <[email-masked]>
Date: Tue, 6 Apr 2010 16:55:05 -0500

> the data for syntactic correctness. 

Not at resolvers, and not at caching DNS servers.

> to use the features it offers.

I understand that.  The point is that it's too easy to edit one zone
file and forget to make a corresponding edit elsewhere, or even to make
two corresponding edits in the same zone file.  Which is why people
layer provisioning tools on top of DNS.  Referential integrity matters.

(15 years ago in my then sysadmin era a huge proportion of the problems
I dealt with on a daily basis were referential integrity problems in
name service data.  DNS, NIS, even X.400 (don't ask).  When we deployed
UName*It all those problems went away.  Note: this all happened at a
different employer than my current one.)

To the "when all you have is a hammer..." comment, my riposte is that
one does still, from time to time, come across bonafide nails needing
hammering.  Layering a DB over DNS is a time-tested solution.

> is because I think they help us solve a long-term problem.

And I agree with that.  I throuroughly dislike the most portable (in the
sense of reliably being available) resolver APIs.  Their biggest
problems, besides being unwieldy and unnecessarily hard to use, is the
lack of async, MT-safe/hot support and support for new RR types.
Additionally, some RR types (e.g., SRV) deserve RR type-specific APIs.

If good DNS resolver APIs existed and were widely adopted we could solve
many more problems more easily.  And we'd see resistance to new RR types
on account of "deployment infeasibility" drop dramatically if not
entirely.

> because of the existing breakage, we can't rely on lookup.  So we're

Sadly, yes.  On the plus side: we agree ;)

> more people want to use them.

I agree, with the caveat that even if lookups worked, if probing is
faster then it could still win out.  For example, if this new XPORT RR
requires multiple DNS queries (one for the XPORT RR, then one for at
least one, if not each name listed in it) prior to connecting, then
that'd already be slower than probing (where you'd do one RR lookup + N
concurrent connects).

Nico
--

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 7 Apr 2010 11:15:49 -0400

> on account of "deployment infeasibility" drop dramatically if not

Ok, this is useful feedback.  Thanks.  If this or that language had
such an extended API available, do you think people would start to use
it instead of the supplied one?  That is, suppose that there were
alt-resolver libs for (say) Python, VB/.Net, or pick your favourite
language (and that they were well-tested, reliable, fast, and had some
evidence of being supported over the long term).  What are the chances
people would pick them up?

A
-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 7 Apr 2010 10:37:31 -0500

> people would pick them up?

I live in the world of C.  Decent resolver APIs for C are crucial
because: a) C is used for much of the user-land of many an operating
system, b) most high-level languages have foreign function call
interfaces suitable for calling C functions, therefore you'll have the
most reach with C.  Async operation means using something like libevent,
which is very portable (Windows, Solaris, Linux, *BSD, ...) and bundled
in most operating systems nowadays.

MT-safety can probably be achieved without using threading and
synchronization APIs if you: a) ensure there's no shared global state,
b) use async I/O for everything, c) declare that objects are accessible
by one thread at a time and leave synchronization to the caller, d) open
separate sockets for each instantiation of the library, e) allow
multiple instantiations of the library in one process.  Done that way
you'll get MT-hot for free.  A hint: use the libevent interfaces that
take an eventbase *.

I know, C is probably not what you were thinking of.  But if you want to
change the world, then that's where you'll start.

Nico
--

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 07 Apr 2010 09:23:51 -0700

On 4/5/2010 10:45 PM, Patrik Fältström wrote:
>> First, the uses of TXT records at an underscore leaf are specified by
>> additional documents.  The fact that the core DNS engine doesn't care is
>> largely the reason that the underscore convention is appealing.
>
> The core DNS engine do not care about new RR types, as it can handle unknown
> RR Types (as compared to say 10 years ago). This is why I reacted on your
> comment on "it has been hard for 20 years", as it was hard 10 years ago. It
> is not hard to add new RR types now.

Unfortunately, the deployed base of client resolver software needs to be counted 
as part of the "DNS engine", if we are going to evaluate end-to-end viability. 
That one or another component of the DNS works well is nice, but what is 
required is end-to-end behavior.

Client software /does/ care about RR types.  It's the reason a many/most client 
will handle the underscore model but not the new RR model (assuming the client 
will handle TXT RRs and, more recently, SRV RRs.)

And as Andrew has pointed out, middleboxes also need to be counted as part of 
the core DNS engine, since they get in the way.

> Too easy to say. Just look at the various attacks we have regarding layering

Where else should the attack analysis be put, except with the specification that 
opens the vector?

Other than that venue for the analysis, where and when should it be performed?

>> If one goes wandering into underscore name space without having any idea
>> what further specifications apply, well, that would be unfortunate.
>
> 1. You can not have two records with the same owner and same class. You want
> that sometimes, as there are RR Types that do by definition have the same
> owner as already existing records. Such as the DNSSEC records.

I should probably fully understand what you've said here, but I'm sorry to say I 
don't.

First, I do not know what mean by the "owner" of a record.  And I particularly 
do not know what you mean about having two records with the same owner and same 
class, in terms of affecting the choice between new RR and underscore-based 
enhancement.

Second, the premise behind the underscore model is that it defines a leaf space 
that is more constrained than an arbitrary DNS leaf.  If you find a TXT record 
underneath a DKIM node, it's for DKIM.  It's not for a vcard or for SPF or for 
anything else.

So the question is whether your "sometimes" observation applies under these 
administratively constrained nodes.

> 2. You can have a zone cut at the periods, and in some cases you explicitly
> do not want a zone cut between records. See the trouble with the NS record
> sets for Microsoft AD that create quite some problems at the moment.

Hmmm.  I suspect your are using the word "record" differently than I understand 
it for the DNS.  I thought it meant an RR instance.  Wikipedia certainly seems 
to think this, too:

    <http://en.wikipedia.org/wiki/List_of_DNS_record_types>

Since zone cuts are limited to name-space boundaries, and cannot occur in the 
middle of a leaf node, I do not understand how the problem you are describing 
can occur.

Beyond this, perhaps you are raising the concern that a zone cut could 
theoretically occur below the base domain name and the associated underscore name?

While, yes, the core DNS allows that, it would be remarkably silly and 
counterproductive for anyone to administer their DNS that way.

In other words, yes, use of underscore naming modifies the "effective" semantics 
of the DNS.  That's a pretty scary fact, but most in-place modifications that 
lay new semantics on top of old semantics are scary.

The reactions to the design of MIME for email were similarly distinctive.  In 
terms of clean computer science design, layering enhancements like this can be 
remarkably ugly.  Such is the constraint of modifying a running system.

> The best path forward is to define a new RR Type, and reusing TXT is just not

Speaking of "too asy to say", I think this assertion qualifies.  We still do not 
see the path for end-to-end viability.

I'm not arguing cleanliness. I'm arguing operational viability.

> Using an underscore name is of course the only selector you have IF records
> have the same RR Type as SRV, so that is "sort of ok", but then the RR Type
> is still structured.
>

ack.

d/
-- 

   Dave Crocker

---

From: Simon Perreault <[email-masked]>
Date: Wed, 07 Apr 2010 12:36:37 -0400

> you'll get MT-hot for free.  A hint: use the libevent interfaces that

I also have been looking for exactly this kind of library very recently. 
I didn't find any open-source one that would fit the bill (C, async, 
MT-safe, and support for new RR types), so I ended up writing my own 
with the minimal set of features I needed.

If somebody with a clue wrote a complete one and made it available under 
a BSD license, the world would be a better place.

Simon
-- 
NAT64/DNS64 open-source --> http://ecdysis.viagenie.ca
STUN/TURN server        --> http://numb.viagenie.ca
vCard 4.0               --> http://www.vcarddav.org

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 7 Apr 2010 11:43:08 -0500

On Wed, Apr 07, 2010 at 09:23:51AM -0700, Dave CROCKER wrote:
> Unfortunately, the deployed base of client resolver software needs
> to be counted as part of the "DNS engine", if we are going to
> evaluate end-to-end viability. That one or another component of the
> DNS works well is nice, but what is required is end-to-end behavior.
> 
> Client software /does/ care about RR types.  It's the reason a
> many/most client will handle the underscore model but not the new RR
> model (assuming the client will handle TXT RRs and, more recently,
> SRV RRs.)

+1

> And as Andrew has pointed out, middleboxes also need to be counted
> as part of the core DNS engine, since they get in the way.

I worry less about middle boxes than about end node resolver
functionality.

Yes, broken middle boxes will mean that probing will continue to be
necessary: end node resolvers can workaround broken middle boxes, yes,
but they shouldn't want to, and discovering the presence of broken
middle boxes may require a timeout, which is Bad News for user-facing
apps.

But pressure to fix broken middle boxes won't materialize until until
end node resolver support for new RR types does.  Resolver support has
to come first.

(There's relatively few DNS middle boxes from any one point on the
Internet, and many are run by providers that have business relationships
with their users or other incentives to keep them happy.  Broken middle
boxes are one killer app from being fixed, but the killer app will have
to come first.)

> >One should NOT do neither (no underscore, no special RR Type).

Well, if probing is deployed and works then we may well do neither.  I
see no need to do more than what works.  (Yes, what works may seem
architecturally impure, but when you consider broken IP middle boxes,
architectural purity goes out the window.)

Nico
--

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 7 Apr 2010 11:50:00 -0500

On Wed, Apr 07, 2010 at 12:36:37PM -0400, Simon Perreault wrote:
> If somebody with a clue wrote a complete one and made it available
> under a BSD license, the world would be a better place.

Some intellectual property is best protected, some is best given away.
If the DNS had been patented back in 1986, and licensed illiberally and
not freely (as in $0), then we'd still have been copying hosts files
until fairly recently.  Some technology is useless unless everyone
adopts it, while some technology gives one an edge especially when not
everyone has adopted it.

DNS _resolver_ technology, IMO, falls into the "best given away freely"
bucket.  But someone has to fund such work, and must consider the cost
of opportunity.  This is why we're stuck with ancient resolver
technology that's widely deployed even though much better resolvers with
much narrower applicability (because not written in C for C and non-C
apps, or because not liberally licensed) may be available.

It's not hard to predict that this state of affairs will continue
indefinitely.

When people say that "it's very hard to get new RR types widely
deployed" what they mean is something like the above (plus broken middle
box problems).

Nico
--

---

From: Joe Hildebrand <[email-masked]>
Date: Wed, 07 Apr 2010 11:13:19 -0600

Simon, would you be interested in just putting your code up on github with a
BSD or MIT license, and telling anyone who complains that they should just
contribute? Sometimes a starting point is all that's needed.

-- 
Joe Hildebrand