Subject: Why DNS?

From: "Fletcher Dunn" <[email-masked]>
Date: Sat, 3 Jul 2004 18:42:02 -0500

Forgive me if this question is misplaced.  I have been a student of spam and
email for years, but I have never officially been involved in the dicsussion
until now.

Basically I am wondering why the "D" in MARID.

To authenticate the sender, a recipient needs to obtain an official copy of
the sender policy.  One way to disseminate this information is through DNS.
But using DNS is a significant complication in implemtation.  (Does
everybody implement TXT records properly?  Isn't it sort of a kludge to
paste together multiple TXT records to hold lengthy sender policies?).  It's
a factor delaying the adoption of a standard for sender authentication.

If the sender policy basically boils down to an XML document, wouldn't
wouldn't a simpler solution be for the sender to deliver the sender policy
itself?  Obviously I don't mean that the sender would deliver it along with
the message - that certainly isn't secure, or optimal. The recipient would
look up "responsible domain"'s MX record (again, depending on exactly what
you want "responsible domain" to mean), and requests the sender policy from
the domain's SMTP server, perhaps using a simple SMTP extension like

SPOLICY <domain>

This places no additional role or burden on DNS.  It also more fairly
distributes the extra communications load requires to maintain this
information on those sending the most messages.  All changes required to
implement sender authentication are in one piece of software: the MTA.

Again, forgive me if this idea is one that was considered and rejected long
ago, I scanned the archives and didn't see the idea mentioned.

- Fletcher Dunn

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Mon, 5 Jul 2004 09:12:42 +0430

On Sat, Jul 03, 2004 at 06:42:02PM -0500,
 Fletcher Dunn <[email-masked]> wrote 
 a message of 33 lines which said:

> and requests the sender policy from the domain's SMTP server,
> perhaps using a simple SMTP extension like
> 
> SPOLICY <domain>

What if the sender is behind a relay (a sender with no permanent
connectivity or a sender using UUCP or X400)? Remember that email does
not require a end-to-end link with SMTP and this is one of its great
strengths.

Surely, the relay managers would not like to have to relay policies
for their customers.

---

From: "Fletcher Dunn" <[email-masked]>
Date: Mon, 5 Jul 2004 17:34:19 -0500

>not require a end-to-end link with SMTP and this is one of its great

How does such a person *receive* email?  Obviously, they must have *some*
sort of permanent connection.  So the answer to the above scenario is this:

The sender policy for domain X is available at the server(s) where domain X
receives email, i.e. the server(s) pointed to by the MX record for that
domain.

This is my suggestion in a nutshell.  Since my original email was brief (and
contained numerous typos!  doh!), please let me state my case once more, in
a bit more detail.  Before I begin with the "why," let me include a simple
example of how it would work, in case it isn't obvious:

1.) smtp.recipient.com receives an incoming mail and determines that the
"responsible domain" is sender.com.
2.) smtp.recipient.com checks for a locally cached policy from sender.com
and finds none.
3.) smtp.recipient.com looks up sender.com's MX record, and contacts
sender.com at the SMTP port exactly as if he were going to send an email to
sender.com.
4.) Using a new SMTP extension, smtp.recipient.com requests the sender
policy for sender.com.  ("SPOLICY sender.com" ??)
5.) sender.com's mail server replies with the sender policy.  (Basically an
XML document?)
6.) smtp.recipient.com consults the sender policy to take appropriate action
regarding the incoming email.

You'll notice that my suggestion only differs from current proposals in
steps 2..5.  Steps 1 and 6 are (tricky) steps that aren't unique to my
suggestion.  I'm not sure if there ever was a discussion of "MAR" outside
the context "ID," or if was always assumed that DNS was the best way?
Clearly, to authenticate the MTA, DNS will have an important role in the
transaction between two unrelated parties - it serves as a mutually-trusted
third party.  But that doesn't mean it needs to actually *deliver* the
authentication records, it only needs to authenticate them.

Using DNS to deliver the policy is a hurdle toward widespread acceptance.
Here are some advantages of storing the sender policy at the sender's mail
server:

1.) It only requires changing one piece of software: the MTA.  Assume for a
moment that we use DNS TXT records to deliver the policy, and by a fortunate
blessing all the DNS servers on the Internet implement the feature properly,
even though it is a relatively obscure feature.  Even if we assume this, it
is still highly likely that there will be changes necessary for performance
reasons.

2.) More importantly, using DNS requires the entire Internet to participate.
By this I don't mean that all the mail servers must participate or that
everybody must publish a sender policy - that's not the case with any sender
authentication mechanism.  I also don't mean that it couldn't function *at
all* without everybody's participation - trial implementations already
exist.  What I mean is that the DNS of the entire Internet must work
properly with these new features being proposed, if we're going to use DNS
to distribute the policy *reliably* and *efficiently* with widespread use.
This is a big hurdle.

3.) We've just increased our "minimum system requiremenets" for a DNS server
on the Internet.  Will current DNS servers require hardware upgrades to be
able to store the extra information?  (In addition to the software upgrades
that will probably be necessary.)  Are we suddenly going to require more DNS
servers to be added to the infrastructure?

In summary, if we use DNS to distribute MTA authentication records, it is
likely that widespread use will create a need for significant software
and/or hardware upgrades to be made to a large percentage (majority?) of the
DNS servers on the Internet.  Until these upgrades are made, any MARID
system will malfunction in the worst case and function inefficiently in the
best case.

There are also philosophical problems with using DNS:

1.) Currently, DNS's primary role is to locate things.  It's the map of the
Internet, matching up names and IP's.  Should we now task DNS with
delivering a document related to a particular Internet application (mail)?
Should we distribute an FTP server's list of mirror sites or policy for
anonymous logins, or an HTTP server's policy for caching documents?  Of
course, those examples seem ludicrous - if we want to know that information,
we get it directly from the server.  The same applies to email - it's more
appropriate for mail policy information to be distributed via the mail
system.

2.) Any sender authentication mechanism will require some additional
communications overhead for the recipient to obtain the sender policy.  If
we obtain the policy directly from the sender, then this extra burden is
more "fairly" distributed.  That is, it is more directly proportionately
borne by those sending (and receiving) the most email - there is no extra
overhead for those not involved in the transaction.  Any sender
authentication system will increase the "cost" of mail slightly, but my
proposal increases the cost of *sending* a mail specifically.  In contrast,
using DNS to store the policy increases the burden on recipients more than
it does on senders!  (This is particularly interesting considering the
anti-spam proposals suggesting that we all agree to *purposefuly* increase
the cost of email - for example by requiring the sender to perform some
unnecessay but time-consuming computation, or perhaps if we waited 10
seconds from the time we receive a mail connection until the time we process
that mail connection.  These proposals make an interesting contrast to
discussions regarding how to avoid extra DNS checks in order to reduce the
delay in processing a message.)

3.) DNS is a distributed database.  For its current role as "map of the
Internet", this distributed nature is necessary for reliability and
performance.  But for the role of delivering a sender policy, I don't see a
distributed database as having any advantage over a non-distributed one.  I
don't need to broadcast my sender policy to everybody - the only people who
need know my sender policy are those to whom I'm sending mail.  It's not a
matter of privacy, of course, just of simplicity and efficiency.  Why spend
the resources distributing and storing the information when most people
aren't going to need it?  When it comes to the information currently held in
DNS, the answer to this question is, "We don't know in advance who will need
what information, so we send everything to everybody (at the top level, at
least)."  But for the sender policy, this is not the case - we know exactly
who will need the information, so there's no need for anybody else to burden
themselves with this information when they don't need it.  I realize that we
aren't actually "broadcasting" our policy, with every server in the world
storing the policy for every domain in the Internet.  However, the general
point remains that unrelated 3rd parties are unnecessarily involved in the
distribution and storage of this information.

All of the above is not to say that there aren't advantages of using DNS.
The fundamental disadvantage of my proposal is that we are placing an
additional burden upon senders to have this server up:

1.) What happens when the sender's mail server goes down?  Mail can't be
authenticated.  (Assuming a cached policy isn't available.)  However, this
really isn't a huge problem, or even a new one.  What if we wanted to reply
to their mail and their mail server was down?  We would retry a certain
number of times and then fail.  So the bottom line is, the new requirement
on mail senders is this: if you want to send a mail, you must have a mail
server pointed to by your MX record which can deliver the sender policy, and
it must be functioning AT THE TIME YOU ARE SENDING THE EMAIL.  This is
admitedly a new and extra burden on mail senders.  But it's certainly not an
unreasonable one - in fact, it's one most senders already meet, since their
receiving mail server will serve this purpose.  (One sticky issue is: what
should the exact behaviour of a recipient be who tries to obtain the sender
policy but cannot because the sender's server is down?  Reject the mail, and
let the sender try again?  Or maybe accept the mail temporarily, but not
deliver it until the sender policy has been obtained and the mail verified?
However, similar problems can arise if we use DNS, although presumably less
frequently.)

2.) What about people with "unusual" relationships with the mail system,
like people behind relays as pointed out by Stephane?  First of all, there's
never a question as to where the policy will be located - it's on the same
server where you receive mail.  So nobody who is capable of receiving mail
is excluded from sending mail.  However, the point is conceded that there is
additional administrative overhead to make it work.  If you have a
relationship with a 3rd party to send/receive your email, then this 3rd
party would need to provide you with some sort of mechanism to manage your
policy for your domain.  Yes, this is extra work for those involved in those
situations.  However, people in such "unusual" situations have extra
complications in any sender-authentication system, since they need to
authorize this 3rd party to send mail on their behalf.  It's going to be
more work for them to *create* their policy than it is going to be to
*distribute* it.  More importantly, if we use DNS, then almost *everybody*
will have to communicate with a 3rd party in order to distribute their
sender policy - their DNS nameservers.  Thus my proposal would actually make
it easier for most senders to publish their email policy versus publishing
it in DNS.

So, in summary:

1.) Basic argument against DNS: Using DNS to distribute the sender policy
entails a significant complication to implementation.  We place a burden on
every DNS server in the Internet, both an immediate burden (upgrading
software/hardware) and also an ongoing burden (carrying around sender policy
records).
2.) Basic argument against my proposal: Senders will have extra requirements
to meet, especially senders in "unusual" situations.

My feeling is that #1 is a much stronger argument than #2.

Again, please shoo me away politely if this idea has already considered and
discarded, or if I should ask this question elsewhere.  After all, if
serious consideration were to be given to my proposal, I suppose it would
require the name of the group to change!  (Please point me to the discussion
of "to DNS or not to DNS" - I'd be interested to hear the reasoning behind
chosing DNS, or what the plan is to overcome the obstacles I've mentioned.)

I really am a big fan of sender authentication.  I was about 15 pages into a
proposal for a public/private key system before Yahoo! announced DomainKeys.
I really believe that sender authentication is necessary, in fact it is
*urgently* needed.  I disagree with those who say that it won't have an
impact on spam - the main impact being the increased effectiveness of
blacklisting and (perhaps more importantly) whitelisting.

Anyway, I am a major fan of the "MAR" part of MARID.  I'd hate to see any
MARID proposal meet resistance from the Internet community once the
complications in implementation of the "ID" part are fuly realized.

- Fletcher Dunn

---

From: "Harold A'Hole" <[email-masked]>
Date: Mon, 5 Jul 2004 17:54:46 -0700

> is still highly likely that there will be changes necessary for
performance

This is the crux of the problem from what I can see.  I think finding DNS
that can correctly deliver TXT responses will be substantially higher than
you'll find MTAs that would support a new protocol exchange.  For example,
BIND supports TXT just fine, but SENDMAIL doesn't support such a protocol
now.

Therefore, this upgrade would take far longer to be adopted because it would
require people change their MTA rather than add a DNS record.

> to distribute the policy *reliably* and *efficiently* with widespread use.

Second, this doesn't seem to be the case.  In order to the MTA solution to
work, the receiving system already has to consult DNS to retrieve the MX
records.  But then it not only has to retrieve the MX records, but it has to
contact the MTAs behind those addresses.  With a DNS-only solution using
TXT, only the DNS must be consulted, but instead of returning MX, it returns
TXT, and there's no overhead of going to an MTA to do yet another protocol
exchange.

> servers to be added to the infrastructure?

Is that even a reasonable assumption?  It would be interesting to note how
much extra capacity a DNS system would need.  We host only a few main
domains (perhaps 20), with each having perhaps 5-10 subdomains.  Adding the
20 TXT records was not much of an issue, and as I said before, the
processing overhead shouldn't be much different since the query for the TXT
is simply exchanged for the query for the MX.

> appropriate for mail policy information to be distributed via the mail

This philosophical question seems more to the point for me.  Why should DNS
have to return SMTP authentication info.  Will others start to abuse DNS for
their various needs?  Of course, DNS already has a specialty task for email
as it contains MX records.  It doesn't have specialty records for HTTP or
FTP today.  Perhaps email is just that special!?

> it does on senders!

This doesn't seem true to me since the TXT record had to be queried from DNS
in one scenario, or the MX records from DNS in the other.

> distribution and storage of this information.

Third parties typically only store them according to their caching policies,
and the MTA solution would also have a caching policy, so the it's got to be
cached someplace.  Why have DNS cache the MX records and the MTA cache the
policies if there's a way to do it with just the TXT record being cached by
DNS and the parsed TXT being cached by the MTA?

> My feeling is that #1 is a much stronger argument than #2.

My basic argument against would be that it requires software changes on the
MTA to function.  The DNS TXT change allows you to participate by publishing
your policy before you actually change the MTAs to enforce them.  So, if you
don't care about receiving and checking policies, you don't have to, but
others can begin to check your policy without your having to install
upgraded MTA software.

Other than that, I see no real problem.  I believe that a pure MTA solution
would have been preferable beacuse I agree that using TXT is a kludge.  The
long term solution, assuming this sender-id change actually has positive
benefits of reducing spam, is to create something like you suggest since it
doesn't mess with DNS.

I do see more attacks against DNS coming, more spams directed at MTAs that
don't implement the receiving side of sender-id so they bounce like before
back to the invalid sender, more spam blasts from ephemeral systems and more
hijacked accounts on outbound SMTP servers so sender-id passes checking.

Harold