Subject: Backward compatibility with deployed SPF records (and choice of domain)

From: Roy Badami <[email-masked]>
Date: Sat, 26 Jun 2004 01:16:34 +0100

To address a couple of questions that Andrew Newton raised in his
concensus statement.

1. Placement of TXT records.

    The co-chairs find that the working group has not come to consensus on
    the use of a record prefix vs TXT records at the zone apex and has not
    reached consensus on how to address domains with wildcard MX records.

First off, I don't think anyone's suggested putting the records at the
_zone_ apex (ie the delegation point); I'm assuming that the chairs
have been spending too much time reading DNSSEC discussion on
namedroppers :-) The term 'zone apex' certainly seemed perfectly
reasonable to me on first reading, precisely for that reason...

The choice is: should the domain of the identity to be verified be
used as the owner name of the TXT record, or should the owner name be
a subdomain of that domain, constructed by prepending a suitable
prefix to the domain of the identity to be verified?

If we wish to support backward compatibility with deployed SPF records --
and I'll note that the statement is worded in such a way as to imply
the that the question is _how_ we will do this, not _whether_ we will
do this -- then of course placing the TXT record anywhere other than
at the domain of the identity in question places extra burden on
implementation by having to query for records in two places.

So I think that if we are going to seek backwards compatibility with
SPF then there should be an assumption towards putting the records in
the same place, all other things being equal.

In fact putting records in two different places whilst maintaining
backwards compatibility adds significant conceptual complexibility --
if a record might exist at either example.com or _ep.example.com then
what are the implications of someone forging mail from
[email-masked] ?

On to the second point, and how to achieve backward compatibility...

If we assume that the changes to SPF syntax and semantics are likely
to be relatively small between draft-mengwong-spf and the final
version of draft-ietf-marid-core (as is currently the case) then it
may well make sense for deployed SPF records to simply be interpreted
as if they were MARID records.

But there are differences between the two drafts; most notably the
identity being checked, and also a minor difference in the
specification of the mx mechanism.

There will, in a few weeks, very probably be a large number of systems
checking SPF records (as defined in draft-mengwong-spf) -- quite
possible a much larger community than the community that currently
publishes SPF records.  The Spamassassin team are preparing for a 3.0
release, which will support SPF checks (though admitedly only if
Mail::SPF::Query is also installed).

Given MARID records are similar, but not identical, to SPF records,
and that it is at least possible that there will shortly be a large
number of SPF checkers in real world use, I'd like to propose the
following strategy:

MARID should define syntax versions 1 and 2 (identified by v=spf1 and
v=spf2) as identical.  MARID-compliant implementations must interpret
both of these records identically, and must use the highest understood
version record if multiple versions are found (this is the same rule
as in SPF).

This gives a publisher three choices:

a. Publish a syntax version 1 record.  This is a valid MARID record
and will also be interpreted by the deployed base of SPF checkers
according to the semantics of draft-mengwong-spf (assuming it is
syntactically also a valid SPF record)

b. Publish a syntax version 2 record.  This is again a valid MARID
record, but will be ignored by the deployed SPF checkers.

c. Publish both.  In this case, given the rule of using the highest
version record that is understood by the checker, MARID checkers will
use the syntax 2 record, and deployed SPF checkers will use the
version 1 record.

Thoughts?

	 -roy

---

From: Greg Connor <[email-masked]>
Date: Fri, 25 Jun 2004 22:27:47 -0700

MOstly agreeing with Roy here, but I'm raising a couple of questions that 
could use extra pairs of eyes and feedback...

>     the use of a record prefix vs [unprefixed] TXT records

I think I would agree with Roy that allowing MARID-compliant MTAs to check 
the unprefixed name is desirable.

In other words, if MARID-Brand LMAP works substantially similar to 
SPF-Brand, let's go ahead and use their data.

One concern here is that we would want the semantics to be similar enough 
to not upset or surprise anyone who isn't expecting their data to be used a 
slightly different way, but I think we are able to do that.  (That 
certainly was the intent of the SenderID merged proposal)

In a straight race, I would probably root for the non-prefixed records to 
win, simply because the underscore might scare and confuse people (it was 
removed from early versions of SPF for mostly that reason).  I do 
understand the issue that the prefix is intended to address, but I don't 
think the research we have shows it is anything to worry about.  I think 
most users will not have TXT records of any other type, and of those that 
do, the records will mostly be small enough to coexist peacefully.  I don't 
have a strong preference in this area, except that backward compatibility 
would allow MARID to harness and redirect the momentum SPF has built up. 
(Of course we will be going through the process of getting our own RR so we 
will eventually have an escape plan)

Does anyone have a strong objection to using unprefixed txt records?

> 1b. [the working group] has not

Actually I think this one is pretty easy to fix... if there is a wildcard A 
or wildcard MX record already there, the site should also create a wildcard 
TXT record.

(I really wish we could go back in time and tell people they need to set up 
MX records for all deliverable domains and hosts and not fallback to A... 
any chance we could declare the fallback-to-A practice deprecated?  It may 
be officially outside our scope, but it really makes the job of publishing 
LMAP records a lot harder.)

Let me add this sort of related point...

1c. The group has not agreed on an alternate placement for TXT records 
other than the label exactly matching the (host part of the) identity being 
checked.  I.e. a MARID record at example.com does not cover 
[email-masked].

That means:  the domain owner should create LMAP records for ALL hosts in 
his domain that have A or MX records.  (Note that a wildcard TXT record 
does not cover names that have A or MX but no TXT, only names with no 
defined data at all)

Is this something we need to work on more, or should we leave it like that? 
We talked about looking for the "closest SOA name" and looking there, and 
applying the TXT record at that location to other subdomains that have no 
TXT of their own.

Another idea might be to have records set up like:
example.com.  IN TXT  "v=xxx1 a mx ptr ?all"   ; only for mail @example.com
mail1.example.com.  IN TXT "v=xxx1 a -all"     ; for @mail1.example.com
_all.example.com.   IN TXT "v=xxx1 -all"       ; for @*.example.com if no 
TXT

In this case I would totally support use of the underscore because this is 
a little bit of an advanced feature anyway, but it doesn't have to be an 
underscore to work.  Then again this could be a performance problem... if 
most domains don't have TXT records we would have to do a second query to 
find the SOA and a third to find the higher-level (either example.com or 
_all.example.com) so I don't want to push too hard for climbing up the 
chain if there isn't a lot of support for it.

Any feedback as to whether we should pursue this any further or just leave 
it as the domain itself only and no searching at higher levels?

> version record if multiple versions are found (this is the same rule

I think the records will be similar enough that we can honor the original 
publisher's intentions.  However, there's nothing really *wrong* with 
defining a new prefix.  MTAs will need to be revised a bit anyway to be 
MARID-compliant, and a TXT lookup will still get both records.

What would folks in the group think about defining our own prefix like 
"v=lmap1"?  I think this is only really needed in the off chance that the 
semantics are different enough for people to notice.  I expect that people 
will continue to publish "v=spf1" for a while until the new 
MARID-compatible MTAs are widespread, then they could publish either. 
Someone would really only want to publish both if they want one policy for 
SPF Classic and a different policy for MARID-Brand LMAP.

(I called it LMAP because that seems to be a better name for the protocol, 
and it leaves the door open for MARID-the-group to work on and publish 
other stuff.  The protocol is important but it's not the only important 
thing the group has worked on :)

--
Greg Connor <[email-masked]>

---

From: "Hector Santos" <[email-masked]>
Date: Sat, 26 Jun 2004 02:21:12 -0400

----- Original Message ----- 
From: "Greg Connor" <[email-masked]>
To: <[email-masked]>
Sent: Saturday, June 26, 2004 1:27 AM
Subject: Re: Backward compatibility with deployed SPF records (and choice of
domain)

> think the research we have shows it is anything to worry about.

Probably nothing, but there might be some ergonomic issues here.

The following was probably a customer misunderstanding, but recently I took
a customer survey to find out who has published SPF or MCEP (_EP.) domains
and how (access to DNS).  One customer provided the following reason for not
implementing the _EP host name for MCEP support.

----- Original Survey Message And Response ----- 

> We need to know the following:
>
> With WCSAP, it has the LMAP features, SPF and CEP.  Please check all that
> applies:
>
> [X] I have defined an SPF domain policy for my Domains in DNS
> [_] I have defined an _EP domain policy for my DOMAINS in DNS
> [X] I have not defined any for the following reason:
> Please state reason:  My DNS host rejected the _EP entry as an invalid
host name.  I contacted their technical support regarding this and they
stated that the leading underscore was a violation of the rules.  When I
pointed out that this was a requirement for CEP as defined by Microsoft they
said they would look into it.  Currently they have not gotten back to me.

> DNS manager:  You manage DNS as follows
>
> [_]  Internal DNS server
> [_]  Via my ISP using a DNS manager they offer
> [X]  Other

www.dnspark.net

--- Back to me ---

Like I said, I'm sure it was a customer understanding, but it is happen
once, so I'm sure it will happen more.  From my standpoint, it was a
non-issue for implementation and it did take alittle extra "documentation"
to explain it.

Anyway, I don't see it as an issue because the way I see it down the road,
at some point we will be adding a Setup Wizard and Configuration to help
customers create the DNS/MARID entry either automatically or as a Cut/Paste
item to be sent to their ISP/DNS server provider, maybe via email for
example.

This is actually not a bad idea.  Back in the fidonet days, before a new
mail server was allowed to be put online and connect to the network,  he had
to install the compliant software and and then send an direct email (called
netmail) to his local area network mail host (akin to an ISP today I guess)
with a fix set of host registration information that got him into the
network node list (akin to DNS).  The network host then would test the new
mail server for network compliance and only then was his "address" (domain)
added to the world wide network node list.

I am not in no way suggesting this, but these day, it will help the mail
operator from A to Z, from installing/upgrading  a system, finding a MARID
compliant ISP to register with, sending DNS/MARID information to the ISP to
setup, etc.  And if the new pressures from the industry asking the ISP to
take more responsibility take hold,  the ISP might want to do some level of
"validation" or background checking before adding the information to his DNS
servers.

In Fidonet, it was a requirement for network membership. For the Internet,
it would just be an optional necessity and prudent/smart business move to
help minimize security/spam issues.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

---

From: "Eric A. Hall" <[email-masked]>
Date: Sat, 26 Jun 2004 03:20:56 -0500

> Does anyone have a strong objection to using unprefixed txt records?

If the migration to a new RR requires queries for ALL (NOTE: this should
be avoided if possible), then it will be easier to implement if there is a
prefix. TXT and new RRs can both be fetched with a single query without
worrying about overload conditions from SOA, NS, MX, etc.

As long as the magic label is short enough, then there's really no reason
not to do it.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

---

From: Greg Connor <[email-masked]>
Date: Sat, 26 Jun 2004 01:57:46 -0700

> As long as the magic label is short enough, then there's really no reason

Hmm, I'm not sure if that was an objection? :)

Anyway, I thought one of Ed's points in his presentation was that adding a 
_prefix had its own set of problems, and wasn't recommended...  Or maybe it 
was just wildcards that had the problem, but as PHB pointed out, you can 
define a wildcard and the prefixed and non-prefixed versions will both 
work.  You can either have fine-grained control, or wildcards, but not both.

I don't see a strong technical reason why not, but I also don't see a 
strong reason reason we need it.  I think I agree with you that ANY-type 
queries should be avoided.

The reason _spf prefix was dropped from SPF was NON-technical, but still an 
issue we should consider... people didn't understand it, and some still 
believed underscore was illegal, etc.

---

From: "Eric A. Hall" <[email-masked]>
Date: Sat, 26 Jun 2004 04:17:13 -0500

On 6/26/2004 3:57 AM, Greg Connor wrote:

> The reason _spf prefix was dropped from SPF was NON-technical, but
> still an issue we should consider... people didn't understand it, and
> some still believed underscore was illegal, etc.

underscore is illegal for hostnames. service names are not hostnames.

---

From: Roy Badami <[email-masked]>
Date: Sat, 26 Jun 2004 10:42:20 +0100

>>>>> "Eric" == Eric A Hall <[email-masked]> writes:

    Eric> underscore is illegal for hostnames. service names are not
    Eric> hostnames.

I think there was a concern that many hosted DNS services do not allow
underscore anywhere.  I'm sure this was discussed on the SPF list some
time ago, but I can't remember if the concern was substantiated...

	 -roy

---

From: "Hector Santos" <[email-masked]>
Date: Sat, 26 Jun 2004 06:19:51 -0400

----- Original Message ----- 
From: "Eric A. Hall" <[email-masked]>
To: "Greg Connor" <[email-masked]>
Cc: <[email-masked]>
Sent: Saturday, June 26, 2004 5:17 AM
Subject: Re: Backward compatibility with deployed SPF records (and choice of
domain)

> underscore is illegal for hostnames. service names are not hostnames.

Did you see my message on this thread?

So the customer was right about his ISP saying it was illegal to use
_EP.hostname.com for a TXT record?

---

From: "Eric A. Hall" <[email-masked]>
Date: Sat, 26 Jun 2004 05:27:25 -0500

> _EP.hostname.com for a TXT record?

A domain name contains a series of labels.

Any label may contain character codes 0x00 through 0xFF.

A hostname may only contain labels with a-z, A-Z, 0-9, and "-" (there are
length and ordering restrictions too).

A TXT RR can be associated with any domain name and is not limited to
hostnames. The ISP is wrong.

---

From: Roy Badami <[email-masked]>
Date: Sat, 26 Jun 2004 11:18:07 +0100

>>>>> "Greg" == Greg Connor <[email-masked]> writes:

    Greg> Is this something we need to work on more, or should we
    Greg> leave it like that?  We talked about looking for the
    Greg> "closest SOA name" and looking there, and applying the TXT
    Greg> record at that location to other subdomains that have no TXT
    Greg> of their own.

Oh, it looks like I'm mistaken then: there _is_ a proposal to put
records at the zone apex.  My apologies to the chairs for suggesting
they were mistaken...

    Greg> (I really wish we could go back in time and tell people they
    Greg> need to set up MX records for all deliverable domains and
    Greg> hosts and not fallback to A...  any chance we could declare
    Greg> the fallback-to-A practice deprecated?  It may be officially
    Greg> outside our scope, but it really makes the job of publishing
    Greg> LMAP records a lot harder.)

I think it's still in moderately widespread use, so in keeping with
the philosophy of pragmatism, I think whatever we come up with has to
accept that.

I don't think I'd be happy with a proposal that recommended rejecting
mail from existing zones that neither publish MARID records nor MX
records.  And anything else adds complexity, since you'd have to
figure out in some way whether you were in a MARID-aware subtree of
the DNS before drawing any conclusions from the lack of an MX record.

    Greg> 1c. The group has not agreed on an alternate placement for
    Greg> TXT records other than the label exactly matching the (host
    Greg> part of the) identity being checked.  I.e. a MARID record at
    Greg> example.com does not cover [email-masked].

    Greg> That means: the domain owner should create LMAP records for
    Greg> ALL hosts in his domain that have A or MX records.  (Note
    Greg> that a wildcard TXT record does not cover names that have A
    Greg> or MX but no TXT, only names with no defined data at all)

    Greg> Is this something we need to work on more, or should we
    Greg> leave it like that?  We talked about looking for the
    Greg> "closest SOA name" and looking there, and applying the TXT
    Greg> record at that location to other subdomains that have no TXT
    Greg> of their own.

I fear that attempting to tackle this problem may be more than we can
achieve in the timescales.  Unless I'm mistaken, there have not been
yet been any IDs submitted to this WG that specify such a solution.  I
agree that it might be a nice feature of a solution, but perhaps it
should be left as an item for future study...?

I think the question should be whether this WG believes that the lack
of such a mechanism will seriously hamper MARID adoption, and whether
there are any obstacles to adding such a mechanism later.  If the
answer to both questions is 'no' then this should probably be left
till later.

    Greg> What would folks in the group think about defining our own
    Greg> prefix like "v=lmap1"?  I think this is only really needed
    Greg> in the off chance that the semantics are different enough
    Greg> for people to notice. 

I think that given draft-mengwonf-spf and draft-ietf-marid check
different identities, the differences are potentially enough for
people to notice.

It might not matter much, but if we don't allow people to publish
records that are _only_ recognized by MARID-compliant systems, then we
could be creating problems that will be difficult to fix later.

On the other hand I suspect that many people will be able to express a
single policy that will be valid for both the MAIL FROM identity used
by SPF and for the PRA identity currently used by the MARID drafts.
In that case I'd hate for people to have to publish the same record
with two different prefixes, since that will increase the size of the
response (at least if the records are put in the same place).  Hence
my proposal.

   -roy