Subject: Re: [spfbis] [dnsext] Obsoleting SPF RRTYPE

From: Hector Santos <[email-masked]>
Date: Mon, 29 Apr 2013 17:15:15 -0400

If anyone wishes to see one aspect of what is wrong with IETF Diversity, 
then see whats going on in SPF BIS WG where a key IETF cog essentially 
attempts to shutdown discussions and communications, attacks posters 
which by my estimate were making progress.

Progress is a status quo - DON'T CHANGE THE RFC4408 SPECIFICATION in 
SPFBIS.

Many in DNS community do not agree with the change of removing a long 
term migration path to a SPF RRTYPE.  In fact, not changing the existing 
specification would end the issue and hopefully satisfy all principles.

--
HLS

On 4/29/2013 5:03 PM, Dave Crocker wrote:
> Folks,
>
> This is one of those threads that will go on for as long as people seem
> to think they need to keep posting, but it won't make any "progress" or
> resolve any issues.
>
> For that matter, I have no idea what anyone thinks they are
> accomplishing by continuing to post on it, which raises the question of
> why they keep bothering.
>
> And 'bothering' is what this thread mostly /is/ doing, in reducing the
> S/N ratio.
>
> Please shut the thread down.
>
> d/

---

From: Mark Andrews <[email-masked]>
Date: Tue, 30 Apr 2013 09:07:42 +1000

The really annoying thing is that SPF is techically superior
	to TXT is lots of ways.

	1. It uniquely identifies the roll of the record.

	2. As SPF records are singletons you don't need to identify
	   and remove the old record when updating.  You can just
	   remove all SPF record and add the replacement.

	   For TXT you need to lookup the existing RRset, extract
	   the v=spf1 record from it.  You then need to create a
	   UPDATE message to delete just that record as well as add
	   the new TXT record.   You then have to hope that no one
	   else is performing a simultanious update as you may get
	   two TXT v=spf1 records in the RRset.

	The complains about using SPF is that there are broken
	firewalls and some servers drop queries for it, some registars
	don't support it.

	For firewalls, fix/replace the firewall if you intend to
	deploy SPF and it doesn't support it.  It is total !@##@#
	that firewall are incapable of handling new DNS record
	types.  New records we exected to occur from the very
	beginning and have been coming out regularly ever since the
	DNS was invented.  Firewall vendors that are incapable of
	handling new DNS types are incompetent and do not deserve
	repeat business.

	For servers than drop SPF queries they really are at the
	noise level.  When you identify one you complain to the
	owners of it.  Yes, that does work.  We needed to do that
	for AAAA records.

	For registrars, change registrar to one that does.

	Mark
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: S Moonesamy <[email-masked]>
Date: Mon, 29 Apr 2013 17:43:17 -0700

Hi Hector,
At 14:15 29-04-2013, Hector Santos wrote:
>If anyone wishes to see one aspect of what is wrong with IETF 
>Diversity, then see whats going on in SPF BIS WG where a key IETF 
>cog essentially attempts to shutdown discussions and communications, 
>attacks posters which by my estimate were making progress.
>
>Progress is a status quo - DON'T CHANGE THE RFC4408 SPECIFICATION in SPFBIS.
>
>Many in DNS community do not agree with the change of removing a 
>long term migration path to a SPF RRTYPE.  In fact, not changing the 
>existing specification would end the issue and hopefully satisfy all 
>principles.

The following messages were posted to the SPFBIS mailing list:

http://www.ietf.org/mail-archive/web/spfbis/current/msg03593.html
http://www.ietf.org/mail-archive/web/spfbis/current/msg03598.html

It has also been mentioned that there are two long threads at:

http://www.ietf.org/mail-archive/web/spfbis/current/msg03412.html
http://www.ietf.org/mail-archive/web/dnsext/current/msg13025.html

Regards,
S. Moonesamy

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 30 Apr 2013 18:28:52 +0200

> 	   two TXT v=spf1 records in the RRset.

That's true, except that one has TXT records anyway.

> 	firewalls and some servers drop queries for it, some registars

Nits, as explained below.  The basic fact that killed the SPF type is
the ability to use TXT as a replacement.  There must be an analogous
of Gresham's law:  "Bad types drive out good ones."

> 	For registrars, change registrar to one that does.

While it's too late for SPF, we can learn this lesson.

---

From: Doug Barton <[email-masked]>
Date: Tue, 30 Apr 2013 10:11:15 -0700

> While it's too late for SPF, we can learn this lesson.

As has been repeatedly pointed out in the discussion on both dnsext and 
spfbis, it is NOT too late for SPF. The way forward is simple:

1. Publish the bis draft which says for senders to publish both SPF and 
TXT versions, and receivers to query first for the SPF RRtype.

2. Update the HOW-TO docs to reflect this change.

3. Update the software to query SPF first (Note, Perl's NET::SPF, used 
by SpamAssassin, already made this change).

4. Let time pass.

5. When the next version of the SPF protocol (v=spf{>1}) comes out make 
it SPF/99 only.

The discussion about this on the spfbis list all revolved around the 
fact that TXT is widespread, SPF/99 is rarely used, so let's just stick 
with TXT. In a pre-3597 world there _were_ problems with querying for 
SPF first, so the fact that historically querying for SPF/99 first was 
painful is a valid data point. However the problems encountered in the 
early days of SPF deployment with servers not handing unknown types 
gracefully haven't been relevant for many years now. Yet, the SPF 
community has continued to push TXT only, in spite of the advice of 
4408. Almost like a self-fulfilling prophecy ...

The reasons brought forward by participants in the spfbis group to not 
make this change all revolve around the fact that it would involve 
additional work. Personally I don't find those arguments compelling. 
First, some of the arguments about the extra work are just plain silly 
(ala, "cut and paste of the same data for 2 RRtypes is too difficult"). 
Second, there are not that many implementations that query SPF, and the 
change is not difficult. Third, most of the "work" to be done is to wait 
for time to pass and for people to upgrade to the new versions of 
software. This is a bog-simple "long tail" problem that we deal with in 
the DNS world all the time.

There was one objection that made some sense, which is that right now, 
because the SPF world has steadfastly distributed the advice to use TXT 
only, querying for SPF/99 first gives you what is likely to be 1 
spurious DNS lookup per e-mail message. The obvious answer to that is to 
do a better job of encouraging folks to publish SPF records. Meanwhile, 
I have a fairly traditional mail server implementation that does a 
variety of anti-spam checks. By rough count it generates about 8 DNS 
queries for every message already. Generating 1 more is "in the noise" 
in the short term, and as shown above goes away in the long term.

Not only is this a case where doing the right thing is good for SPF, the 
SPF example of "let's just go with TXT because using a new RRtype is 
hard" has spread like wildfire, especially in the mail authentication 
world (DKIM, DMARC, etc.). The slippery slope has been well-greased at 
this point, and it would be nice to move things back in the right 
direction (see 5507 for example).

To be fair, there was one other objection raised, which is that DNS 
provisioning systems haven't caught up with the need for new RRtypes. So 
some can't go to their registrar's/web hoster's/etc. web interface and 
enter a proper SPF record. That's a problem, sure. But it's a problem 
that has to be solved, assuming we're actually going to take the advice 
in 5507 seriously. Personally I'm not willing to allow all progress 
forward in DNS to be stymied by those unwilling to make an investment in 
their own infrastructure.

In case Dave is still wondering what all the fuss is about, and because 
the folks in spfbis seem completely unwilling to deal with this issue in 
the group, consider this my first draft of an IETF last call objection 
to the fact that 4408bis wants to deprecate use of the SPF RRtype.

Doug

---

From: Dave Crocker <[email-masked]>
Date: Tue, 30 Apr 2013 10:13:15 -0700

On 4/30/2013 10:11 AM, Doug Barton wrote:
> spfbis, it is NOT too late for SPF.

As has repeatedly been pointed out, the market has already had plenty of 
time to adopt the RR and has overwhelmingly rejected it.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Edward Lewis <[email-masked]>
Date: Tue, 30 Apr 2013 14:02:11 -0400

On Apr 30, 2013, at 12:28, Alessandro Vesely wrote:
> ...The basic fact that killed the SPF type is

I disagree with the assertion that what killed SPF is "the ability to use TXT as a replacement."   It has nothing to do with one option being superior to the other, it was the lack of technical incentive to switch from one to the other.  I post this in the sense that if the root cause is not understood, no solution will stick.  Here is a message with my recounting of what led us to this point:

http://www.ietf.org/mail-archive/web/dnsext/current/msg12681.html

I don't see the death of SPF has a harbinger of things to come.  There's a strong case to be made the failure happened before 2004 and the root cause has since been corrected by changing the RRTYPE type allocation policies.  Still that fix was too late to ever let SPF sprout wings.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis             
NeuStar                    You can leave a voice message at +1-571-434-5468

There are no answers - just tradeoffs, decisions, and responses.

---

From: David Conrad <[email-masked]>
Date: Tue, 30 Apr 2013 11:08:19 -0700

Dave,

On Apr 30, 2013, at 10:13 AM, Dave Crocker <[email-masked]> wrote:
>> As has been repeatedly pointed out in the discussion on both dnsext and spfbis, it is NOT too late for SPF.
> 
> As has repeatedly been pointed out, the market has already had plenty of time to adopt the RR and has overwhelmingly rejected it.

What is the IETF-approved timeframe in which "the market" is allowed to accept/reject a particular technology?

Given no one appears to be making the argument that using the TXT RR and the MAIL FROM/HELO identity is the optimal solution (to put it mildly), I'm a bit confused why deprecating the SPF RR is being pushed so hard at this point in time. Is there some forcing function here?

Regards,
-drc

---

From: Dave Crocker <[email-masked]>
Date: Tue, 30 Apr 2013 11:12:42 -0700

On 4/30/2013 11:08 AM, David Conrad wrote:
>> As has repeatedly been pointed out, the market has already had
>> plenty of time to adopt the RR and has overwhelmingly rejected it.
>
> What is the IETF-approved timeframe in which "the market" is allowed
> to accept/reject a particular technology?

I've no idea what the lower limit is or should be, but I'm quite sure 
that 7 years exceeds it by a very comfortable margin.

> Given no one appears to be making the argument that using the TXT RR
> and the MAIL FROM/HELO identity is the optimal solution (to put it
> mildly), I'm a bit confused why deprecating the SPF RR is being
> pushed so hard at this point in time. Is there some forcing function
> here?

Surely this has been amply covered, multiple times.  Rehashing it won't 
be productive.

d/

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 30 Apr 2013 11:29:04 -0700

Doug,

Aren't you tired of repeatedly pointing out your half of the argument?  I
am of ours.  The Monty Python argument clinic sketch comes to mind.

On Tue, Apr 30, 2013 at 10:11 AM, Doug Barton <[email-masked]> wrote:

> The discussion about this on the spfbis list all revolved around the fact
> that TXT is widespread, SPF/99 is rarely used, so let's just stick with
> TXT. In a pre-3597 world there _were_ problems with querying for SPF first,
> so the fact that historically querying for SPF/99 first was painful is a
> valid data point. However the problems encountered in the early days of SPF
> deployment with servers not handing unknown types gracefully haven't been
> relevant for many years now. Yet, the SPF community has continued to push
> TXT only, in spite of the advice of 4408. Almost like a self-fulfilling
> prophecy ...
>

As has been repeatedly pointed out, the "haven't been relevant for many
years now" conclusion you've reached is contradicted by both anecdotal and
empirical data.

I assure you that I hear all of the points you are making.  I also assure
you that none of them resolve any of the points being made on the other
side.  There's no dialog happening here.

> additional work. Personally I don't find those arguments compelling.

Sure, because you don't have to do the work, nor are you confronted with
the weight of the long tail in this case.  I submit that it is presumptuous
to assert that the properties of one community's long tail problems are the
same as they are in others.

> (ala, "cut and paste of the same data for 2 RRtypes is too difficult").

I don't think anyone said that's difficult.  I would also point out that
it's not difficult, given a jumble of TXT RRs in a reply, to find any that
start with a particular identifying substring which would mean "this is an
SPF record", so let's nip that one in the bud right now.

These are all distractions from the real points being made.

> change is not difficult.

Nobody said that's a problem either.

> Third, most of the "work" to be done is to wait for time to pass and for
> people to upgrade to the new versions of software. This is a bog-simple
> "long tail" problem that we deal with in the DNS world all the time.
>

...during which time the service arguably operates in a degraded state.

The assertion that "You should just fix your DNS infrastructure!" ignores
the fact that the interfering piece of the infrastructure may not be under
the control of the operator who can't complete an SPF operation, or, worse,
that the owner of the problem lacks the resources or interest to get it
fixed for a long while, if at all.

Do we just have SPF operators call you for emotional support during this
protracted transition time?

> SPF example of "let's just go with TXT because using a new RRtype is hard"
> has spread like wildfire, especially in the mail authentication world
> (DKIM, DMARC, etc.). The slippery slope has been well-greased at this
> point, and it would be nice to move things back in the right direction (see
> 5507 for example).
>

This is also a separate argument given the underscore prefix debate.

> enter a proper SPF record. That's a problem, sure. But it's a problem that
> has to be solved, assuming we're actually going to take the advice in 5507
> seriously. Personally I'm not willing to allow all progress forward in DNS
> to be stymied by those unwilling to make an investment in their own
> infrastructure.
>

What are you prepared to do to help address the broken infrastructure?
It's all well and good to lob directives over the wall, but I'd like to
hear some advice about how to compel inattentive infrastructure operators
to make these changes, especially when those operators aren't necessarily
the ones affected.

-MSK

---

From: David Conrad <[email-masked]>
Date: Tue, 30 Apr 2013 12:52:56 -0700

Murray,

On Apr 30, 2013, at 11:29 AM, Murray S. Kucherawy <[email-masked]> wrote:
> I would also point out that it's not difficult, given a jumble of TXT RRs in a reply, to find any that start with a particular identifying substring which would mean "this is an SPF record", so let's nip that one in the bud right now.

SPF using TXT and hence, SPFBIS forces the uniquification of the DNS response into the application instead of in the DNS library. Given the ordering of individual TXT RRs within an RRset is not guaranteed, I suspect the chances that every application is going to do that parsing correctly is relatively low (btw, the example in 3.3 in spfbis-14 is a bit misleading: assuming "second string" is in a separate TXT RR (which is implied), the concatenation might be "second stringv=spf1 ..... first").  The more interesting bit is if/when another protocol uses TXT at the same ownername.

> What are you prepared to do to help address the broken infrastructure?

Argue against deprecating the SPF RR :).

The RR has been allocated and it is supported in most DNS servers and resolver libraries in one form or another.  Provisioning systems take much longer, but that isn't surprising and shouldn't be an argument to deprecate (if it is, we might as well close the RRtype registry for new entries).

In the past, the IETF used to make decisions based on long-term technical/architectural correctness, even in the face of pragmatic complications (e.g., the choice to mandate strong crypto despite real world challenges some companies faced in exporting that crypto in products). In this particular case, there seems to be an argument to explicitly disallow the long-term technically/architecturally correct approach because some folks choose not to add an RR to their zone files or support that RR in their provisioning systems.  As I said in DNSEXT, this seems like the wrong approach to me.

Regards,
-drc

---

From: David Conrad <[email-masked]>
Date: Tue, 30 Apr 2013 12:54:11 -0700

On Apr 30, 2013, at 11:12 AM, Dave Crocker <[email-masked]> wrote:
> I've no idea what the lower limit is or should be, but I'm quite sure that 7 years exceeds it by a very comfortable margin.

By that logic we should abandon IPv6, DNSSEC, EDNS0, etc.

Thanks,
-drc

---

From: Dave Crocker <[email-masked]>
Date: Tue, 30 Apr 2013 13:20:35 -0700

> By that logic we should abandon IPv6, DNSSEC, EDNS0, etc.

Gosh, David.  I guess you win.

d/

---

From: David Conrad <[email-masked]>
Date: Tue, 30 Apr 2013 14:37:05 -0700

> Gosh, David.  I guess you win.

Gee Dave, great way to elevate the discussion.

*plonk*

---

From: Dave Crocker <[email-masked]>
Date: Tue, 30 Apr 2013 14:41:44 -0700

> Gee Dave, great way to elevate the discussion.

You were discussing?

d/

---

From: Mark Andrews <[email-masked]>
Date: Wed, 01 May 2013 07:54:46 +1000

In message <[email-masked]>, Dave Crocker writes:
> >> I've no idea what the lower limit is or should be, but I'm quite sure that
>  7 years exceeds it by a very comfortable margin.
> >

When just about none of the documentation mentions the SPF record type
nor has examples with both SPF and TXT records it isn't the "market"
choosing.  You can't choose of you really havn't been informed.

		BEWARE OF THE LEOPARD

    "But Mr Dent, the plans have been available in the local planning
     office for the last nine months."

    "Oh yes, well as soon as I heard I went straight round to see
     them, yesterday afternoon. You hadn't exactly gone out of your
     way to call attention to them, had you? I mean, like actually
     telling anybody or anything."

    "But the plans were on display ..."

    "On display? I eventually had to go down to the cellar to find them."

    "That's the display department."

    "With a flashlight."

    "Ah, well the lights had probably gone."

    "So had the stairs."

    "But look, you found the notice didn't you?"

    "Yes," said Arthur, "yes I did. It was on display in the bottom
     of a locked filing cabinet stuck in a disused lavatory with a
     sign on the door saying 'Beware of the Leopard'."

     The Hitchhiker’s Guide to the Galaxy
     Douglas Adams

Mark

---

From: Sam Hartman <[email-masked]>
Date: Tue, 30 Apr 2013 20:45:04 -0400

I think I understand this issue well enough to comment and so I will.

1) I totally believe it is reasonable to consider operational challenges
when designing protocols. "Just upgrade your infrastructure, just use
another registrar, just upgrade the infrastructure of the people you
communicate with," are not generally reasonable advice to give people;
ingoring these concerns tend to lead to things that cannot be deployed.

2) I think it's fine to come up with a solution that   is harder to
implement because it is easier to deploy given the sorts of concerns in
1.

3) having reviewed the DNS arguments, I generally would recommend txt
records with distinct owner names over new RR types for most
protocols. I understand there are residual issues with that approach.

4) Using txt RRs the way SPF does--where the owner name is shared
potentially with other applications is kind of unfortunate.  I don't
know it's unfortunate enough to push for the SPF RR. But the issues
people have brought up with regard to updates and RR order are very
real.
But so are the operational issues with SPF.

So my personal opinion is that this is a valid discussion to be having
even if we're having it again in IETF LC.  However, to be successful we
need to get new participants and to ad more respect for arguments to the
discussion.

---

From: Mark Andrews <[email-masked]>
Date: Wed, 01 May 2013 11:04:52 +1000

> That's true, except that one has TXT records anyway.

	nsupdate
	update del example.com SPF
	update add example.com 3600 SPF v=spf1 ....
	send

	txt=`dig +short example.com TXT | \
	sed -n -e '/^"v=spf1 /s/^/update del example.com TXT /p' \
	       -e '/^"v=spf1"$/^/update del example.com TXT /p'`
	nsupdate << EOF
	$txt
	update add example.com 3600 TXT v=spf1 ....
	send
	EOF

	But that doesn't work for 'example.com TXT "v" "=" "s" "p" "f" "1"'
	which is a perfectly legal SPF record.

	sed -n -e '/^"v=spf1 /s/^/update del example.com TXT /p' \
	       -e '/^"v" "=spf1 /s/^/update del example.com TXT /p' \
	       -e '/^"v" "=" "spf1 /s/^/update del example.com TXT /p' \
	       -e '/^"v" "=" "s" "pf1 /s/^/update del example.com TXT /p' \
	       -e '/^"v" "=" "s" "p" "f1 /s/^/update del example.com TXT /p' \
	       -e '/^"v" "=" "s" "p" "f" "1 /s/^/update del example.com TXT /p' \
	       -e '/^"v" "=" "s" "p" "f" "1" " /s/^/update del example.com TXT /p' \
	       -e '/^"v=spf1"$/^/update del example.com TXT /p'`
	
	And keep going because the delete needs the rdata to be a
	perfect match to identify the record to be removed.

	I'm sure I could come up with a more compact way of identifying
	a spf record but it wouldn't be needed if people published type
	SPF.

>p > 	firewalls and some servers drop queries for it, some registars
> > 	don't support it.
>

---

From: Mark Andrews <[email-masked]>
Date: Wed, 01 May 2013 11:16:48 +1000

> by SpamAssassin, already made this change).

As has libspf2.

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 30 Apr 2013 22:24:33 -0400

On Wednesday, May 01, 2013 07:54:46 AM Mark Andrews wrote:
> > >>> to a
> > ccept/reject a particular technology?
> > 
> > >> that

I've run what I believe to be the most popular online SPF record validator 
since 2005.  You'll find it frequently linked to by other parties:

http://kitterman.com/spf/validate.html

The published SPF record validation part of the tool checks both for records 
of type TXT and of type SPF and mentions the absence of the type SPF record.  
It has done this since I first published the validator (in 2005 I believe, it 
may have been 2006).

So it has been both supported and mentioned.  The only real documentation, RFC 
4408, mentions it effectively enough that some people were fooled into 
believing that publishing type SPF only would be a good idea, fortunately only 
a few were confused by the official documentation.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 30 Apr 2013 22:28:01 -0400

> > by SpamAssassin, already made this change).

Actually it's Mail::SPF and as we already discussed on the spfbis list this 
design choice caused operational problems and in some cases applications that 
use Mail::SPF have disabled type SPF queries as a result.

One perl module designer being overly theoretically correct doesn't really add 
much to your overly theoretical argument.

So far, compared to the discussion we had in the working group at the time, 
there isn't any new information to evaluate.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 30 Apr 2013 22:03:08 -0700

On Tue, Apr 30, 2013 at 12:52 PM, David Conrad <[email-masked]> wrote:

> SPF using TXT and hence, SPFBIS forces the uniquification of the DNS
> response into the application instead of in the DNS library. Given the
> ordering of individual TXT RRs within an RRset is not guaranteed, I suspect
> the chances that every application is going to do that parsing correctly is
> relatively low (btw, the example in 3.3 in spfbis-14 is a bit misleading:
> assuming "second string" is in a separate TXT RR (which is implied), the
> concatenation might be "second stringv=spf1 ..... first").  The more
> interesting bit is if/when another protocol uses TXT at the same ownername.
>

Yes, I understand all of that, and I've written code to deal with it.  It's
not rocket science.

> The RR has been allocated and it is supported in most DNS servers and
> resolver libraries in one form or another.  Provisioning systems take much
> longer, but that isn't surprising and shouldn't be an argument to deprecate
> (if it is, we might as well close the RRtype registry for new entries).
>

We're not only talking about provisioning systems here.  We're also talking
about interference with queries and replies at the DNS protocol level.  The
survey work done for RFC6686 showed that something on the order of
thousands of domains in the sample set suffered from this.  It is a very
real issue for a deployed protocol.

> In the past, the IETF used to make decisions based on long-term
> technical/architectural correctness, even in the face of pragmatic
> complications (e.g., the choice to mandate strong crypto despite real world
> challenges some companies faced in exporting that crypto in products). In
> this particular case, there seems to be an argument to explicitly disallow
> the long-term technically/architecturally correct approach because some
> folks choose not to add an RR to their zone files or support that RR in
> their provisioning systems.  As I said in DNSEXT, this seems like the wrong
> approach to me.
>

All things being equal, I would agree with you.  And if SPF were starting
anew today, I suspect I might have a different opinion.  The question,
then, is the weight of the mitigating circumstances, which obviously the
two communities evaluate quite differently.

-MSK