Subject: Trouble with Sender Authentication

From: "K.J. Petrie" <[email-masked]>
Date: Thu, 2 Nov 2006 16:12:13 +0000

I hate to spoil a good party, but it seems to me there are several minor flaws 
and one major flaw with all the proposed methods of reducing spam through 
authenticating senders, which will either prevent their adoption or render 
them useless once widely adopted.

SPF, for instance, breaks forwarding. This will stop it catching on widely, 
because domain owners are likely to use forwarding for their incoming mail. 
If their ISP adopts SPF they could lose all incoming mail from domains with 
an SPF record. To prevent this, the forwarding company, over which the domain 
owner may have no control, must take action. So we have a proposed system 
which requires unrelated entities to take co-ordinated action if it is not to 
disrupt mail delivery to the very people responsible for enabling it. This 
could be fixed by adding a Recipient Protocol Framework (my name - apologies 
if something called this already exists) to the standard, specifying servers 
which were to PASS all mail addressed To: (or Cc: Envelope-To: etc) the 
domain without checking the sender's domain. The RPF record could then be set 
up by the domain owner, either as a separate string or an entry in the same 
string, at the same time as the SPF record, and so forwarding could be 
enabled by the person who should control it- the domain owner, without 
worrying about whether one company or another had implemented the checks. 
However, this is only a minor flaw.

Much more significant is the fact that all these systems depend on publishing 
information. What is published for the use of the good guys is also available 
to the baddies, and that's the real problem. For among the hundreds (or, more 
likely thousands) of requests for SPF records name servers would receive from 
MTA's every second could be hundreds of identical requests from a data-mining 
program seeking to build a reverse-lookup database. Once the spammers have 
such a database it will be relatively simple for them to write software which 
spoofs credible domains for every spam injection point they use. And where 
will that leave us? Back where we started, but with increased network traffic 
and DNS load doing useless checks.

Certainly, my experience with spam is that, wherever it appears to come from, 
there are only five or six basic variants being sent repeatedly at any one 
time. Therefore, it is likely all this mail is coming from only five or six 
real sources which have managed to infiltrate the Internet at a large number 
of points. Do we really imagine people who do that will be put off by the 
need to run a little more automated research?

Identifying spammers (or, more likely, trojanised mailers) and their reply 
websites (or, again more likely, trojanised proxies) is only as good as the 
abuse desks which take action, and overstretched resources take time to close 
down these gateways. The spammers know this, and presumably make their profit 
in the intervening interval, and are always ready to move on. They are the 
Internet's suitcase boys, standing on a street corner and ready to run when a 
police officer comes within sight. To make spamming and the associated 
activities unprofitable requires an automated system which either cannot be 
abused, or which punishes abuse so rapidly it cannot be worthwhile.

I don't want to be negative, but it seems evident that a lot of highly 
talented effort is being directed at something which is not a solution, and 
that effort could be better used devising something more radical and 
effective. I realise such a solution might be off-topic for this list. I have 
no ideas at present, I'm afraid.

I was directed to this list by the spf-discuss sign-up response.

Best wishes,

K.J. Petrie.

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 02 Nov 2006 19:18:13 +0100

K.J. Petrie wrote:
 
> I hate to spoil a good party

The party was closed more than two years ago.  You're a bit late... :-)
<http://en.wikipedia.org/wiki/MARID>  This is a historic battle field.
 
> SPF, for instance, breaks forwarding.

The variant explained in RFC 1123 5.3.6(a) or RFC 2821 3.10.1 doesn't
work directly if it's checked "again", yes.  Actually SPF works only
at the first MX of the receiver, and not (without tricks) behind it.
Working as designed (back in 2003, before this list existed).

> the forwarding company, over which the domain owner may have no 
> control, must take action.

Nope, they can ignore it.  The mail FAILs, they bounce it, and the
"sender can make other plans" (quote stolen from an unrelated draft).
A perfectly sound situation, if you grep for "551" in RFC 821.  

> This could be fixed by adding a Recipient Protocol Framework (my
> name - apologies if something called this already exists)

Other names for ideas to arrange things on the side of the receiver
are SRS (sender rewriting system), and various "forward plans", the
next hop can white-list known "good" forwarders (saving the time for
pointless SPF checks resulting in predictable FAIL rejects).  There
are many ways to solve this, some are mentioned in RFC 4408.

> However, this is only a minor flaw.

ACK, actually it's the design, SPF enumerates IPs allowed to send 
mail from the domain in question.  For obvious reasons it cannot
enumerate IPs somewhere on the side of the various receivers.  One
of the predecessors was "RMX", like what we know as MX today (for
receivers), only "reverse", the IPs of the sender.

> What is published for the use of the good guys is also available
> to the baddies, and that's the real problem.

If a spammer comes to the conclusion that forging my address isn't
in the interest of his "business" it's no problem, it's good for me.

> Once the spammers have such a database it will be relatively 
> simple for them to write software which spoofs credible domains
> for every spam injection point they use.

"Relatively".  How ISPs arrange mail submission is a separate issue,
they should use SMTP AUTH and enforce submission rights.  If I can
get a PASS, then I could still get it if I'm a zombie sending spam.

A PASS from unknown strangers has a very limited value, wrt SPF it
only means that a bounce won't hit innocent bystanders.  If it's a
PASS from me and you know me I could still be a fresh zombie.  But
you'd think twice if somebody claiming to be me with SPF PASS etc.
suddenly talks about phar*aceu*ical*.

For more serious PASS senders like PayPal I'd hope that they limit
the zombies sending from within their "borders".  So far it works :-)

> Do we really imagine people who do that will be put off by the

No, it's okay if they just stay away from protected addresses.  SPF
is about avoiding bogus bounces, and encouraging legit bounces, not
a magic wand to eliminatie all spam.    
 
> I was directed to this list by the spf-discuss sign-up response.

Oops, that's wrong, I'll post my reply there too, let the old mxcomp
rest in peace, it's dead.

Frank

---

From: wayne <[email-masked]>
Date: Thu, 02 Nov 2006 12:33:19 -0600

In <[email-masked]> "K.J. Petrie" <[email-masked]> writes:

> I hate to spoil a good party,

Hi.

I'm afraid there isn't much of a party here, good or otherwise.  Yours
is the first post to this list since May, and there hasn't been any serious
discussion on this list since the MARID working group was shut down.

> SPF, for instance, breaks forwarding. [snip]
>
> [...] What is published for the use of the good guys is also available 
> to the baddies, [...]  Once the spammers have 
> spoofs credible domains for every spam injection point they use.

Both of these points that you raised are very old arguments that have
been discussed to death many times over the last 3-4 years.  See the
archives of the ASRG, spf-discuss, and this MARID mailing lists.

None of the things you said haven't been discussed before.  Some of
the points have been well rebutted by various people, others see these
as points where reasonable people can disagree on.  I think it is
unlikely that you will change many minds, and, since this mailing list
is pretty dead, it is probably not a very good place to try discussing
it any more.

> I was directed to this list by the spf-discuss sign-up response.

Thanks for pointing that out.  It looks like the spf-discuss "welcome"
message has not really been updated since this working group was
closed down.  I have edited it to bring things up to date.

In particular, it no longer says that if you think SPF is
fundamentally flawed, you should take your discussions here.  That was
reasonable advice back when the MARID WG was trying to create
something better, but I don't think it is a reasonable request any
longer.

-wayne

---

From: Douglas Otis <[email-masked]>
Date: Thu, 2 Nov 2006 14:08:42 -0800

On Nov 2, 2006, at 8:12 AM, K.J. Petrie wrote:

> SPF, for instance, breaks forwarding. This will stop it catching on  
> widely, because domain owners are likely to use forwarding for  
> their incoming mail.

Forwarding is a problem in any address-path registration scheme.   
Sender-ID alters rfc2822 by asking for a Resent-From header,  
superseding either Sender or From headers, to be added by forwarding  
agents.  Those advocating SPF-Classic want the MailFrom altered by  
the forwarding agent where the original domain is folded into the  
local-part along with a hash and a time-stamp.  DSNs are relayed  
through the last forwarding agent.

There is an alternative to SPF-Classic that uses DKIM signed  
messages.  An association of the Mail-From domain with that of the  
signing-domain can validate the DSN address through an association  
scheme.

An association could look something like:

<base32(sha1(signing-domain))>._DKIM-M.<mail-from> TXT "d=signing- 
domain"

See:
http://tools.ietf.org/wg/dkim/draft-otis-dkim-dosp-02.txt

This scheme assures the DSN address in one reliable and small DNS  
transaction.  There would not be any concerns related to forwarding  
breaking an address path registration scheme.  An association scheme  
works without the cooperation of forwarding agents as well.

> And where will that leave us?  Back where we started, but with  
> increased network traffic and DNS load doing useless checks.

It is much worse actually.  SPF scripts allow each spam to generate  
hundreds of targeted DNS transactions using the resources of those  
recipients foolish enough to run these scripts.  Poorly conceived  
timeouts in SPF libraries compounds the problem, as this also removes  
congestion avoidance.  SpamAssassin's default timeout is 5 seconds,  
for example.  Any SPF domain can target any other domain with 100 DNS  
transactions per instance, name, and recipient.

See:
http://www.ietf.org/internet-drafts/draft-otis-spf-dos-exploit-01.txt
  or
http://www.sonic.net/~dougotis/maawg/SPF%20Exploit%20Concerns-maawg.pdf

> Certainly, my experience with spam is that, wherever it appears to  
> come from, there are only five or six basic variants being sent  
> repeatedly at any one time. Therefore, it is likely all this mail  
> is coming from only five or six real sources which have managed to  
> infiltrate the Internet at a large number of points. Do we really  
> imagine people who do that will be put off by the need to run a  
> little more automated research?

More than 70% of these messages are introduced from compromised  
systems, often in concert in the tens of thousands.  SPF allows for  
exploits far more nefarious than research.

> Identifying spammers (or, more likely, trojanised mailers) and  
> their reply websites (or, again more likely, trojanised proxies) is  
> only as good as the abuse desks which take action, and  
> overstretched resources take time to close down these gateways. The  
> spammers know this, and presumably make their profit in the  
> intervening interval, and are always ready to move on. They are the  
> Internet's suitcase boys, standing on a street corner and ready to  
> run when a police officer comes within sight. To make spamming and  
> the associated  activities unprofitable requires an automated  
> system which either cannot be abused, or which punishes abuse so  
> rapidly it cannot be worthwhile.

DKIM always signed by the transmitting entity in conjunction with  
standardized opaque identifiers and an opaque-identifier revocation  
scheme would be a solution that could streamline abuse reporting  
aimed at eliminating much of the spam.

See:
http://tools.ietf.org/wg/dkim/draft-otis-dkim-security-concerns-01.txt

This list remains available for continued discussions of any MARID  
related issues. The spf-discuss reflector required participants to  
first agree with the promotion of SPF.  An insurmountable barrier for  
some. : )

A primary use of SPF is for white-listing bulk senders (otherwise  
frequently being block-listed).  DKIM in conjunction with an SMTP  
Client association scheme more reliably fulfills that purpose, where  
the DNS transactions are significantly reduced as well.  When  
validating the EHLO is first, a DDoS attack is rather improbable.

-Doug

---

From: wayne <[email-masked]>
Date: Thu, 02 Nov 2006 19:56:43 -0600

In <[email-masked]> Douglas Otis <[email-masked]> writes:

> [long rant that DougO has repeatedly given over the last 3 years snipped]

> first agree with the promotion of SPF.  An insurmountable barrier for

The SPF-discuss mailing list has never required you to agree with the
promotion of SPF.  Promotion of SPF is one of the on-topic discussions
for the list, as is *constructive* discussions.  There have been
*lots* of people who have talked about lots of different problems with
SPF on the SPF-discuss list.  I've asked several times now if any of
the other list moderators have ever thrown anyone off, and so far we
can't come up with one case.  

Doug, I've read a lot of your postings about SPF over the last several
years, and I can't say I can recall you ever trying to make a
*constructive* suggestion on how to improve SPF, other than to throw
it out.  If the requirement that you need to be constructive has kept
you off the spf-discuss list, I can't say I'm sorry about it.

As far as the SPF DoS stuff goes, I was the first to really raise the
issue of the DoS potential of SPF back in 2003.  The constructs that
you have mentioned in your I-D were analyzed by me before you first
wrote a post about SPF.  The lack of sufficient process limits was one
of the major reasons why I split with Mark Lentczner on the
SPF-classic draft and started my own, which eventually became RFC4408.
I have discussed this at length on the SPF list.  The difference
between you and me is that I offered *constructive* ways to change SPF
so that the DoS potential is greatly reduced.

I really think your huge exaggerations and running around like
chicken-little claiming that the sky will fall if SPF is adopted has
done more to hurt my push to take SPF DoS potentials seriously than
anything else.  I've read your draft.  Your numbers don't make sense.
You spend way too much time promoting yourself and ranting, and far
too little time actually presenting data.

It wasn't until your -01 version of your draft that you actually
presented hard data in your Appendix A.  Your data doesn't back up
your 1000x claims.

As I said, I've done this analysis before.  I actually *tried* to set
something up that would DoS Meng's box and tested it, but it turns out
that this is *MUCH* harder in practice than your theoretical
hand-waving analysis makes it seem.  People *don't* have the MTAs set
up in stupid ways that make it easy.

I'm tired of your rants Doug.  I'm tired of your rants making
reasonable discussions on this subject harder.  And, yes, I realize
I'm probably burning a bridge here.

-wayne

---

From: Douglas Otis <[email-masked]>
Date: Fri, 3 Nov 2006 11:08:24 -0800

On Nov 2, 2006, at 5:56 PM, wayne wrote:

>> first agree with the promotion of SPF.  An insurmountable barrier  
>> for some. : )
>
> The SPF-discuss mailing list has never required you to agree with  
> the promotion of SPF.

Review the archive and you'll find a posting of the agreement  
initially required before subscribing to spf-discuss.  While it may  
have changed, this was the reason for not participating on that  
list.  The MARID list is still functional.

More than 70% of emails are spam, of which more than 70% are sent  
from compromised systems.  SPF offers powerful tools to these  
senders.  Consuming disproportionate amounts of a recipient's  
resource remains a philosophical barrier preventing support for an  
SPF scheme that expects large, powerful, and dangerous scripts be  
executed by recipients at every stage of delivery. : (

> Doug, I've read a lot of your postings about SPF over the last  
> several years, and I can't say I can recall you ever trying to make  
> a *constructive* suggestion on how to improve SPF, other than to  
> throw it out.

In Dusseldorf, Julian and Ming received *constructive*  
recommendations to allow record scoping, but this was ignored and  
became the basis for Julian's ignored complaint made to the IESG.   
Macros contained within SPF scripts require separate transaction sets  
per each possible and undefined scope, even when the same domain is  
resolved. : (

> As far as the SPF DoS stuff goes, I was the first to really raise  
> the issue of the DoS potential of SPF back in 2003.  The constructs  
> that
> wrote a post about SPF.  The lack of sufficient process limits was  
> one of the major reasons why I split with Mark Lentczner on the SPF- 
> classic draft and started my own, which eventually became RFC4408.

The solution adopted in your libraries harms domains by randomizing  
results of those that utilize more than 5 targets in their MX  
records. : (

> between you and me is that I offered *constructive* ways to change  
> SPF so that the DoS potential is greatly reduced.

The design of SPF as a massive script forecloses meaningful  
solutions.  You have yet to recognize SPF creates an impossible  
situation. : (

> chicken-little claiming that the sky will fall if SPF is adopted  
> has done more to hurt my push to take SPF DoS potentials seriously  
> than anything else.  I've read your draft.  Your numbers don't make  
> sense.  You spend way too much time promoting yourself and ranting,  
> and far too little time actually presenting data.

You are free to ask questions regarding figures not understood.   
Several proposals are solutions that avoid threats created by SPF  
(while also improving email integrity).  This is thinking outside the  
SPF box.  No personal gains are achieved by offering IETF drafts or  
fundamentally opposing SPF.  Let's stick to the merits of the concerns.

> presented hard data in your Appendix A.  Your data doesn't back up  

45 pages in the addendum trace SPF resolving a single name at about  
64:1 increase in traffic.  Consider the distribution methods used by  
spammers.  Messages come from tens of thousands of systems at slow  
rates to avoid detection.  This is an ideal platform to leverage SPF  
for a number of very bad things.  Gains stated in the SPF-DoS draft  
underestimate the risk.  Bad actors are sending spam where the  
possible attack is a gift provided by SPF.

Email as well as a long lived SPF records should be excluded from the  
attack overhead.  The same SPF record can utilize components of a  
local-part to produce infinitely variable transaction sets.  Only in  
rare cases is the intended scope of SPF defined, which means the  
executions per message, per MHS, and per recipient is unknown.  Both  
the MTAs and MUAs execute SPF scripts.  Bad actors have no difficulty  
using SPF to canvass recipients to determine who is receiving and  
executing SPF scripts.  There has been some rather scary discussions  
suggesting SPF might control DKIM replay abuse.  Such would add any  
number of new names resolved using SPF.  : O

> As I said, I've done this analysis before.  I actually *tried* to  
> set something up that would DoS Meng's box and tested it, but it  
> turns out that this is *MUCH* harder in practice than your  
> theoretical hand-waving analysis makes it seem.  People *don't*  
> have the MTAs set up in stupid ways that make it easy.

Consider how spam is currently being distributed.  The attack  
scenario does not require anything more than executing SPF scripts  
referenced from the spam received.

> I'm probably burning a bridge here.

Your frustration is appreciated and no bridge is burned.  However,  
please no not dismiss these concerns as rants.

-Doug

---

From: Dean Anderson <[email-masked]>
Date: Mon, 6 Nov 2006 14:34:29 -0500 (EST)

On Fri, 3 Nov 2006, Douglas Otis wrote:

> from compromised systems.  

Most of this isn't commercial, either. Commercial bulk email is no
longer a problem.

> > a *constructive* suggestion on how to improve SPF, other than to

"Throwing it out" is a constructive suggestion. Wasting time on schemes
that can't succeed is, well, a waste of time and resources.  This is how
perpetual motion schemes motivated the development of thermodynamics.  
SPF is just another perpetual motion scheme which is shown impossible by
information theory.  The promoters of perpetual motion was similarly
"urgently" interested in spending money to build their machines.  
Thermodynamics demonstrated the scam.

Likewise, certain people seem to be making a lot of money consulting on
such anti-spam schemes.  Or making money on blacklists, and other
schemes.  In my view, those people aren't solving spam problems. It is
questionable what interests those people have in solving spam problems,
given that a 'solution' would mean an end to their financial prosperity.  
It is very odd that non-commercial abuse continues to grow. Who benefits
from that?

The following may be helpful:

--------------

Subject: Re:  spam content filtering

Queueing and information theory is your friend.  Blacklists are
generally not your friends, even if well intentioned, DHCP changes
virus-infected IP addresses far faster than a human oriented blacklist
could keep up.  The ones that do keep up tend to be connected to the
abuse, and often those are also defamatory of someone.

I worked out several years ago, using information theory, that one
cannot stop spam with any technical means.  The info. theory argument is
easy to follow, if you have some little background. It is a known result
that one cannot prove the non-existance of a covert channel in a
communications system.  It is easy to establish that email is a
communications channel for information theory.  It is also easy to
establish that spam is an unwanted communication channel use, equivalent
to a covert channel. It then follows that one can't stop spam since that
would be the equivalent of proving that a covert channel couldn't exist,
which is a contradiction of a known result.

One can of course discover spamming, and temporarily stop it on some
parameter (e.g. IP address), but this is always a "whack-a-mole"  
operation. You can never fundamentally improve upon whack-a-mole. [Same
is true of spying]

What now? Quit? Not at all: Queues are your friend.  Place messages from
previously known senders in faster queues, while unknown messages are in
slower queues.

"known senders" can be identified in a variety of ways: email addresses, 
smtp server IP addresses, etc. 

Everyone actually gets their spam, and can search it for previously
unknown good senders.  It just doesn't show up so directly.  It just
winds up in a slower queue and a different mailbox.  These messages can
be expired, and/or used to to identify bad senders, and make complaints
as you like...  Known bad-senders can be expired quicker, etc.

One of the good things to happen was the CAN-SPAM Act. Genuine
commercial bulk emailers (CBE) are not the cause of abuse, and don't
need to be blocked. Genuine CBE's don't abuse open proxies, nor abuse
open relays, nor forge headers, nor use viruses to install proxies, etc.  
This leaves one to wonder who is doing those things. And that brings us
back to the certain of the blacklists who were previously found to be
doing that sort of thing.  It is telling that those same people really,
really hate CAN-SPAM, and really, really hate any distinction between
genuine CBE and abuse that isn't actually commercial.

-- 
Av8 Internet   Prepared to pay a premium for better service?
www.av8.net         faster, more reliable, better service
617 344 9000

---

From: Greg Hewgill <[email-masked]>
Date: Mon, 6 Nov 2006 20:41:44 +0000

> Most of this isn't commercial, either. Commercial bulk email is no

Is that right? Virtually all the spam I see relates to pharmacy, stock,
mortgage, pirated software, or are email worms. The whole idea of each
of those are to make money for somebody. I don't even know what
"non-commercial spam" would look like today.

Greg Hewgill
http://hewgill.com

---

From: wayne <[email-masked]>
Date: Mon, 06 Nov 2006 16:06:45 -0600

In <[email-masked]> Douglas Otis <[email-masked]> writes:

> have changed, this was the reason for not participating on that  list.
> The MARID list is still functional.

Yes, I saw your post.  People pointed out to you that you were wrong
back then, and you are still wrong today.

Constructive critisism is on-topic for the spf-discuss list.  So is
the promotion of SPF.  I have removed the suggestion that if you think
SPF is fundementally flawed that you should come here instead.

> 64:1 increase in traffic.

Nice to see that you agree that your data doesn't back up your claims,
but even your 64:1 number is bogus.

-wayne

---

From: wayne <[email-masked]>
Date: Mon, 06 Nov 2006 16:10:15 -0600

In <[email-masked]> Greg Hewgill <[email-masked]> writes:

> Is that right?

That has been Dean's position for many years now.  Before trying to
debate him on the issue, I strongly suggest you search some of his
previous rants on the issue of spam.  Like Doug Otis, Dean has some,
uh, unusual ideas.

-wayne

---

From: Julian Mehnle <[email-masked]>
Date: Mon, 6 Nov 2006 23:12:49 +0000

Douglas Otis wrote:
> In Dusseldorf, Julian and Ming received *constructive* recommendations to
> allow record scoping, but this was ignored and became the basis for
> Julian's ignored complaint made to the IESG.

Sorry to barge in on this particular topic (which is mostly unrelated to 
the issues raised by K.J. Petrie or the alleged DoS issue), but this ain't 
correct.  The v=spf1/pra re-use issue was indeed discussed at the MAAWG 
meeting in Düsseldorf.  I'm not sure what you mean when you say that
"allowing record scoping" was proposed, but what was actually proposed were 
two different things (depending on whom I talked to):

 A. Accept the v=spf1/pra re-use and start promoting v=spf1 as having a
    different meaning than what had been defined back in 2003.

 B. Abandon v=spf1 entirely in favor of spf2.0 (or some completely
    different) scheme in order to avoid the misinterpretation of v=spf1
    records for PRA purposes.

At that point (2005-06), materially changing the definition of v=spf1 was 
out of the question as SPFv1 (and many, many records) had existed 
essentially unchanged since at least early 2004.  And agreeing to jump 
ship towards spf2.0 (Microsoft's Sender ID) was out of the question, too, 
due to the idiotic patent issue which many just could not simply ignore.

(Now Microsoft seems to have seen the light after all and has subjected 
their patent to their Open Specification Promise, which might make the 
issue go away.  I'll leave it up to Debian, the ASF, or others to decide 
whether spf2.0 AKA Sender ID is now free-software compatible.)

And finally, the appeal to the IESG was not "ignored".  It was rejected.  
The reason being that the IESG did not want to take sides in the conflict 
and change the Sender ID I-D or even bar it from publication as an RFC.
On a technical level, the IESG even confirmed in their response[1] to the 
appeal that they "have found merit in [the] technical concerns".

When people discuss past matters as delicate as those above, I do expect 
some accuracy.  Otherwise the truth will fall through the cracks and 
disappear all too easily. 

References:
 1. http://www.ietf.org/IESG/APPEALS/appeal-response-julian-mehnle.txt

---

From: Douglas Otis <[email-masked]>
Date: Mon, 6 Nov 2006 17:22:05 -0800

On Nov 6, 2006, at 2:06 PM, wayne wrote:

>> initially required before subscribing to spf-discuss.  While it  
>> may have changed, this was the reason for not participating on  
>> that  list. The MARID list is still functional.
>
> Yes, I saw your post.  People pointed out to you that you were  
> wrong back then, and you are still wrong today.
>
> the promotion of SPF.  I have removed the suggestion that if you  
> think SPF is fundementally flawed that you should come here instead.

It seems this suggestion is still remembered, judging by advice given  
K.J.  There is not much point arguing about which list.  Something  
other than SPF must be considered as a solution.  Promotion of SPF  
effectively thwarts any consideration of alternatives on spf- 
discuss.  : (

>>> presented hard data in your Appendix A.  Your data doesn't back  
>>> up your 1000x claims.
>>
>> 45 pages in the addendum trace SPF resolving a single name at  
>> about 64:1 increase in traffic.
>
> Nice to see that you agree that your data doesn't back up your  
> claims, but even your 64:1 number is bogus.

How so?  The number assumes a local DNS resolver is seeded with  
attacking SPF script and MX records.  The same SPF script can use  
local-part macros to generate any number of attack queries.  The MX  
records can repeat following a negative caching interval.  The 64:1  
gain is based upon the email itself and just _one_ instance of SPF  
being executed.  The message overhead likely represents spam that  
would have been sent anyway.  The entire SPF related traffic  
represents a gift given an attacker by those executing SPF script. : (

The higher gains noted recognize compromised systems might utilize a  
provider's SMTP server.  The message can include a number of  
recipients as yet another multiplier.  In addition, each message  
might be assessed at multiple locations.  Each instance represents a  
multiplier of network network traffic generated by SPF script.  Just  
8 recipients evaluated at the MTA and MUA reach the gain of 1000:1.   
The local-part macro also facilitates canvassing recipients that make  
use of SPF, and those that do so twice. : (

-Doug

---

From: Douglas Otis <[email-masked]>
Date: Mon, 6 Nov 2006 17:29:40 -0800

On Nov 6, 2006, at 3:12 PM, Julian Mehnle wrote:

> Sorry to barge in on this particular topic (which is mostly  
> unrelated to the issues raised by K.J. Petrie or the alleged DoS  
> issue), but this ain't correct.  The v=spf1/pra re-use issue was  
> indeed discussed at the MAAWG meeting in Düsseldorf.  I'm not sure  
> what you mean when you say that "allowing record scoping" was  
> proposed, but what was actually proposed were two different things  
> (depending on whom I talked to):
>

It seems v=spf1/pra will be more disruptive than spf2.0/mailfrom?   
What is your view about forcing use of different scripts?

-Doug

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 07 Nov 2006 20:42:24 +0100

Douglas Otis wrote:

> Promotion of SPF effectively thwarts any consideration of alternatives
> on spf- discuss.

You'll tell us when you've found a better way to partition all IPs into
three sets PASS, FAIL, or DUNNO (with as many variants of DUNNO as you
need).

 
> How so?

Because 100/11 is a bit more than 9, and 100/12 is a bit less than 9.

> a gift given an attacker by those executing SPF script.

The stuff is called "SPF policy" or "SPF record", not "SPF script".

It's a simple sequence of mechanisms telling receivers to which of
the three sets PASS, FAIL, or DUNNO a given IP belongs.  

Three of the mechanisms (ip4, ip6, all) need zero DNS queries.
Two of the mechanisms (a, exists) need one additional DNS query.
One mechanism (include) needs two DNS queries (TXT + SPF).
One modifier (redirect=) behaves like include wrt DNS queries.

One mechanism (ptr) needs up to 10 DNS queries per connecting IP,
no matter how often it's used in various SPF policies evaluated
during that SMTP session.

One mechanism (mx) needs up to 10 queries.  There can be at most
10 mechanisms causing any additional DNS query at all per policy.

10+1 is 11, 10+2 is 12, 10*10 is 100, and 100/11 is about 9, not
64, 1000, or other random numbers you care to name.

Frank

---

From: Julian Mehnle <[email-masked]>
Date: Tue, 7 Nov 2006 20:02:35 +0000

Douglas Otis wrote:
> >  A. Accept the v=spf1/pra re-use and start promoting v=spf1 as
> >     having a different meaning than what had been defined back in 2003.
> >
> >     different) scheme in order to avoid the misinterpretation of
> >     v=spf1 records for PRA purposes.
>

If you mean that the interpretation of v=spf1 for the PRA scope would be 
more disruptive than recommending that everyone publish spf2.0 (essen- 
tially what B says) instead of v=spf1, then, yes, I'd agree.

> What is your view about forcing use of different scripts?

I don't understand what you're suggesting here.  What do you mean by
"forcing use of different scripts"?

---

From: Douglas Otis <[email-masked]>
Date: Tue, 7 Nov 2006 15:01:48 -0800

On Nov 7, 2006, at 11:42 AM, Frank Ellermann wrote:

>
> Douglas Otis wrote:
>
>> Promotion of SPF effectively thwarts any consideration of  
>> alternatives on spf- discuss.
>
> You'll tell us when you've found a better way to partition all IPs  
> into three sets PASS, FAIL, or DUNNO (with as many variants of  
> DUNNO as you need).

The RFC3123 permits more than 50 CIDRs per transaction when address  
registration is desired.  A better choice would be to authenticate  
the EHLO as an address literal or host name.  The EHLO in any form  
can then be associated with an originating domain.  Any number of  
flags may then indicate the nature of the association and the level  
of assurance offered.  With this technique only _one_ small  
transaction is made against a message targeted resource and is much  
safer from a DDoS concern.

For examples of such a scheme see:
http://www.sonic.net/~dougotis/id/draft-otis-dkim-dosp-02.html

> Because 100/11 is a bit more than 9, and 100/12 is a bit less than 9.

There is about 640 bytes of traffic for each of the 100 transactions  
representing 64,000 bytes of generated traffic.  When a message is  
about 1000 bytes, the ratio of SPF traffic per message is then 64:1  
or 64K bytes of DNS traffic per message.

> The stuff is called "SPF policy" or "SPF record", not "SPF script".

A possible >6K bytes of SPF script (a series of instructions carried  
out in a specific order) must be executed to obtain an answer.  The  
answer might involve the comparison of multi faceted PTR  
transactions, two stage MX transactions within CDIRs, and/or direct  
addresses transactions within CIDRs at macro generated locations  
compared against SMTP client addresses.  There is also checks for any  
answer from address transactions at macro constructed locations.  No  
answer is assured without executing the instructed transactions and  
applying the indicated Boolean logic.  This script also contains  
includes, redirects and default statements.  SPF is the epitome of a  
script.  In contrast, APL does not involve subsequent instructed  
transactions to obtain an answer and would not be described as a script.

-Doug

---

From: Douglas Otis <[email-masked]>
Date: Tue, 7 Nov 2006 15:41:34 -0800

On Nov 7, 2006, at 12:02 PM, Julian Mehnle wrote:

> I don't understand what you're suggesting here.  What do you mean  
> by "forcing use of different scripts"?

Obsoleting existing libraries and related scripts as needed due to  
the DDoS potential.

-Doug

---

From: Julian Mehnle <[email-masked]>
Date: Wed, 8 Nov 2006 00:27:14 +0000

> Obsoleting existing libraries and related scripts as needed due to

The SPF project isn't convinced just yet that there is significant 
potential for a DoS attack, and if there's any, how real it is, so any 
statements on consequences would be hypothetical at this time.  But trust 
us, we are taking this seriously.  However, we consider it unlikely that 
obsoleting v=spf1 and the existing libraries would be necessary to 
mitigate any serious DoS potential.  Tightening the limits, perhaps.

The problem with your analysis, Doug, is that (1) it attributes several 
attack vectors to SPF which are really orthogonal, like SMTP's multi- 
recipient feature or the use of many compromised systems for sending mail, 
and (2) with a high probability it overrates both the negative effects 
(like the victim/attacker traffic ratio) of an attack staged as described, 
and the net incentive for doing so in the first place.

We are currently investigating the issue further, so expect a thorough 
analysis from us within the coming weeks.

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 7 Nov 2006 21:10:40 -0500

> Obsoleting existing libraries and related scripts as needed due to

When you say the word script, do you mean SPF records (Please, this is meant 
to be a yes/no question, there's no need to write a dissertation that repeats 
the arguements made earlier in the thread.  I read those already.)?

If no, then please explain what you mean by a script.

Scott K

---

From: Douglas Otis <[email-masked]>
Date: Wed, 8 Nov 2006 09:15:29 -0800

On Nov 7, 2006, at 6:10 PM, Scott Kitterman wrote:

> When you say the word script, do you mean SPF records (Please, this  
> is meant to be a yes/no question, there's no need to write a  
> dissertation that repeats the arguements made earlier in the  
> thread.  I read those already.)?

The concern is with libraries that execute scripts labeled as SPF 1  
or 2 contained within DNS resource records.  A potential attack  
utilizing just the access of these resources records has not been  
described.

-Doug

---

From: Douglas Otis <[email-masked]>
Date: Wed, 8 Nov 2006 11:23:12 -0800

On Nov 7, 2006, at 4:27 PM, Julian Mehnle wrote:
>
> The problem with your analysis, Doug, is that (1) it attributes  
> several attack vectors to SPF which are really orthogonal, like  
> SMTP's multi-recipient feature or the use of many compromised  
> systems for sending mail, and (2) with a high probability it  
> overrates both the negative effects (like the victim/attacker  
> traffic ratio) of an attack staged as described, and the net  
> incentive for doing so in the first place.

Execution of SPF script is not orthogonal to SMTP's current use (or  
abuse).  An SPF script exploit requiring little (if any) resources of  
the attacker offers an incentive.  Difficult forensics offers an  
incentive.  Acceptance of responsibility and subsequent changes to  
infrastructure requires time.  The delay for an effective response  
offers an incentive.  These incentives and risks grow as SPF script  
execution increases.  Protection necessitates the general removal of  
the related scripts.  Malicious script detection at each DNS is not  
reasonable.

-Doug

---

From: Dean Anderson <[email-masked]>
Date: Wed, 8 Nov 2006 15:56:05 -0500 (EST)

I don't recognize Greg Hewgill, so I presume he has a genuine question.

> "non-commercial spam" would look like today.

Tried purchasing anything?  I have.  I opened a bank account just for a
special credit card.  But it was never charged. If money changed hands,
I figured it would be easy to find them. No money ever changed hands,
even though, ostensibly, the spam _looks_ commercial. But wait, if no
money changes hands, it isn't commercial, no matter what it _looks_
like.  I also found a few blacklists abusing open relays. I did this by
creating non-production relays, logging TCP connections to them, and
then submitting them to the blacklist for scanning. After only scanning
by the blacklist, they started getting abused. There's more, but you get
the idea.  And now, the abusers are easily distinguished from the
genuine mailers by CAN-SPAM---this is why "anti-spam" people hate
CAN-SPAM so much. (quotes because they really aren't anti-spam people)

By contrast, the genuine commercial bulk emailers never use proxies,
never use viruses, nor other abuse tactics. The genuine commercial bulk
emailers comply with CAN-SPAM, and rarely violate the law.

Some time ago, I created a classification of spam types.
http://www.av8.net/SpamTypes.txt Certain people have tried to discredit
this for a long time. Usually, they try to confuse CAN-SPAM compliant
mailers with non-compliant abusers by lumping them all together. They
realy, really, really hate the notion that there might be a distinction.  
The reason why is plain.

The very same people also tried to claim that the ECPA didn't apply to
ISPs. (I said it did, and cited the law and a number of court cases, but
to no avail.) Then there was U.S. V Councilman, a criminal case.  
People don't dispute me on this anymore, but no one has actually
admitted to being wrong. I suppose I shouldn't be too surprised by that.  
But it discredits them, and credits me.

And about 10 years ago, the same people disputed that anti-trust law
would apply to blacklists-- again, with no basis in law or fact. Exactis
V.  MAPS proved me right.  Trolls still dispute this, however, but still
have no basis for their claims.  They typically assert that Exactis V.  
MAPS isn't a "precedent". Literally, that is true: Exactis V. MAPS was
settled out of court. However, one doesn't need a court precedent before
the law will apply. The law is clear; the constitutionality of the law
is clear; and there is no reasonable dispute on the law to take to
court.  Hence, MAPS lawyer was chastised by the Judge.

BTW, this may be interesting: 

Subject: Re: SPF [WAS: Best practices for hosting web but NOT email?]

Hmm. Ehlke...sounds familiar. So familiar, I think you already know the
answer to your question. This is a response to a known troll, so hit
delete now, unless you like history.

Exactis V. MAPS. 

Exactis sued for anti-trust violation, just as I predicted would happen
about 10+ years ago. I made that assertion from information based on
queries I made around 1990 to the LPF attorneys regarding the operation
of boycotts.  The script-kiddie trolls have long disputed this---of
course, with no basis for their claims.  They assert there is a first
amendment right to operate a blacklist.  There isn't.  They assert
sometimes that there are no laws without court cases (e.g. the federal
register reference below). That isn't true either, but it hasn't stopped
them.  Of course, they lie about lots things--Just look up 130.105/16 in
SORBS.

But MAPS countered the suit with a First Amendment assertion, just as
trolls asserted who disputed my reports. The constitutional legal
argument against anti-trust that was tossed about a century back.
Indeed, the MAPS lawyer was chastisted for this frivolous defense, and
the Judge recessed. MAPS settled during recess, giving Exactis
everything it demanded.  This suit should never have reached court. The
court papers are interesting, though. A good read is the memoranda in
support of the TRO by Exactis. This is at
http://www.dotcomeon.com/exactis1.html.  It gives a good look at the
MAPS true nature. Compare what you read in the Memo with page 254 of
Brian McWilliams book "Spam Kings", where MAPS employees are working for
_spammers_, "washing" their lists from spam-trap addresses.  Then
compare with additional information on http://www.iadl.org.

I'm surprised you didn't dispute the ECPA assertions, too. Or has U.S.  
V. Councilman finally convinced you that I was right about that, too?  
It is always interesting how you tend to never admit your past mistakes.

                --Dean

On Tue, 7 Nov 2006, Pete Ehlke wrote:

> On Mon Nov 06, 2006 at 15:16:42 -0500, Dean Anderson wrote:
> >
> >Blocking non-spam email is a violation of federal anti-trust law
> >(participation in unlawful group boycott) and also state and federal
> >electronic privacy laws (no authorization to block non-spam email).  
> >Some letters to Verizon may be necessary.
> >
> ( Just can't resist tossing a cookie to the troll... )
> 
> And your citations in the Federal Reporter for the decisions that 
support
> these contentions are exactly what?
> 
>

---

From: Douglas Otis <[email-masked]>
Date: Wed, 8 Nov 2006 18:37:58 -0800

On Nov 8, 2006, at 12:56 PM, Dean Anderson wrote:

> I also found a few blacklists abusing open relays. I did this by  
> creating non-production relays, logging TCP connections to them,  
> and then submitting them to the blacklist for scanning. After only  
> scanning by the blacklist, they started getting abused. There's  
> more, but you get the idea.  And now, the abusers are easily  
> distinguished from the genuine mailers by CAN-SPAM---this is why  
> "anti-spam" people hate CAN-SPAM so much. (quotes because they  
> really aren't anti-spam people)

It is not surprising that listing an open-relay exposes such services  
defeating a security through obscurity scheme.  While the CAN-SPAM  
act is not entirely bad, the greatest harm was from over-riding  
reasonable State Opt-in requirements.  This act still allows  
acceptable use policies.  Neither this act nor the suit you mention  
changed these acceptable use policies, and there is a growing  
adoption of these policies worldwide.  Opt-in use policies continue  
despite CAN-SPAM requirements established for Opt-out.  In general,  
it remains a bad idea for a recipient to Opt-out unless they are sure  
of the sender.  Just as it remains a bad idea for recipients to run  
scripts in messages or obtain photos when they are unsure.  It also  
remains a bad idea for the sender to post bulk email unless their  
mailing lists have been confirmed by the recipients.

-Doug

---

From: Julian Mehnle <[email-masked]>
Date: Fri, 10 Nov 2006 00:47:36 +0000

Dean Anderson wrote:
> > Is that right? Virtually all the spam I see relates to pharmacy,
> > stock, mortgage, pirated software, or are email worms. The whole idea
> > of each of those are to make money for somebody. I don't even know
> > what "non-commercial spam" would look like today.
>
> like.

And this specific experiment of yours is representative exactly _how_?

If money never changes hands, how come the SPF support team gets support 
requests like the following?

| Topic: Support request
| Name: M* P*
| 
| My e-mail at Mp*@*.net is not compatible somehow with you recieving
| messages.  Please use my alternate e-mail address at [email-masked].  My 
| question is-where are the meds I ordered?  Payment has been deducted from
| my bank account, but I have recieved no meds.  Thankyou for your
| help..................MP*

(Yes, I had a good laugh when reading that.)

Oh, right, next you're going to claim that this was a legitimate buying 
transaction with a non-spammer, not some poor bastard falling for a 
spamming fraudster (before then also tripping over an SPF policy 
violation).

Sorry for having barged in once more, but I found your argument too funny 
not to comment on it...

---

From: Dean Anderson <[email-masked]>
Date: Fri, 10 Nov 2006 12:00:37 -0500 (EST)

> | help..................MP*

These questions happen to real businesses all the time, and don't
usually represent frauds, but rather honest mistakes; mistakes that the
business corrects promptly.  You example doen't show any evidence that
the person in that case was defrauded.

> (Yes, I had a good laugh when reading that.)

Too bad. I'm sure your job isn't for your amusement.  Wait....  You are
handling support for a pharmacy?  A genuine commerical bulk-emailing
pharmacy?

> Oh, right, next you're going to claim that this was a legitimate
> buying transaction with a non-spammer, not some poor bastard falling
> for a spamming fraudster (before then also tripping over an SPF policy
> violation).

Yes, you forgot to mention whether the person bought a product from a
CAN-SPAM compliant emailer, or using a web site not involved in bulk
email, and then just had a problem with email, later.  Funny that, huh?  
But it seems to have come from your company....

But you do have a point about there existing real frauds.  However, real
frauds are already crimes. SPF is not going to stop real frauds, nor
real criminals.  Criminals will put up SPF records. Spammers were in
fact the early SPF adopters. Criminal con-artists used to (probably
still do) rent storefronts, too, to appear legitimate during the con.  
The notion that SPF records somehow prove email legitimacy just helps
promote frauds by confusing users about what defines legitimacy.

I'm not saying that there aren't real frauds out there.  But I am saying
that the 600 spams I recieve a day mostly do not represent real frauds.  
Only about 70 of them represent genuine commercial bulk emailers.  
Practically all of the ~70 are CAN-SPAM compliant That leaves an awful
lot that are neither real frauds, nor genuinely commercial.

		--Dean