Subject: RE: Spammers answering TMDA Queries

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Tue, 2 Oct 2007 18:02:00 -0700

Sounds reasonable to me.

Tdma for personal email protection is rude and unacceptable. For mailing lists it is entirely acceptable. Cost far outweighs benefit as the inconvenience to the single sender is much less than the benefit to the community.

Should also consider if spf or dkim checks could cull the paypal spam.

Sent from my GoodLink Wireless Handheld (www.good.com)

 -----Original Message-----
From: 	Russ Housley [mailto:[email-masked]]
Sent:	Tuesday, October 02, 2007 04:12 PM Pacific Standard Time
To:	[email-masked]
Subject:	Spammers answering TMDA Queries

The Secretariat tells me that Spammers are responding to TDMA queries 
so that their mail goes through.  They have made the suggestion that 
we clear the list of people once per year.  This would mean that a 
legitimate user of a list that uses TDMA would get a TDMA query once 
a year if they are not subscribed to any ietf.org mail list.  There 
is no TDMA query for people who are on at least one ietf.org mail list.

Here is the info that I have:

>  > Russ wants to know how many people have responded to the TMDA
>  > challenge but are not on any IETF mailing list.
>
>1025 mail addresses have "confirmed" their address.  I would bet that
>at least 20% of the confirmed are spam addresses (or autoconfirmed
>addresses)

Thoughts?

Russ 

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf
Sounds reasonable to me.

Tdma for personal email protection is rude and unacceptable. For mailing lists it is entirely acceptable. Cost far outweighs benefit as the inconvenience to the single sender is much less than the benefit to the community.

Should also consider if spf or dkim checks could cull the paypal spam.

Sent from my GoodLink Wireless Handheld (www.good.com)

 -----Original Message-----
From: 	Russ Housley [mailto:[email-masked]]
Sent:	Tuesday, October 02, 2007 04:12 PM Pacific Standard Time
To:	[email-masked]
Subject:	Spammers answering TMDA Queries

The Secretariat tells me that Spammers are responding to TDMA queries 
so that their mail goes through.  They have made the suggestion that 
we clear the list of people once per year.  This would mean that a 
legitimate user of a list that uses TDMA would get a TDMA query once 
a year if they are not subscribed to any ietf.org mail list.  There 
is no TDMA query for people who are on at least one ietf.org mail list.

Here is the info that I have:

>  > Russ wants to know how many people have responded to the TMDA
>  > challenge but are not on any IETF mailing list.
>
>1025 mail addresses have "confirmed" their address.  I would bet that
>at least 20% of the confirmed are spam addresses (or autoconfirmed
>addresses)

Thoughts?

Russ 

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf
_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Fred Baker <[email-masked]>
Date: Thu, 4 Oct 2007 07:35:19 -0400

> Should also consider if spf or dkim checks could cull the paypal spam.

how many of us are now sending with DKIM or Microsoft's scheme? It  
might be worthwhile making ietf.org apply a policy to senders that  
would recognize normal participants and disallow known spam domains.

---

From: John Levine <[email-masked]>
Date: 4 Oct 2007 15:43:48 -0000

>would recognize normal participants and disallow known spam domains.

Um, spammers haven't sent mail from "known spam domains" since about
2001.  These days spam has 100% forged return addresses.  DKIM and
Sender-ID help tell forgeries from legit mail, but I haven't heard
anyone say that forged mail purporting to be from list participants is
an issue.

Unless I am missing something, the amount of spam leaking into IETF
lists is currently about zero.  What problem are we trying to solve?

R's,
John

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Thu, 4 Oct 2007 08:56:18 -0700

The problem is the amount of time it is taking to moderate mail sent by non subscribers.

So far the score for KEYPROV has been 98% spam. But there were a couple of messages that were very important that got trapped. 

> From: John Levine [mailto:[email-masked]] 
> Sent: Thursday, October 04, 2007 11:44 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: Spammers answering TMDA Queries
> 
> Um, spammers haven't sent mail from "known spam domains" 
> since about 2001.  These days spam has 100% forged return 
> addresses.  DKIM and Sender-ID help tell forgeries from legit 
> mail, but I haven't heard anyone say that forged mail 
> purporting to be from list participants is an issue.
> 
> Unless I am missing something, the amount of spam leaking 
> into IETF lists is currently about zero.  What problem are we 
> trying to solve?
> 
> R's,
> John
> 
>

---

From: Fred Baker <[email-masked]>
Date: Thu, 4 Oct 2007 13:43:51 -0400

> The problem is the amount of time it is taking to moderate mail  

yes. For example, every email from @cisco.com is dkim-signed. The  
IETF can automagically dump any such email that is not signed, or for  
which the signature doesn't check out. I know that [email-masked] is  
one of many commonly-spoofed email addresses - I can tell that from  
the backscatter I find in my junk box.

For how many of us is that true?

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Thu, 4 Oct 2007 12:16:39 -0700

Absolutely, and in fact I see mailing list management as a natural early adopter for DKIM filtering.

The vast bulk of the spam I am moderating off the KEYPROV list is phishing spam against five particular addresses, all of which implement DKIM. My workload as a moderator can be cut by 80% by rejecting any message from those addresses that is not DKIM signed.

Mailing lists do not in general subscribe to mailing lists so the normal arguments against discarding messages for failing DKIM compliance do not apply. 

> From: Fred Baker [mailto:[email-masked]] 
> Sent: Thursday, October 04, 2007 1:44 PM
> To: Hallam-Baker, Phillip
> Cc: John Levine; [email-masked]
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> 
> > The problem is the amount of time it is taking to moderate 
> mail sent 
> > by non subscribers.
> 
> yes. For example, every email from @cisco.com is dkim-signed. 
> The IETF can automagically dump any such email that is not 
> signed, or for which the signature doesn't check out. I know 
> that [email-masked] is one of many commonly-spoofed email 
> addresses - I can tell that from the backscatter I find in my 
> junk box.
> 
> -----BEGIN PGP SIGNATURE-----
> 
> iD8DBQFHBSZXbjEdbHIsm0MRAt9UAJ9xVCpDMdC3spmPkmsTFCqZTNWY6ACffR0R
> lUEQvoA8i0OZXuU4r8TroLs=
> =0xUE
> -----END PGP SIGNATURE-----
>

---

From: Keith Moore <[email-masked]>
Date: Thu, 04 Oct 2007 15:38:51 -0400

> Absolutely, and in fact I see mailing list management as a natural
early adopter for DKIM filtering.

the problem I have with DKIM filtering is that it is only effective for
domains that can reasonably insist that all of the mail originated by
users at that domain go through that domain's submission servers.   this
is a corner case, not the general case.   sure the spammers will learn
to not use DKIM domains, but they'll just move to other domains, and the
vast majority of domains won't be able to use DKIM without seriously
impairing their users' ability to send mail.  of course, some of the
large ISPs and MSPs like it that way. 

frankly I don't think IETF should have backed a proposal that was so
unfairly biased toward a particular business model.

Keith

---

From: Michael Thomas <[email-masked]>
Date: Thu, 04 Oct 2007 13:03:41 -0700

Keith Moore wrote:
> is a corner case, not the general case.   

Back in the day, we didn't have any of this VeePeeEn tomfoolery. I could
just telnet in and that was that. I'm sure that our IT folks paid dearly 
in time,
equipment, and support to throw up that wall, yet they did it and as far 
as I
can tell we all survived the move.  I don't see anything especially 
different
with mail: if you want accountability, you have to do real live work -- 
part
of which is placing restrictions on access. TANSTAAFL.

> sure the spammers will learn
> to not use DKIM domains, but they'll just move to other domains, 
This is a feature, not a bug: I don't have to outrun the bear, I just need
to outrun you.

       Mike

---

From: SM <[email-masked]>
Date: Thu, 04 Oct 2007 12:53:01 -0700

>the backscatter I find in my junk box.

Cisco.com would have to publish a SSP (draft-ietf-dkim-ssp) 
indicating that it signs all mail.  Alternatively, one might apply 
heuristics to increase the spam probability if an email from 
@cisco.com is not dkim-signed.

Spam can pass SPF, Sender-ID and are even DK and DKIM signed 
nowadays.  One can't blame spammers for not being early adopters. :-)

TMDA may cause backscatter.  That could be avoided by rejecting email 
which is most likely spam at the MTA level and have the rest held for 
manual review.  That should decrease the amount (98%) of mail 
requiring manual review.  With such a system, you still need an 
alternate contact channel which the sender can use to get his/her 
message though if the it is rejected.

Sometimes it's easier to alleviate the problem instead of solving 
it.  draft-ietf-sipping-spam-05 ( 
http://www.ietf.org/internet-drafts/draft-ietf-sipping-spam-05.txt ) 
provides an interesting insight.

Regards,
-sm

---

From: Simon Leinen <[email-masked]>
Date: Thu, 04 Oct 2007 22:08:02 +0200

> For how many of us is that true?

FWIW, about 12% (14 out of 114) of the active non-spam senders to this
list had DKIM-Signature headers in the past two weeks.  I don't know
enough about DKIM to tell whether the same assumption holds for the
non-cisco.com sender domains (mostly gmail.com plus a few smaller
ones): that mail from them can be considered spoofed if the DKIM
headers are absent.

: leinen@diotima[lists.ietf.censored]; cat `egrep -l -i '^DKIM-Signature:' *`  | egrep -i '^From:' | sort | uniq -c | wc -l
14
: leinen@diotima[lists.ietf.censored]; cat * | egrep -i '^From:' | sort | uniq -c | wc -l
114
-- 
Simon.

---

From: Fred Baker <[email-masked]>
Date: Thu, 4 Oct 2007 16:17:00 -0400

I will disagree with you there. DKIM allows the concept of a  
corporate signature - "I'm Cisco and I know who my employee is" or  
"I'm Yahoo and I know who my user is" - but it doesn't require it.  
What it does require is that if you are not going to use the  
corporate servers you need to provide and support the signature you  
use. The former is, IMHO, an important step in scalability. The  
latter is status quo with PGP and S/MIME.

On Oct 4, 2007, at 3:38 PM, Keith Moore wrote:

> and the

---

From: Fred Baker <[email-masked]>
Date: Thu, 4 Oct 2007 16:20:29 -0400

On Oct 4, 2007, at 3:38 PM, Keith Moore wrote:

> the problem I have with DKIM filtering is that it is only effective  
> for domains that can reasonably insist that all of the mail  
> originated by users at that domain go through that domain's  
> submission servers.   this is a corner case, not the general  
> case.   sure the spammers will learn to not use DKIM domains, but  
> they'll just move to other domains, and the vast majority of  
> domains won't be able to use DKIM without seriously impairing their  
> users' ability to send mail.  of course, some of the large ISPs and  
> MSPs like it that way.

well, at some point it seems to me that we can take the next step,  
which is to require all email to IETF lists to be signed. Offer to  
accept DKIM, Microsoft's (as in "gmail"), PGP, and S/MIME, but  
require the signature and require it to verify.

We're probably not yet at that point, but for companies that follow  
the kind of policy in question, we can take a step.

---

From: Fred Baker <[email-masked]>
Date: Thu, 4 Oct 2007 16:29:59 -0400

As I understand it, the spammers were among the first adopters of  
dkim. My point is not that "spammers don't sign". Some spammers don't  
sign and can (eventually, not now) be dropped because they don't.  
Other spammers do sign and can be identified and shunned by policy.  
But certainly, spammers spoofing source addresses will be unable to  
sign as the spoofed sending domain, and can have their traffic  
summarily discarded as either being unsigned but purporting to come  
from a domain that signs or as having invalid signatures. Traffic  
with spoofed source addresses from domains that sign needs no  
moderation. The moderation load is the problem we're solving.

On Oct 4, 2007, at 4:08 PM, Simon Leinen wrote:

> : leinen@diotima[lists.ietf.censored]; cat `egrep -l -i '^DKIM- 
> Signature:' *`  | egrep -i '^From:' | sort | uniq -c | wc -l
> 14

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Thu, 4 Oct 2007 13:25:05 -0700

I fail to see your point here.

Anyone can deploy DKIM, there is nothing unfair about the DKIM architecture.

The 'unfairness' that you appear to be complaining about is that DKIM solves a problem that only targets a relatively small number of Internet domains, although the effects of that attack are seen by everyone. 

Impersonation of a trusted brand is always going to assit a social engineering attack if this is possible. I do not understand the ideological calculus under which we should do nothing to protect consumers against attacks of this nature because we can't all have a trusted brand.

> From: Keith Moore [mailto:[email-masked]] 
> Sent: Thursday, October 04, 2007 3:39 PM
> Cc: Fred Baker; [email-masked]
> the problem I have with DKIM filtering is that it is only 
> effective for domains that can reasonably insist that all of 
> the mail originated by
> to not use DKIM domains, but they'll just move to other 
> domains, and the vast majority of domains won't be able to 
> use DKIM without seriously impairing their users' ability to 
> send mail.  of course, some of the large ISPs and MSPs like 
> it that way. 
> 
> frankly I don't think IETF should have backed a proposal that 
> was so unfairly biased toward a particular business model.
> 
> Keith
> 
>

---

From: Keith Moore <[email-masked]>
Date: Thu, 04 Oct 2007 16:52:35 -0400

>> submission servers.   this is a corner case, not the general case.   
>
> Back in the day, we didn't have any of this VeePeeEn tomfoolery. I
> could just telnet in and that was that. I'm sure that our IT folks
> paid dearly in time, equipment, and support to throw up that wall, yet
> they did it and as far as I can tell we all survived the move.  I
> don't see anything especially different with mail: if you want
> accountability, you have to do real live work -- part of which is
> placing restrictions on access. TANSTAAFL.
what you are failing to see is just how much reliance on VPNs (and
source IPs) to do authentication cripples the network.  sure it's better
than nothing, but it's also very inflexible and an architectural dead end.

(and the problem with TANSTAAFL is that you can use it to justify any
kind of brain damage you want, as long as there's some minor associated
benefit)
>> sure the spammers will learn to not use DKIM domains, but they'll
>> just move to other domains, 
> This is a feature, not a bug: I don't have to outrun the bear, I just
> need to outrun you.
I'll remind you that as a condition to working in IETF we are all
pledged to use our judgment as to what's best for the Internet as a
whole...not just for those who can run faster than others.

Keith

---

From: Keith Moore <[email-masked]>
Date: Thu, 04 Oct 2007 17:08:21 -0400

> Anyone can deploy DKIM, there is nothing unfair about the DKIM architecture.
it artificially changes the relative value of domain names.  it makes
them more like brand names, where you have to work to build a domain's
reputation in order to get people to trust it.  it means that domains
which are associated with large user communities with a good reputation
will be more trusted than domains with small user communities, even when
those domains are equally diligent.  in that way DKIM favors the
interests of large concerns over small ones.  so it's not surprising
that several large concerns backed it.  but that doesn't mean it's a
good thing for the Internet as a whole.
> The 'unfairness' that you appear to be complaining about is that DKIM solves a problem that only targets a relatively small number of Internet domains, although the effects of that attack are seen…
indeed, DKIM might help address the phishing problem, if that's what
you're talking about.  and large concerns are disproportionally affected
by phishing.  but ultimately I think there's only a small chance of DKIM
helping the phishing problem much, because of user interface issues and
because there are lots of ways to fool people into thinking that they're
responding to a FemtoSquishy email without having femtosquishy.com in
the From address or signature. 
> Impersonation of a trusted brand is always going to assit a social engineering attack if this is possible. I do not understand the ideological calculus under which we should do nothing to protect c…
using DKIM to discourage phishing is a different use case than using it
to authenticate to IETF lists.  just because it might work well for the
former (if indeed it does) does not mean it can be relied on to work
well for the latter.

Keith

---

From: Dave Crocker <[email-masked]>
Date: Thu, 04 Oct 2007 14:09:57 -0700

Folks,

Fred Baker wrote:
> I will disagree with you there. DKIM allows the concept of a corporate 
> signature - "I'm Cisco and I know who my employee is" or "I'm Yahoo and 
> I know who my user is" - but it doesn't require it. What it does require 

This is a key point.  A DKIM is signature is an affirmative statement of 
responsibility by the Domain owner, *for that message*.  So when a signature 
is present, you have an accountable entity.

Whether you actually have any trust in that entity is a separate (and more 
interesting) question.  Assessment mechanisms for an authenticated domain 
name, do not have any standards yet.

For that matter, a standard that signals that a site signs all mail containing 
their domain in a particular field is also a matter still awaiting 
standardization.  At the moment, the "I sign everything" construct is ad hoc. 
  A domain can know it about itself, of course, so that cisco can detect 
inbound mail that forges cisco's domain.  For now, other recipient sites 
require ad hoc lists.

What DKIM has not yet been established for, is filtering out bad mail. 
Although the "I sign everything" construct is expected to help this, there is 
no meaningful track record that it really works.

More generally, this thread has been dominated by views that there are single, 
simple, well-understood solutions for the problem(s) being cited.  Among the 
anti-abuse community, the consensus is that effective mechanisms are not 
singular, not simple, and not yet well-understood.

On the average, the public community -- and I'm afraid that the IETF mailing 
list appears to fall into the broad, non-technical category -- entirely 
underestimates the sophistication of modern email abuse mechanisms.

John Levine and others have been making this point on the thread, but it does 
not seem to be registering.

Having mail receivers at ietf.org take note of email authentication is a Good 
Thing.  Assuming that this is going to "solve" any particular email problem is 
not.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Keith Moore <[email-masked]>
Date: Thu, 04 Oct 2007 17:08:21 -0400

Keith

---

From: Dave Crocker <[email-masked]>
Date: Thu, 04 Oct 2007 14:09:57 -0700

Folks,

d/

-- 

   Dave Crocker

---

From: Michael Thomas <[email-masked]>
Date: Thu, 04 Oct 2007 15:44:28 -0700

> than nothing, but it's also very inflexible and an architectural dead end.

C'est la guerre. In fact, I'm well aware of all of those things, and 
I'll even allow
that our IT folks were probably aware of all of those things too -- they 
undoubtedly
took a lot of flak from the Eldar who probably said the same thing. I'm 
also pretty
sure that they would dismiss anybody who told them to tear out their VPN 
gear
because it cripples the network and is an architectural dead. Same goes 
for email.

> whole...not just for those who can run faster than others.
I guess I must have been in the bar when they had that pledge of 
allegiance. But
even allowing that there is any such pledge, to the degree that we 
enable domains
to control who uses their name and be accountable when they behave badly is
certainly a net good thing IMO. Your original makes it sound like 
there's some
inherent right to be heard. There isn't. If you don't want to be 
accountable, then
maybe I just don't want to bother sorting your wheat from chaff.

       Mike

---

From: Keith Moore <[email-masked]>
Date: Thu, 04 Oct 2007 20:25:03 -0400

> certainly a net good thing IMO.
domains don't behave well or badly.  they're just names.   and I don't
think it's in the internet's interest to require people to associate
themselves with what is essentially a brand name in order to be heard. 
using DKIM for spam filtering pretty much does that.

---

From: Douglas Otis <[email-masked]>
Date: Thu, 4 Oct 2007 22:13:28 -0700

On Oct 4, 2007, at 5:25 PM, Keith Moore wrote:
>> allegiance. But even allowing that there is any such pledge, to  
>> the degree that we enable domains to control who uses their name  
>> and be accountable when they behave badly is certainly a net good  
>> thing IMO.
>
> domains don't behave well or badly.  they're just names.   and I  
> don't think it's in the internet's interest to require people to  
> associate themselves with what is essentially a brand name in order  
> to be heard.  using DKIM for spam filtering pretty much does that.

DKIM might ensure a message, about to be dropped, generates a non- 
delivery notification instead.  With extensions to DKIM, such as TPA- 
SSP, even email-addresses within different domains from those used  
for DKIM signing could make assurances.  When a message hits a snag,  
TPA-SSP offers assurances that the domain in question is not being  
spoofed.  TPA-SSP is extensible and allows a user to associate their  
email domain with any number of DKIM signing domains.

Individuals may be where TPA-SSP finds support.  TPA-SSP also allows  
sub-domains differentiate signing policies.  Secure use of sub- 
domains and Third-Party domains might be a feature corporations put  
to good use as well.  The TPA-SSP mechanism allows principal domains  
to sign transactional emails and yet safely permit employees to send  
to mailing lists that also sign with DKIM.

DKIM can be very flexible.  However, the DKIM cryptographic process  
may place a sizeable burden upon receivers, especially when spam is  
in excess of 99%.

-Doug

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 8 Oct 2007 13:37:34 +0200

> TMDA may cause backscatter.

After an SPF PASS the "backscatter" by definition can't hit an
innocent bystander.  By the same definition any "backscatter"
after an SPF FAIL hits an innocent bystander, and therefore is
net abuse.

> provides an interesting insight.

It certainly explains why [18]...[21] are unnecessary for SIP ;-)

 Frank

---

From: Tony Finch <[email-masked]>
Date: Mon, 8 Oct 2007 23:56:06 +0100

On Thu, 4 Oct 2007, Keith Moore wrote:
>
> the vast majority of domains won't be able to use DKIM without seriously
> impairing their users' ability to send mail.

You seem to be assuming that the vast majority of domains have really
shitty message submission servers or connectivity. Maybe true, but if so
they're already losing so much that lack of DKIM probably doesn't matter
to them.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
IRISH SEA: SOUTHERLY, BACKING NORTHEASTERLY FOR A TIME, 3 OR 4. SLIGHT OR
MODERATE. SHOWERS. MODERATE OR GOOD, OCCASIONALLY POOR.

---

From: Keith Moore <[email-masked]>
Date: Mon, 08 Oct 2007 19:54:12 -0400

Tony Finch wrote:
> shitty message submission servers or connectivity.
It's a combination of several things - one, requiring that a domain
operate its own mail submission servers which sign their mail (and all
that that implies, like maintaining the private keys).  Two, many
domains will be too small to develop enough of a reputation to be
whitelisted, and any spammer can create a temporary domain which will
have about as good a reputation as the vast majority of those domains. 
Three, as long as people use Windows boxes, spammers will be able to
compromise them and hijack them to use them to originate mail on behalf
of their domains, thus degrading those domains' reputation. 

So basically if you're a small domain, you're SOL.  If you're a large
domain, people can't afford to blacklist you unless you originate a lot
of spam anyway.

Keith

---

From: Douglas Otis <[email-masked]>
Date: Mon, 8 Oct 2007 17:59:13 -0700

On Oct 8, 2007, at 4:54 PM, Keith Moore wrote:

>>> the vast majority of domains won't be able to use DKIM without  
>>> seriously impairing their users' ability to send mail.
>>
>> You seem to be assuming that the vast majority of domains have  
>> really shitty message submission servers or connectivity.
>
> operate its own mail submission servers which sign their mail (and  
> all that that implies, like maintaining the private keys).  Two,  
> many domains will be too small to develop enough of a reputation to  
> be whitelisted, and any spammer can create a temporary domain which  
> will have about as good a reputation as the vast majority of those  
> domains.
> Three, as long as people use Windows boxes, spammers will be able  
> to compromise them and hijack them to use them to originate mail on  
> behalf of their domains, thus degrading those domains' reputation.
>
> So basically if you're a small domain, you're SOL.  If you're a  
> large domain, people can't afford to blacklist you unless you  
> originate a lot of spam anyway.

Keith,

The DKIM component that establishes reputation is being discussed  
within the DKIM WG.  The DKIM signature offers an alternative to the  
IP address which serves as perhaps the only other assured basis for  
reputation.  Of course the IP address also shares all of these  
problems.  A DKIM signature can help avoid some of the reputation  
problems associated with shared use of an IP address (which is a  
larger problem for smaller domains).  For larger domains, there might  
be some concern related to replay abuse, where again, smaller domains  
also enjoy an advantage in being able to squelch compromised systems.

Don't be too quick to condemn DKIM.  There should be a simple  
mechanism which allows email-domains to autonomously authorize DKIM- 
domains.  This feature should defray some of your concerns.   
Delegating a zone of one's domain would be expensive to manage but is  
currently the only means now permitted.

-Doug

---

From: Keith Moore <[email-masked]>
Date: Mon, 08 Oct 2007 21:23:53 -0400

> mechanism which allows email-domains to autonomously authorize
> DKIM-domains.  
that feature should have been there from day one.

Keith

---

From: Douglas Otis <[email-masked]>
Date: Mon, 8 Oct 2007 19:06:31 -0700

On Oct 8, 2007, at 4:37 AM, Frank Ellermann wrote:

> After an SPF PASS, the "backscatter" by definition can't hit an  
> innocent bystander.  By the same definition any "backscatter" after  
> an SPF FAIL hits an innocent bystander, and therefore is net abuse.

There is a real risk SPF might be used as basis for acceptance,  
rather than just for qualifying DSNs.  As a basis for acceptance,  
this can cause email to fail.  The macro expansion of SPF records  
permits the _same_ DNS record within a spam campaign to generate a  
large number of subsequent and different DNS transactions to be sent  
by recipients to "innocent bystanders".  Much of the danger of auto  
responses has to do with DDoS concerns.  Unfortunately, SPF  
represents a far graver concern than that caused by auto-responses.

A safer approach would be to format all DSNs per RFC3464 and remove  
original message content.  This reduces incentives for abusing the  
automated responses.  Mailman made a mistake where an error caused a  
DSN that returned original content without first verifying the  
validity of the return path.  Had TMDA been a requisite for initial  
acceptance, just those white-listed would have been prone to this error.

-Doug

---

From: "Frank Ellermann" <[email-masked]>
Date: Tue, 9 Oct 2007 12:59:36 +0200

Douglas Otis wrote:

> There is a real risk SPF might be used as basis for acceptance

You can combine white lists with SPF PASS as with DKIM "PASS",
the risk is very similar.   

> Much of the danger of auto responses has to do with DDoS 
> concerns.

It depends on the definition of "DDoS".  From my POV as POP3
user over a V.90 connection 10,000 unsolicited mails are just
bad, no matter what it is (spam, worm, DSN, or auto-response).

It's not really a "DDoS".  SPF at least helped me to get rid
of the bogus DSNs and other auto-responses since three years,
smart spammers are not interested to forge SPF FAIL protected
addresses.

BTW, I think the definition of "Joe job" in the sieve EREJECT
draft is obsolete, the mere abuse of "plausible" addresses is
no "Joe job" and IMO also not a real DDoS.  But it's certainly
bad for the victims, it can be bad enough to make a mailbox
unusable for some victims.
 
> A safer approach would be to format all DSNs per RFC3464 and 
> remove original message content.

I'd hope that a majority of receivers already does this, that's
state of the art for some years now.  Or rather "truncate" is
state of the art, not complete removal of the body.

> Mailman made a mistake where an error caused a DSN that returned
> original content without first verifying the validity of the
> return path.

Auto responders aren't in a good position to verify the validity
of the return path.  Good positions to do this are the MSA based
on RFC 4409 and later the MX based on RFC 4408.

 Frank

---

From: Tony Finch <[email-masked]>
Date: Tue, 9 Oct 2007 12:21:24 +0100

On Mon, 8 Oct 2007, Keith Moore wrote:
> that that implies, like maintaining the private keys).

That's just part of running a mail system.

> Two, many domains will be too small to develop enough of a reputation to
> be whitelisted, and any spammer can create a temporary domain which will

Free domain tasting is a problem that affects lots of reputation system,
not just ones based on DKIM. If ICANN were to eliminate it lots of things
would become easier.

Also, at the moment negative reputation is more useful (or at least easier
to use) than positive reputation so I don't see neutral reputation as a
bad thing (er, by definition it isn't).

> of their domains, thus degrading those domains' reputation.

The criminals can steal infected users' online banking credentials too,
which is far more worrying. Everyone has to keep their networks clean for
many reasons, not just spam.

Tony.

---

From: Keith Moore <[email-masked]>
Date: Tue, 09 Oct 2007 09:11:45 -0400

> That's just part of running a mail system.
yes, but it's not inherently part of running a mail domain.  it's
unreasonable to require everyone to use mail submission servers that are
entrusted with their domain's DKIM private keys.
> not just ones based on DKIM. If ICANN were to eliminate it lots of things
it's a problem even without "free domain tasting".
> bad thing (er, by definition it isn't).
negative reputation of a domain is of minimal value, because spammers
will just get a new domain (or several) every time they wish to spam,
and the new domains will have neutral reputation.

> many reasons, not just spam.
nuclear war is more worrying too.  but that doesn't mean that the ease
in compromising PCs isn't a big contributor to the spam problem.  as for
"keeping...networks clean", well, of course people should try to do
that.  but as far as I can tell, so far it's more of a laudable goal
than a practical reality.

Keith

---

From: Douglas Otis <[email-masked]>
Date: Tue, 9 Oct 2007 11:51:25 -0700

On Oct 9, 2007, at 3:59 AM, Frank Ellermann wrote:

> You can combine white lists with SPF PASS as with DKIM "PASS", the  
> risk is very similar.

Due to the macro nature of SPF, full processing is required for each  
name evaluated.  With SPF, the recipient might be interested in the  
PRA, or MAIL FROM.  Of course DKIM now adds d= or i= identities.   
Since DKIM is prone to replay abuse, when an SPF hammer is the only  
tool, more names are likely to be added to the evaluation.  These  
additional evaluations are intended to reduce the number of messages  
lost since neither the PRA and MAIL FROM are limited to the path  
registered by an SPF address list.  Now imagine an iterative process  
of developing comprehensive lists of all addresses used on behalf of  
a domain which attempts to embrace IPv6. : (

>> Much of the danger of auto responses has to do with DDoS concerns.
>
> It depends on the definition of "DDoS".  From my POV as POP3 user  
> over a V.90 connection 10,000 unsolicited mails are just bad, no  
> matter what it is (spam, worm, DSN, or auto-response).

Perhaps IANA should allow ISPs to register their dynamic address  
space.  Greater protection is afforded when excluding dynamic address  
space.  Nevertheless, more protection would be afforded by a  
convention that excludes message content within a DSN.  For each  
directly addressed spam source, there are 3 DSN sources that include  
spam message content.  Cleaning up how DSNs are constructed would be  
_far_ more effective than asking that all DSNs be dropped when SPF  
does not produce a PASS.  Perhaps that is why Bell Canada ensures all  
addresses achieve a PASS. : (

> It's not really a "DDoS".  SPF at least helped me to get rid of the  
> bogus DSNs and other auto-responses since three years, smart  
> spammers are not interested to forge SPF FAIL protected addresses.

Spammers are equally uninterested in abusing MTAs that produce DSNs  
compliant to RFC3464 where message content has been removed.  This  
strategy also avoids the SPF overhead and related risks. : )

> BTW, I think the definition of "Joe job" in the sieve EREJECT draft  
> is obsolete, the mere abuse of "plausible" addresses is no "Joe  
> job" and IMO also not a real DDoS.  But it's certainly bad for the  
> victims, it can be bad enough to make a mailbox unusable for some  
> victims.

Yes, DSNs that include content are a problem.  Dropping NDN or DSN  
that indicate a failure of some sort also makes email less reliable.   
Email has become far less reliable as a result.  The TPA-SSP scheme  
for DKIM allows a return path to authorize the DKIM signature only to  
encourage the issue of DSNs.  Again, those DSNs should still exclude  
original message content.

> state of the art for some years now.  Or rather "truncate" is state  
> of the art, not complete removal of the body.

It would be rather tempting to make this mode of DSN operation a  
requirement.  At any point of time, we see some 20 million different  
MTAs which do not remove message content are currently being  
exploited.   Perhaps we should add a new list that indicates which  
MTAs do not remove content on DSNs?  We could then let our customers  
decide whether they want traffic from these MTAs as well.  Not all  
that different from open-proxies and aimed at restoring DSN integrity.

> Auto responders aren't in a good position to verify the validity of  
> the return path.  Good positions to do this are the MSA based on  
> RFC 4409 and later the MX based on RFC 4408.

RFC4408 did not mitigate a potential DDoS exploit.  This exploit  
exists when recipients attempt to process SPF as specified.  There  
are several SPF parsing libraries that lack even suggested limits on  
subsequent address resolutions for MX records, for example.  There  
are some libraries that restrict the number of MX targets and cause  
some domains to not be fully resolved at times.  Even this reduction  
in possible MX targets will not make these libraries much safer.

The SPF DoS draft example clearly illustrates an SPF process (current  
open source compliant with RFC4408) might generate more than 50 to  
100 DNS transactions per name evaluated.  The level of this risk  
depends upon the negative caching of recipients and how frequently  
the local-part of the spam campaign repeats.  The latter only needs  
to be a bit longer than the former.  The evaluation process may still  
result in an SPF PASS.  Any reliance upon SPF is likely to cause a  
list of names being evaluated to grow.  An SPF iterative process also  
means poisoning DNS is made far easier.  Keep in mind there are now  
plug-ins for web-clients and MUAs.

As it is now, those clever enough to manipulate SPF libraries are  
also able to instigate a reflected attack on par with leveraging  
large RRs against a phalanx of open recursive servers.  SPF however  
makes this attack free for those that also wish to spam.  SPF would  
also obscure the origin of the attack.  The source might not be found  
in mail logs either. : (

-Doug

---

From: Keith Moore <[email-masked]>
Date: Tue, 09 Oct 2007 16:52:35 -0400

Returned message content in DSNs is often essential information for
debugging of mail system problems.   Blindly insisting that DSNs should
not return subject message content is shortsighted.  We have already
crippled the mail system too much as the result of naive and
shortsighted spam countermeasures.

Keith
>> I'd hope that a majority of receivers already does this, that's state
>> of the art for some years now.  Or rather "truncate" is state of the
>> art, not complete removal of the body.
>

---

From: Douglas Otis <[email-masked]>
Date: Tue, 9 Oct 2007 16:45:50 -0700

On Oct 9, 2007, at 1:52 PM, Keith Moore wrote:

> debugging of mail system problems.   Blindly insisting that DSNs  
> should not return subject message content is shortsighted.  We have  
> already crippled the mail system too much as the result of naive  
> and shortsighted spam countermeasures.

The recommendation was to conform to requirements of RFC3464, but  
could have been a bit more explicit in what was meant by original  
content.  The concern is the abuse of DSNs as an indirect content  
delivery mechanism.   Including original content runs a much greater  
risk that any DSN then becomes blocked or dropped.  A more difficult  
problem to solve occurs when no DSN is found after a message delivery  
fails for some reason.  Less is more in terms of what should be  
included of original message content.

Per recipient:
  original-recipient-field
  final-recipient-field
  action-field "failed" / "delayed" / "delivered" / "relayed" /  
"expanded"
  status-field
  remote-mta-field
  diagnostic-code-field
  last-attempt-date-field
  final-log-id-field
  will-retry-until-field

Per message:
  original-envelope-id-field
  reporting-mta-field
  dsn-gateway-field
  received-from-mta-field
  arrival-date-field

Is this what you would like to see in a DSN?

-Doug

---

From: David Morris <[email-masked]>
Date: Tue, 9 Oct 2007 19:00:15 -0700 (PDT)

> Is this what you would like to see in a DSN?

As an end user, this would be about useless ... at least w/o a much
smarter client than I'm used to. Some DSNs take days to arrive. Figuring
out what message got lost w/o some portion of the actual user oriented
content is difficult at best. As an end user, I would prefer to get back
exactly what I sent, content wise. Easier to figure out what went wrong
and easier to resend. Also easier to recognized I didn't orginate the
email.

My spam filtering system does quite well eliminating backscatter while
retaining valid DSNs so my feeling is we should focus on making
backscatter detectable and not on crippling the system.

Dave Morris

---

From: Keith Moore <[email-masked]>
Date: Tue, 09 Oct 2007 22:45:05 -0400

Douglas Otis wrote:
>> already crippled the mail system too much as the result of naive and
Probably the biggest architectural problem with Internet email is that
errors are not reliably reported to either the sender of the subject
message or the party that is able to fix the problem.  DSNs didn't
change that, they just attempted to encourage a more uniform reporting
format.
> [details deleted]
I'd like to see as much information as possible that has a bearing on
why the message delivery failed.  So if the error is "recipient not
found" or some such, then it might be acceptable to include only the
header of the subject message.  But if the error has something to do
with the content (like a media conversion error), then it's useful if
that content is returned so that it becomes possible to analyze why the
error occurred.

Of course it does little good to return a DSN to someone whose mail
address was forged by a spammer.   But to solve that problem without
impairing the reliability of mail delivery reporting, an MTA needs a
reliable way of knowing whether a particular return-path is correct.  
(where "reliable" includes being reasonably immune to replay attacks)

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 09 Oct 2007 22:55:12 -0400

> smarter client than I'm used to. 
Well, of course, part of the intention behind creating a uniform DSN
format was to facilitate smarter mail user agents.  Your MUA should be
able to intercept DSNs (and MDNs)  that it receives and use that
information to report the status of each message that you've sent - or
at least, the ones for which you've received nondelivery reports.  I
could speculate about why we don't have such capability, but it would be
just speculation.

Keith

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 9 Oct 2007 23:56:52 -0400

> (where "reliable" includes being reasonably immune to replay attacks)

I'm curious how you would propose to do this?

Scott K

---

From: "Frank Ellermann" <[email-masked]>
Date: Wed, 10 Oct 2007 09:12:50 +0200

Douglas Otis wrote:
 
> Due to the macro nature of SPF, full processing is required for
> each name evaluated.

If what you call "full processing" is the procedure to fetch the
policy record of the domain in question, and match the connecting
IP against this policy for a PASS / FAIL / "DUNNO" verdict, then 
this doesn't depend on using macros or not in the policy.  

Maybe you mean that macros can reduce the chances for a DNS cache
hit, e.g. for per-user-policies based on the local part macro.

> With SPF, the recipient might be interested in the PRA, or MAIL
> FROM.

When I talk about SPF it's generally RFC 4408, i.e. MAIL FROM and
HELO, not the Sender-ID PRA.  The PRA doesn't necessarily help to
identify a permitted or forged MAIL FROM.  In other words the PRA
or FWIW DKIM are not designed to avoid backscatter.  

> when an SPF hammer is the only tool

Folks didn't like the idea to reintroduce RFC 821 return routes ;-)
Of course receivers are free to use any tool to avoid backscatter,
but SPF is today the only tool designed for this job.

> Cleaning up how DSNs are constructed would be _far_ more effective
> than asking that all DSNs be dropped when SPF does not produce a
> PASS.

No.  When I was the victim (back in 2004 before SPF started to work)
the _size_ of DSNs and other backscatter wasn't the main issue, the 
_number_ of bogus auto replies was the real problem.  

JFTR, "not PASS" can be FAIL or "DUNNO".  FAILs should be rejected.

Your idea to abuse DSNs for indirect spam or malware delivery is a
valid consideration, but in practice the volume of all backscatter
not limited to DSNs is the real problem.  

Limiting the size of DSNs is okay, but it's not "far more effective"
than "reject a.s.a.p.", in fact it misses the point wrt backscatter.

> Perhaps that is why Bell Canada ensures all addresses achieve
> a PASS. : (

Oops, bell.ca still has this ridiculous policy, that certainly won't
help them to reduce backscatter.

> exists when recipients attempt to process SPF as specified.

Actually they've to ignore a SHOULD in the security considerations
of RFC 4408 if they wish to participate in your convoluted attack
scenario.  And the SPF RFC did mitigate this potential with several
MUSTs in the security considerations in comparison with pre-MARID
drafts.

 Frank

---

From: Douglas Otis <[email-masked]>
Date: Wed, 10 Oct 2007 21:01:33 -0700

> this doesn't depend on using macros or not in the policy.

IP address evaluations to determine a PASS / FAIL / DUNNO verdict may  
require the expansion of macros inserted into SPF records.  SPF  
macros have the following syntax:

%{symbol | rev-label-seq | # of right-most labels | chars-into “.” }

rev-label-seq = “r”

chars-into “.” = “-” | “+” | “,” | “/” | “_” | “=”

# of right-most labels = 1 - 128

symbol =
  s = email-address or EHLO (initial parameter)
  l = left-hand side of email-address (initial parameter)
  o = right-hand side of email-address  (initial parameter)
  d = email-address domain or EHLO (initial parameter)
  i = SMTP client IP addr decimal octet labels (initial parameter)
  p = r-PTR domain with IP addr validated (not initial parameter)
  v = "in-addr" IPv4, or "ip6" if IPv6 (auto generated)
  h = EHLO host-name  (initial parameter)

Macro expansion and text strings can be combined with any SPF record  
mechanism and CIDR mask.  Any email-address differing in just the  
local-part must then be iteratively processed across the SPF record  
set (up to 10 follow-on SPF records or mechanisms).

For example, an MX record may be located using a label derived from  
some component of the email-address local-part.  The address for each  
target name within the MX record must then be resolved and perhaps  
masked prior to checking for a match.  The macro expansion therefore  
permits the same SPF record to cause a different set of DNS  
transactions to occur over the duration of a spam campaign.  The  
macro expansion provides the spammer a means to fully leverage  
recipient resources any number of times against any domain.  The  
domain being queried can be wholly unrelated to a domain within the  
email being evaluated.   In otherwords, a flood of queries generated  
by spam recipients can target any innocent bystander.  The SPF record  
is able to conclude with "+all" and still obtain a PASS while staging  
their completely free attack.

> hit, e.g. for per-user-policies based on the local part macro.

The per-user-policy feature of SPF makes a reflected amplification  
attack entirely free for the spammer.  Each new name evaluated from  
their cached records can generate 508 KB in DNS traffic, a higher  
amplification than that achieved in an open recursive attack.

The SPF group dismissed this concern by suggesting DNS amplifications  
could be further increased by including a series of bogus NS records.

A single message can be sent to 100 different recipients.  100 x 100  
= 10,000 DNS transactions!  Each message might be evaluated twice,  
once for the PRA, and then again for the MAIL FROM.  Who knows,  
perhaps this might include the DKIM domain if the use of SPF is not  
denounced.

>> With SPF, the recipient might be interested in the PRA, or MAIL FROM.
>

DKIM could be extended to avoid backscatter and replay abuse.   
Unfortunately, Sender-ID is being heavily promoted as the anti- 
phishing solution.  A problem not addressed by RFC4408.

> but SPF is today the only tool designed for this job.

I do not agree.  Discouraging abuse by using RFC3464 is a far safer  
tool.  SPF is dangerous and breaks email.

> No.  When I was the victim (back in 2004 before SPF started to  
> work) the _size_ of DSNs and other backscatter wasn't the main  
> issue, the  _number_ of bogus auto replies was the real problem.

SPF now gives bad actors a much more dangerous tool than auto-replies  
ever represented. : (

> Limiting the size of DSNs is okay, but it's not "far more  
> effective" than "reject a.s.a.p.", in fact it misses the point wrt  
> backscatter.

This was in terms of reducing risk.  SPF is a dangerous mechanism  
which attempts to leverage a recipient's resources.  Had the  
recipient known who sent the message, SPF would not be needed.  As a  
general rule, when the source is unknown, any scripted use of  
resources offered MUST be extremely limited.  SPF violates this rule.

>> Perhaps that is why Bell Canada ensures all addresses achieve a  
>> PASS. : (
>
> Oops, bell.ca still has this ridiculous policy, that certainly  
> won't help them to reduce backscatter.

Obviously that is not why the SPF record is published.  At least the  
"+all" discourages use of dangerous SPF libraries while also reducing  
the breakage and permitting white-listing.

> MUSTs in the security considerations in comparison with pre-MARID  

No!  Although there are many SPF parsing libraries in distribution  
that ignore the SHOULDs, the example attack scenario is not prevented  
by any RFC4408 SHOULD or MUSTs!  The attack was based upon a _fully_  
RFC4408 compliant SPF parser!

-Doug

---

From: "Frank Ellermann" <[email-masked]>
Date: Thu, 11 Oct 2007 11:23:18 +0200

> set (up to 10 follow-on SPF records or mechanisms).

Yes, an attacker can use 10 mx mechanisms in his malicious policy for
domains under his control with names containing 10 different parts of
the LHS (local part) in his envelope sender address.

> The SPF record is able to conclude with "+all" and still obtain a
> PASS

Sure, a PASS for "mail from" unknown strangers only guarantees that 
bounces and other auto-replies won't hit innocent bystanders.  A PASS
can be still spam or an attack, but it won't create backscatter, that
is more or less the idea of SPF.

> = 10,000 DNS transactions!

The number of RCPT TO for a given MAIL FROM at the border MTA where
SPF is checked is irrelevant, it's evaluated once.  Actually the same
MAIL FROM at a given border MTA has no further effect while the SPF
and MX records of the attacker are still cached.  But of course an
attacker can send more mails to more domains with different envelope
sender addresses, and he can tune the TTL of his SPF and MX records.

Whatever he does, your attack scenario depends on the mx mechanism,
and you get an amplification factor of about 10 DNS queries to the
victim per DNS query to the attacker.

The attacker could also try to abuse "call back verification" with
his bogus MX records for a better amplification factor without SPF.

AFAIK SPF is so far the only protocol specifying "processing limits"
for DNS based attacks.  RFC 2821 certainly doesn't talk about it,
and I'm not aware of any processing limits wrt SRV or NS records.

Maybe you should propose some "MX processing limits" for 2821bis.

> DKIM could be extended to avoid backscatter and replay abuse.

Hardly.  DKIM is intentionally independent of SMTP, it doesn't
look at the envelope sender address.  The envelope sender address
(aka return path, mail from, or bounce address) and the MX concept
are the essence of SMTP, anything else is syntactical sugar.

The PRA is the best approximation of the envelope sender address,
but DKIM also doesn't look at the PRA, it's not designed to fight
backscatter.  

> phishing solution.  A problem not addressed by RFC4408.

RFC 4408 doesn't discuss PRA or phishing or 2822, it's about SMTP
and backscatter.  For discussions why the best approximation PRA
isn't good enough see the <http://www.openspf.org> pages.  

Using PRA as anti-phishing solution might make sense, with several
caveats addressed in the IESG note.  Unlike PRA pure DKIM offers
no FAIL, it can help to identify various kinds of "non-phishing".
That's not the same as "anti-phishing".

> Had the recipient known who sent the message, SPF would not be
> needed.

Indeed, SPF is used to figure out when "originator as indicated
in the reverse path" in RFC 2821 is acceptable (PASS) or should
be rejected (FAIL) at a border MTA.  Other uses of SPF are at best
nice to have, and in the worst case dangerous.

> Obviously that is not why the SPF record is published.

If all they want is a PASS they could use a short "v=spf1 +all"
without the other PASSing gibberish they're publishing now.

> At least the "+all" discourages use of dangerous SPF libraries

No, "v=spf1 +all" and more verbose policies with the same effect
are perfectly valid and supported by SPF libraries, dangerous or
otherwise.  

> the example attack scenario is not prevented by any RFC4408 
> SHOULD or MUSTs

 <http://tools.ietf.org/html/rfc4408#section-10.1>
| SPF implementations SHOULD limit the total amount of data obtained
| from the DNS queries.  For example, when DNS over TCP or EDNS0 are
| available, there may need to be an explicit limit to how much data
| will be accepted to prevent excessive bandwidth usage or memory usage
| and DoS attacks.

 Frank

---

From: Douglas Otis <[email-masked]>
Date: Thu, 11 Oct 2007 12:15:52 -0700

On Oct 11, 2007, at 2:23 AM, Frank Ellermann wrote:

> Douglas Otis wrote:
>
>> Macro expansion and text strings can be combined with any SPF  
>> record mechanism and CIDR mask.  Any email-address differing in  
>> just the local-part must then be iteratively processed across the  
>> SPF record set (up to 10 follow-on SPF records or mechanisms).
>
> Yes, an attacker can use 10 mx mechanisms in his malicious policy  
> for domains under his control with names containing 10 different  
> parts of the LHS (local part) in his envelope sender address.

Each of 10 MX records may necessitate resolving 10 target addresses  
for a total of 100 targeted DNS transactions per SPF name  
evaluation.  By using the local-part in a spam campaign, one cached  
SPF record is able to reference an unlimited sequence of MX records.   
Targets within MX records can also utilize a small variable label  
together with large static labels to leverage DNS name compression.   
Without MX records being cached with dependence upon negative cache  
expiring, the network amplification of an SPF related attack using MX  
targeting would be about 10.  After a recipient's negative caching  
expires, repeating a sequence of local-parts in a spam campaign could  
thereafter use cached MX records.  Negative caching is not always  
under the control of the victim.  Once a set of MX records becomes  
cached and their target's negative response expire, the entire attack  
becomes entirely _free_ for spammers.  Spammers often send messages  
continuously.  A spam campaign can come from any source, purport to  
be from any domain, and yet attack the same innocent victim which can  
not be identified by examining the message.  : (

> bounces and other auto-replies won't hit innocent bystanders.  A  
> PASS can be still spam or an attack, but it won't create  
> backscatter, that is more or less the idea of SPF.

Compared to an SPF related attack, most auto-replies will consume a  
greater amount of an attackers resources, identify the source of the  
attack, and not achieve the same level of amplification.  In the case  
of SPF, the attack can become entirely free and completely hidden  
while spamming!  Rate limiting auto-replies would be a much safer  
solution for the recipient to perform.

>> A single message can be sent to 100 different recipients.  100 x  
>> 100 = 10,000 DNS transactions!
>
> SPF is checked is irrelevant, it's evaluated once.  Actually the  
> same MAIL FROM at a given border MTA has no further effect while  
> the SPF and MX records of the attacker are still cached.  But of  
> course an attacker can send more mails to more domains with  
> different envelope sender addresses, and he can tune the TTL of his  
> SPF and MX records.

Each recipient can be within a different domain where a MAIL FROM and  
the PRA may be both evaluated.  By expanding upon the local-part of  
the email-address, caching SPF records actually aids the attack.  The  
only tuning required would be that of the local-part sequence.   
Duration of the sequence only needs to be a bit longer than the  
negative caching of the recipient.

> victim per DNS query to the attacker.

Do you really think a spammer is unable to attack at a gain of 10,  
and then continue the attack at no cost once the duration of the  
attack exceeds the negative caching (determined by the recipients).   
SPF enables a long duration DDoS attack.  Good luck at attempting to  
prevent this type of attack or at identifying the source.  Block all  
SPF records?

> his bogus MX records for a better amplification factor without SPF.

The SPF can also attack those using wildcard MX, TXT or A records.   
This attack would be instantly free and much simpler as it would not  
require waiting for negative caching to expire.  : (

> AFAIK SPF is so far the only protocol specifying "processing  
> limits" for DNS based attacks.  RFC 2821 certainly doesn't talk  
> about it, and I'm not aware of any processing limits wrt SRV or NS  
> records.

SPF introduces a long list of macros locating a sequence of records,  
that then often involve additional DNS transactions.  Even the time  
limit applied by SPF disables DNS congestion avoidance!

> Maybe you should propose some "MX processing limits" for 2821bis.

Most systems are fairly careful about where they send messages to  
avoid being blocked, nor would the normal use of MX records require  
all targets be resolved.  Timeouts recommended by RFC2821 ensure this  
operation remains fairly safe, unlike that for SPF.  Even the packet  
limitation of DNS provides a fairly reasonable limit.

> Hardly.  DKIM is intentionally independent of SMTP, it doesn't look  
> at the envelope sender address.  The envelope sender address (aka  
> return path, mail from, or bounce address) and the MX concept are  
> the essence of SMTP, anything else is syntactical sugar.

A policy record at the MAIL FROM domain could reference an authorized  
DKIM signature within a single DNS transaction.  The same type of  
policy record could be applied against the SMTP client to curtail  
replay abuse.  The MAIL FROM policy could thereby ensure a DSN is  
issued when there is a problem, and the SMTP client could avoid  
possible grey-listing should the recipient find replay abuse to be  
problematic.

> but DKIM also doesn't look at the PRA, it's not designed to fight  

DKIM can be processed prior to acceptance, and could safely help  
prevent backscatter with a few minor by-name extensions.

> isn't good enough see the <http://www.openspf.org> pages.

Agreed, but that does not mean that two names PRA and MAIL FROM will  
not be checked.

> caveats addressed in the IESG note.  Unlike PRA pure DKIM offers no  
> FAIL, it can help to identify various kinds of "non-phishing".  

 From my point of view, anti-phishing requires examination of message  
content more than just email headers.  When the message appears to be  
misleading, and is not signed as expected, it should be marked as  
suspicious or rejected.

> Indeed, SPF is used to figure out when "originator as indicated in  
> the reverse path" in RFC 2821 is acceptable (PASS) or should be  
> rejected (FAIL) at a border MTA.  Other uses of SPF are at best  

When desired, path registration should be by-name and not by-address  
to avoid the unscalability, complexity, and risk.  Something as  
simple as CSV and a MAIL FROM policy record is far more scalable,  
easier to manage, and much safer.

> without the other PASSing gibberish they're publishing now.

I suspect they felt pressured by ISPs like AOL who use SPF records to  
white-list other large providers.  This indicates being part of the  
club.

> are perfectly valid and supported by SPF libraries, dangerous or  

It also removes any possible advantage for running these libraries  
and avoids problems where the path of the message and the path  
expressed by SPF legitimately differ.

>> the example attack scenario is not prevented by any RFC4408 SHOULD  
>> or MUSTs
>

The example attack scenario never exceeded typical DNS transaction  
sizes.  It did not use TCP or EDNS0.  It did not exceed the SHOULD  
limits on the number of mechanisms or subsequent address  
resolutions.  SPF, as current defined, remains very dangerous.  The  
use of libraries implementing SPF per this specification are  
dangerous as they permit a totally free attack, where the source is  
hidden, and there is not a means to defend against it.

-Doug

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 15 Oct 2007 14:51:38 +0200

Douglas Otis wrote:

> By using the local-part in a spam campaign, one cached SPF record
> is able to reference an unlimited sequence of MX records.

In theory IANA could publish one _spf.arpa "v=spf1 mx a aaaa -all"
record, and everybody could use it with "v=spf1 redirect=_spf.arpa".
That one SPF record can (kind of) reference an unlimited number of
MX records doesn't depend on SPF's local-part macro.  

And e-mail providers wishing to offer "per user policies" could also
create corresponding subdomains replacing [email-masked] 
addresses by say [email-masked] addresses.  Likewise 
attackers trying to cause havoc can use [email-masked]
addresses, they don't need SPF's local part macro for this purpose.

After all in your attack scenario it's the attacker who controls the
MX records pointing to bogus addresses in the zone of the victim.

> Spammers often send messages continuously.  A spam campaign can
> come from any source, purport to be from any domain, and yet attack
> the same innocent victim which can not be identified by examining
> the message.

Your attack scenario has nothing to with a spam campaign, the goal
of a spam campaign is to send unsolicited mails to receivers.  The
goal of your attack scenario is to flood a zone with bogus and huge
A or AAAA queries.  And in your scenario the mail cannot purport to
come from any domain, it has to come from domains under the control
of the attacker where he manages his bogus MX records.

> Compared to an SPF related attack, most auto-replies will consume
> a greater amount of an attackers resources, identify the source
> of the attack, and not achieve the same level of amplification.

Backscatter doesn't consume resources of the spammer (apart from
sending the UBE with a forged envelope sender address), and it can
be "mail from" any "plausible" address, not identifying the real
source.  That's precisely the problem solved by SPF FAIL and PASS.

> SPF enables a long duration DDoS attack.

Sorry, from my POV you still arrive at "MX records can be abused",
not limited to SPF (or rather only limited if done using SPF).

> Block all SPF records?

At least we won't need a new rfc-ignorant zone to implement this 
proposal... ;-)  

> limitation of DNS provides a fairly reasonable limit.

I'm not sure that "call back verification" schemes follow the "sending
strategy" (4.5.4.1) in RFC 2821, after all they don't send any DATA.

You could squeeze quite a lot of names in the reply to an MX query,
that's why SPF has an explicit limit 10.  SMTP also doesn't specify
size limits for MX queries.

> DKIM can be processed prior to acceptance

Yes, but unlike SPF not before DATA.  I skip the "anti-phishing"
discussion because it's not directly related to "backscatter".

 Frank

---

From: Douglas Otis <[email-masked]>
Date: Mon, 15 Oct 2007 11:35:03 -0700

On Oct 15, 2007, at 5:51 AM, Frank Ellermann wrote:

> record, and everybody could use it with "v=spf1  
> redirect=_spf.arpa". That one SPF record can (kind of) reference an  
> unlimited number of MX records doesn't depend on SPF's local-part  
> macro.

This adds perhaps 21 useless DNS transactions for each email  
received?  Do you expect everyone to use inbound servers to send?

> And e-mail providers wishing to offer "per user policies" could  
> also create corresponding subdomains replacing [email-masked]  
> After all in your attack scenario it's the attacker who controls  
> the MX records pointing to bogus addresses in the zone of the victim.

Use of MX records are normally predicated upon the sending of  
messages.  When sending a message, MX records may have a potential  
DNS traffic amplification of about 10.  However, overall  
amplification is much less when a message triggers an automated  
response.  An automated response is likely to be filtered, and the  
sender risks being blocked, whether or not the MX record is doing  
something nefarious.  Many situations can achieve this level of  
network amplification.

However, SPF's macros transforms a spam related attack into being  
completely free without tainting the messages.  Sending spam should  
be considered a given as it represents a large portion of today's  
traffic.  SPF permits an enormous amount of DNS traffic to be  
directed toward an innocent, uninvolved domain.  Once the attack  
duration exceeds a recipient's negative caching, no additional  
resources of the attacker are consumed!  The attack can come from  
anywhere, is not easily traced, and, beyond universal banning of SPF  
records, there is no defence!  ACLs on DNS will not help, nor will  
BCP 38.  This attack is likely to continue for a long duration.

>> come from any source, purport to be from any domain, and yet  
>> attack the same innocent victim which can not be identified by  
>> examining the message.
>

SPF adds a layer of indirection.  SPF allows the local-part of spam  
to generate DNS transactions without consuming additional resources  
of attackers once their SPF record has been cached!  The attack can  
then query any MX, A, AAAA, PTR, and TXT records from any domain  
unrelated to message content.  Even the attacking MX record might be  
unrelated to any domain found within the message.  SPF's added  
indirection provides spammers a complete disguise.  Recipients using  
SPF are to know which messages are staging an attack.  This is true  
even when they do not wish to play a role in a reported ongoing  
attack.  Expect spammers to take advantage of this generous SPF  
gift. : (

>> a greater amount of an attackers resources, identify the source of  
>> the attack, and not achieve the same level of amplification.
>

RFC3464 represents a far better solution as it removes incentives.

> not limited to SPF (or rather only limited if done using SPF).

Of course MX records can be abused.  SPF increases the potential  
level of abuse by a factor of 10 or 20.  With SPF macro's, abuse also  
becomes completely free.

> At least we won't need a new rfc-ignorant zone to implement this  

Prevention might require DNS servers be modified to exclude SPF  
records.  Do not expect compliance without some form of blocking  
imposed.

>> avoid being blocked, nor would the normal use of MX records  
>> require all targets be resolved.  Timeouts recommended by RFC2821  
>> ensure this operation remains fairly safe, unlike that for SPF.   
>> Even the packet limitation of DNS provides a fairly reasonable limit.
>
> I'm not sure that "call back verification" schemes follow the  
> "sending strategy" (4.5.4.1) in RFC 2821, after all they don't send  
> any DATA.

Call back verification is not part of RFC2821?

> discussion because it's not directly related to "backscatter".

Other solutions are able to prevent the DATA phase for spam. : )

-Doug

---

From: "Frank Ellermann" <[email-masked]>
Date: Tue, 16 Oct 2007 14:00:58 +0200

> received?

No, a "v=spf1 redirect=_sp.arpa" would add precisely one transaction
in comparison with directly stating "v=spf1 mx a aaaa -all".  Don't
worry about it, it was only a theoretical example.  

The SPF policy for xyzzy.claranet.de uses this technique, admittedly
unnecessary in this case.  Lame apology, the massive abuse of xyzzy-
addresses in 2004 forced me to check out SPF before I understood the
fine print (here: avoid unnecessary DNS queries).  It's not really
"my" policy, otherwise I'd get rid of the (in this case) unnecessary
DNS transaction for the "redirect=" step.

> Do you expect everyone to use inbound servers to send?

No.  Of course I'd expect that mail to <postmaster> to the IP of an
MTA talking to an MX I care about works.  BTW, it would be nice if
only the MXs of the envelope sender address talk to other MXs, we
could then scrap SPF in this "inherent RMX" parallel universe.  Just
decree that everybody has an implicit "v=spf1 mx a aaaa -all" policy,
what a dream.  Actually SPF is a clumsy approximation of this dream.

> messages.

Yes, I proposed a simplification of your DNS attack scenario based
on "call back verification" instead of SPF.  AFAIK "CBV" works by
connecting to an MX of the alleged sender, and trying a RCPT TO the
envelope sender address.  If that results in "unknown mailbox" the
receiver knows that the envelope sender address is dubious at best.

Spammers also know this, that's why they forge "plausible" addresses.
An SPF FAIL cuts them off at this stage.  As long as there are more
than enough unprotected "plausible" (= surviving CBV) addresses the
spammers can carry on as is.

> However, overall amplification is much less when a message triggers
> an automated response.

When I got about 1,000 bogus bounces and other auto-replies per day
in 2004 I didn't care about other side effects, my mailbox was almost
unusable.

> An automated response is likely to be filtered

Filtering behind a POP3 modem connection is tricky, after all I still
wanted to get "good" bounces, challenges, and other auto-replies.

Filtering or rejecting auto-responses before final delivery needs
something in the direction of BATV plus good heuristics to identify
"non-standard" backscatter (as outlined in RFC 3834).

BATV doesn't directly fight forgeries, it only stops backscatter 
before final delivery (ideally rejecting it at the MX of the alleged
originator).  It's a nice scheme, use it if you can.  But it won't
help me (as receiver) to reject forged "mail from" Douglas Otis.  

> SPF permits an enormous amount of DNS traffic to be directed toward
> an innocent, uninvolved domain.

Well, I'd try something more direct when I'd wish to attack a domain,
your idea is IMO far too convoluted.  And your claim that a domain
under attack by bogus A or AAAA queries caused by MX records of the
attacker, based on CBV, SPF, or what else, has no chance to find out
who the attcker is, might be also wrong.  The querying hosts can find
out why they do this, and at that point one or more name servers of
the attacker (where he manages the bogus MX records) are also known.

> Once the attack duration exceeds a recipient's negative caching, no
> additional resources of the attacker are consumed

BTW, who determines the TTL of NXDOMAIN, can a zone under attack tune
this ?
  
> Even the attacking MX record might be unrelated to any domain found
> within the message.

They're referenced by mx-mechanisms in the SPF policy of the envelope
sender, i.e. the attacker.  And this SPF policy record still exists
at least in the DNS cache of the receivers.  FWIW, the querying hosts
can find out what's going on.

> Expect spammers to take advantage of this generous SPF gift.

Don't confuse "spammers" and "attackers".  This might be the same
persons, but the roles are very different.  Spammers in their role
as spammer might wish to attack successful anti-spammers, but they
have no reason to prefer your attack strategy based on SPF where
they'd be forced to risk one of their own name servers.  I think
all attackers prefer to risk more anonymous resources like zombies.

> RFC3464 represents a far better solution as it removes incentives.

No, that's not true.  When my addresses were abused it was the
number of bogus DSNs and other auto-replies that bothered me most,
not the size of the DSNs.

And the incentive isn't to send spam indirectly in the form of a
DSN to the forged envelope sender, the incentive is to abuse an
unprotected "plausible" address.  SPF FAIL offers the protection.

> Call back verification is not part of RFC2821?

True, but I was talking about abusing CBV for an MX attack instead
of SPF's mx-mechanism, and you then hinted that normal RFC 2821
sending strategies won't query for addresses of all names found in
the MX records at once.  Actually I don't know what real MTAs do
if they get NXDOMAIN, do they wait before they try the next name ?

 
> Other solutions are able to prevent the DATA phase for spam. : )

Whatever it might be, it didn't work for me back in 2004.  Victims
of backscatter can't wait for worldwide adoption of MTAMARK, CSV,
or "other solutions", they need something working now.  SPF works
as designed because it's in the best interest of spammers to stay
away from FAIL-protected addresses.

 Frank

---

From: Douglas Otis <[email-masked]>
Date: Tue, 16 Oct 2007 19:22:15 -0700

On Oct 16, 2007, at 5:00 AM, Frank Ellermann wrote:

> could then scrap SPF in this "inherent RMX" parallel universe.   
> Just decree that everybody has an implicit "v=spf1 mx a aaaa -all"  
> policy, what a dream.  Actually SPF is a clumsy approximation of  
> this dream.

Requiring MX for discovery eliminates additional transactions needed  
to assert no policy record or valid email domain.  However, the SPF  
or RMX concept is seriously flawed.

Defining valid sources might scale when done "by-name" but never "by- 
address".  By-Name is how MX records declare destinations.  PTR  
record-sets adjacent to an MX record could associate valid client  
domains.  Perhaps special names like "_NONE" or "_ANY" could be  
defined for PTRs under a _MX-OUT leaf.  DNS would then be able to  
return addresses needed to validate a specific client host-name  
within a single transaction, but not for all possible host-names.   
Being able to validate clients within an authorized source domain  
should safely satisfy your need for a protection mechanism.

Fully resolving just one MX record for either IPv4 and IPv6 addresses  
might necessitate 20 DNS transactions when permitting 10 MX targets.   
SPF allows 10 MX records, where each may contain 10 targets.  Unless  
IPv6 is excluded, subsequent transactions for both A and AAAA records  
might be needed where a high level of NXDOMAINs would be ignored.   
This means 10 x (10 + 10) or 200 DNS transactions satisfy SPF  
semantics for each email-address checked.  However, some suggest  
using SPF to authorize DKIM domains or PRAs first.  When DKIM or PRAs  
are not authorized, it is common to then check MAIL FROM.  Each  
message might therefore invoke 400 DNS transactions per message.   
Even after allowing this number of transactions, the resulting list  
may not encompass the addresses of all valid outbound hosts.

> Spammers also know this, that's why they forge "plausible"  
> addresses. An SPF FAIL cuts them off at this stage.  As long as  
> there are more than enough unprotected "plausible" (= surviving  
> CBV) addresses the spammers can carry on as is.

See above.

>> However, overall amplification is much less when a message  
>> triggers an automated response.
>
> in 2004 I didn't care about other side effects, my mailbox was  
> almost unusable.

See above.

> Filtering behind a POP3 modem connection is tricky, after all I  
> still wanted to get "good" bounces, challenges, and other auto- 
> replies.
>
> before final delivery (ideally rejecting it at the MX of the  
> alleged originator).  It's a nice scheme, use it if you can.  But  
> it won't help me (as receiver) to reject forged "mail from" Douglas  
> Otis.

See above.

>> SPF permits an enormous amount of DNS traffic to be directed  
>> toward an innocent, uninvolved domain.
>
> Well, I'd try something more direct when I'd wish to attack a  
> domain, your idea is IMO far too convoluted.

SPF enables a cached record to redirect the subsequent DNS  
transactions of a recipient.  In addition, SPF provides an attacker  
the means to manage a sequence of MX records independently from what  
is seen as the originator.

> And your claim that a domain under attack by bogus A or AAAA  
> queries caused by MX records of the attacker, based on CBV, SPF, or  
> what else, has no chance to find out who the attcker is, might be  
> also wrong.

Logs of the DNS and mail servers may provide clues about which  
domains appear to have staged the attack, but the domain visible to a  
mail server is able to transition rapidly which prevents any filter  
from curtailing an ongoing attack.

> The querying hosts can find out why they do this, and at that point  
> one or more name servers of the attacker (where he manages the  
> bogus MX records) are also known.

Finding one of perhaps hundreds of DNS servers used by an attacker  
will not offer a meaningful defence.  The ability to redirect DNS  
transactions afforded a cached SPF record ensures an attacker can  
escape any blocking effort.

>> Once the attack duration exceeds a recipient's negative caching,  
>> no additional resources of the attacker are consumed
>
> BTW, who determines the TTL of NXDOMAIN, can a zone under attack  
> tune this?

The recipient decides.  They may follow RFC2308 recommendations which  
is the lesser of either the SOA MINIMUM field or the SOA TTL.   
However some experience problems and truncate the duration.  Here is  
a common tip given to Windows users:

"By default these negative entries are cached for 5 mins.  But we can  
tweak the registry to NOT store negative entries at all!"

>> Even the attacking MX record might be unrelated to any domain  
>> found within the message.
>
> They're referenced by mx-mechanisms in the SPF policy of the  
> envelope sender, i.e. the attacker.  And this SPF policy record  
> still exists at least in the DNS cache of the receivers.  FWIW, the  
> querying hosts can find out what's going on.

How long will these records remain?  A single cached SPF record can  
do untold damage with its ability to redirect recipient  
transactions.  Rapidly changing these records will not consume much  
of an attackers resources.  Attackers are extremely adept at fast- 
fluxing DNS.  It is not 2004 any more.

> all attackers prefer to risk more anonymous resources like zombies.

Why do you think there is a difference between a zombie and a name  
server?  Why shouldn't a spammer offer an attacking service when it  
will not impact their spamming service?

> not the size of the DSNs.

RFC3464 removes incentives which affords greater protection at much  
smaller costs.  SPF is not the only tool available!

> unprotected "plausible" address.  SPF FAIL offers the protection.

Despite intentions, SPF creates a far graver danger.

> the MX records at once.  Actually I don't know what real MTAs do if  
> they get NXDOMAIN, do they wait before they try the next name ?

Regardless, it should not happen more than the number of targets  
contained within a single MX record.  This attack would also incur  
the message and MX record overhead.  While a concern, it is not in  
the same league.

> away from FAIL-protected addresses.

The same could have been said about the use of DDT as a pesticide.   
This attitude will likely require banning SPF before its use can be  
irradiated. : (

-Doug