Subject: [Asrg] SPF and other spam approaches

From: [email-masked]
Date: Tue, 17 Jun 2003 16:45:18 -0500

I'm going to combine responses to several related posts from the last digest.

> ...SPF avoids the quicksand of
definition entirely.  SPF is merely an *extension* of SMTP which
*closes a security hole*; 

Well, I think that "closes" is rather misleading.  It shrinks it, but doesn't 
really close it in any positive, meaningful way.  If it really were the single, 
unobstructive 'silver bullet' that everyone seems to want, its numerous problems 
might be more tolerable.

> it improves the veracity of sender addresses among consenting hosts.  

"Improves", yes, but not by ENOUGH that globally one can act in a positive way 
on the conclusion.  Yes, it can indicate that a given message, IF it comes from 
the indicated sender, was sent from an 'atypical' originating domain for that 
sender.  The problem is that that could happen QUITE LEGITIMATELY.  Also, the 
fact that a forging spammer could with TOTAL impunity still impersonate the 
victimized From: address, as long as they're able to send from within the same 
range of IP addresses (which is fairly easy if it's a big ISP domain, like 
"swbell.net", which might have literally millions of subscribers and IP 
addresses).  Similarly, spam from legitimate (but throwaway) E-mail accounts is 
likewise unaffected... yeah, the sender is exactly who they say they are, 
"[email-masked]".  And they really do reside at 123 Nosuch Street, I'm 
sure...

> It also fights worms and viruses; 

No, it really doesn't.  As long as they are sent from "legitimate" senders and 
through their configured SMTP servers, they'll go out just fine.

> ...it also fights forgery.  

It DOES detect (but not eliminate) many (but not all) kinds of forgeries.  
Unfortunately, that comes at the cost of hugely inconveniencing (or making it 
impossible) for many legitimate, honest E-mail users to continue using their 
Internet services the way they always have been able to do.  Because of that 
simple fact, it's not clear that 'detecting' a possible/likely From: address 
forgery (according to spf) allows an ISP to really do much about that.

> Right now lots of our customers misconfigure their
home machines as pobox.com, and send bounce messages using From:
[email-masked].  The double-bounces end up in my mailbox.  SPF
helps solve this problem also.  If SPF works against spam, what a
happy coincidence!

>| ...some objections to SPF have come from people who
| 
| 3) conclude that because technical solutions have not solved spam in
|    the past, they cannot possibly solve spam in the future; this is
|    the "spam is ultimately a social / economic / legal problem" camp;
| 
| 4) observe that no single solution is perfect, and fall into despair;

> (3) is an objection unique to pessimists.  We technical types, being
professional practitioners of pessimism, must be especially wary.  We
have nothing to lose by trying a new approach.  

If that were true, then testing the approach would make more sense.  
Unfortunately, there are a number of problems I foresee that SPF does not 
address or resolve.  Meanwhile, it ignores the legitimate if inconvenient needs 
and requirements of a fair number of honest/legitimate users.  The (potentially 
large) costs to those ignored users don't seem to concern the spf folks very 
much.

> Optimism in this case is more productive than pessimism.  

Optimism is fine when there are no visible, obvious problems with a given 
approach.  When there ARE problems that are plainly obvious in advance, then 
forging ahead regardless smacks less of "optimism" and more of "denial".

> (4) is a problem for those who are both pessimists and perfectionists.
An 80% solution that works is better than a 100% solution that doesn't
exist.  

I certainly agree.  The "permissions" system I propose doesn't purport to 
eliminate spam;  what it DOES do is to block a LARGE percentage of it, to make 
the stuff that DOES get through smaller and less costly, and easier for content 
filters like SpamAssassin to deal with.  What's more, my approach is 
incrementally and independently implementable at the individual ISP level (or 
even, with less benefit, at the individual end user level!) and doesn't require 
reaching any kind of global technology consensus or re-engineering of basic 
Internet systems architecture in order to achieve HUGE and immediate, direct 
benefits.

> Spam costs ISPs money; they are looking for answers today.

Absolutely.  Another reason why spf, IMHO, is NOT the answer.

>  Forwarding systems and kiosks can
easily achieve SPF compliance by using a sender rewriting scheme
(SRS); I am developing a simple I-D to describe one possible SRS.

Yes, but the mere fact that you're expecting EVERY sender WORLDWIDE to change 
their systems to comply with your suggested standard, to me, is pretty 
outrageous.  It's just NOT likely to happen.

> ...The pain of a world where SMTP permits forgery,
anonymity, and spam is greater to me than the pain of a world where
SASL+SMTP+SPF require authenticated logins to home servers.  

Yes, but the notable part of your contention is "to ME" [emphasis mine].  
Unfortunately, you don't seem to take the legitimate needs of users into much 
account where other people's answer to that question might be different.

> I expect leading ISPs to act for the majority of their users, 

Depends entirely on what the agreements are that they have with their users.  
For them to make a voluntary change which blows some (even small) percentage of 
those (legitimate, honest, reasonable) users out of the water could represent 
"breach of contract" and subject the ISP to legal liability.

> ...and I expect the majority of the users to agree with me.  

Individual rights, to be meaningful, doesn't just mean the right to take 
popular, majority positions.  The right to free speech doesn't mean much if it 
only means you have the right to say stuff that nobody else finds objectionable.

Now, I am *not* saying that spammers have the unassailable right to shovel their 
fraudulent, misrepresented, and sometimes lurid garbage indiscriminately into 
the E-mail inboxes of children, or even other adults.  But one CAN'T just ignore 
the needs and legitimate rights of minorities.

> SPF tries to cushion the blow to those with fragile eggs by providing the 
mechanism of Sender Rewriting and by advocating SASL.

For spf to be "mostly useful", it must be adhered to by a large percentage of 
ISPs worldwide.  Frankly, I don't honestly expect to see that happen.

> Damian Conway: "if nothing you've tried so far has worked, try
something new."  SPF counts as new.  

So do any of a number of other ideas, some of which seem to have a much lower 
cost and a much higher probability of immediate payback... and which don't have 
a visible brick wall a little further down the path.

> I have seen several people criticize SPF as unworkable.  

I don't accuse SPF as being "unworkable".  I do however feel that it is 
logistically improbable, doesn't really solve the problems it's purported to 
solve, and may create more problems (and problems even more intractable, 
perhaps) than those it DOES solve.

> I invite them to propose a specific alternative.  

I have done exactly that.  My problem attacks the spam/virus/worm problems from 
a different perspective, but is more readily implementable and comes at 
essentially ZERO cost or inconvenience for the GREAT majority of legitimate 
E-mail users... (those who send plain ASCII text E-mails, without 
HTML-burdening, attachments, or encoded text).  It allows receipt of unexpected 
E-mails from previously unknown correspondents, but forces those to be in a more 
compact and efficient form that is readily processed by better-conceived content 
filters.

Note that my proposed solution would block the majority of today's spam, shrink 
the average size of tomorrow's spam, improve the effectiveness and efficiency of 
content filters, and would almost totally eliminate virus and worm and trojan 
threats for the great majority of users.  And it would do that with almost no 
impact on most E-mail users... and without requiring ANY global technological 
consensus.

> If they will not, then these people won't lead, and they
won't follow; they need to get out of the way.

Hopefully you won't try to put me into that class.  :-)

> I've already thought of one response; the "throwaway domain" scenario
is addressed in the FAQ at

>  http://spf.pobox.com/faq.html#noprevent

Addresses those points, yes.... unfortunately though there are a LOT of holes 
and glib presumptions in those "answers" that just don't hold much water in the 
end analysis.

>| ...some objections to SPF have come from people who
| 
| 9) prefer a different approach;
| 

> This is a perfectly respectable objection.  In this war on spam we can
use as many weapons as we can get.  

SPF actually does relatively little to prevent or dissuade spam.  Unfortunately, 
it is FAR more effective at preventing (even legitimately) anonymized E-mail... 
such as E-mails dealing with politically, professionally, or personally 
sensitive subjects (e.g. "whistle blowing" or commentary from within a 
repressive regime).

> We don't have to pick just one solution; they can work together.  

On that point we very much agree.  I see my sender/recipient permissions list as 
working hand-in-hand with content filters such as SpamAsssassin to produce a 
HUGE reduction in spam/virus/worm/trojan message volume, bulk, bandwidth, and 
fraud/mischief potential.

> It's time to review the proposals and unite behind the best ones.  

That suggests an either/or approach, and the desire at some point to stop 
listening to new approaches and ideas.  I don't think that's a good plan.

> If you evaluate the proposals along the dimensions of scalability, 
backward-compatibility, transparency, level of inconvenience to users, universal 
applicability, implementation timeframe, and cost of new code and configuration, 
SPF comes out near the head of the pack.

I disagree.

I find that SPF requires a global concurrence that we're unlikely to achieve.

SPF expects that eventually nearly all MTAs will become spf-compliant;  those 
that do not comply end up being a sore, sticking point essentially forever.

SPF blows some users depending on personal domains straight out of the water.

SPF creates major problems for public-access E-mail terminals (kiosks, internet 
cafes, cruise ships, airport waiting lounges, etc).

While SPF offers some evidence of possible/likely forged domain, basically NONE 
of the results it gives are 100% sure.  The existence of ONE noncompliant MTA 
(or open mail relay) within a large domain (say swbell.net) means that anyone 
using that domain for outgoing E-mail can impersonate any other sender with an 
address within that domain, and SPF will indicate that (as far as it's 
concerned) the From: address is perhaps[/likely] not forged.

Even if ISPs concur on SPF essentials, it would take YEARS to achieve widespread 
compliance (and even then, the spam problem still isn't really solved).

> One problem, as I see it, are the ever increasing number of zomboid
thrall servers; PC's with viruses which turn them into spam delivery
slaves.

To begin with, there are all manner of potential "denial of service attacks" and 
not all of them involve E-mail or spam.  That said, blocking E-mails containing 
scripting, ActiveX, HTML, and attachments from anyone not AUTHORIZED 
SPECIFICALLY to send such stuff to a given recipient will go a LONG way towards 
reducing the ability so sneak such stuff into an unwary user's machine.

> "Spammers" is not a single entity.  It's possible that even single
spammers will try many different approaches, but certainly as
group, they will try almost everything they can think of.
Thus, they won't switch to empty envelopes, they'll do 
empty envelopes /as well/.  Or rather, some will do empty envelopes,
while some will continue to forge envelopes, and some will buy 
dozens of new domains to spam "from".

If those new domains are unable to get HTML-burdened E-mail, or attachments, or 
image/hypertext links, or scripting, or encoded text, through to users... then 
their ability to deceive and defraud while cloaking their actions are SEVERELY 
damaged.  Cleaning up the residual using content filters ought to mop up a 
sizeable percentage of whatever is left.

> I've seen seemingly endless talk recently about the characteristics
of people who don't like SPF and various bits of what I assume are
deep philosophies, but nothing about what SPF is.

Essentially, SPF is a way using DNS to allow a MTA to query for a given E-mail 
(sender) address to find out which IP addresses that E-mail address is 
"authorized" to send from.  The idea is to detect obvious (the most egregious, 
anyhow) cases of From: address forgeries.  What one can actually DO with that 
information, once determined, is a lot less obvious than might originally 
appear.

> ...SPF may be based on
the false notion that many (or any) major ISPs object or could be
forced to object to people sending mail ossensibly from their domain
names but via unrelated SMTP clients.

And that, indeed, is one of the big problems (IMHO) with SPF.

>> Spam with an empty envelope isn't ever going to be effective, 
> since most will filter it out.  

My "permissions" approach would (by default) t-can any incoming E-mail with a 
blank sender, IF it contained HTML, scripting, attachments, or encoding... 
although the individual recipient could easily set a "blank sender" to allow 
those IF they wanted to allow it.

> How about all the "robots" or CR systems that send their mails with
empty envelope senders, because they don't want an answer if the
challenge was faked. How about all the autoresponders?
There are lots of valid emails with empty envelopes.

Sure, but vanishingly small numbers of those advisories need to send HTML or 
attachments.

> Sorry, I don't think it's reliable if bounces get lost. Just like
discarding spam qualified emails automatically is .. hmmm .. unwise,
because if it's a false classification the sender has no chance to get
informed what happend. At times where email is considered a trustworthy
medium this is unacceptable.

Note that the rules for discarding HTML, attachments, and encoding as proposed 
in my permission-list system are at least intended to be unequivocal and don't 
require statistical interpretation.  Unknown mail senders sending attachments or 
HTML or encoded text bodies are refused/bounced/trashed, period.

I agree fully that content filters like SpamAssassin can occasionally generate 
false positives, so those SHOULD use a different, and more auditable, mechanism 
(possibly redirecting such suspect messages to [email-masked] or 
some such.)

> So you want to force about 90% of all mailserver administrators (most of
the clueless) to change the configuration of their mailserver? Have fun.

Exactly... A lot of these folks are not going to make these changes, or will 
make them wrong.  I like approaches much better that don't require global 
change, or global consensus.

Gordon Peterson                  http://personal.terabites.com/
1977-2002  Twenty-fifth anniversary year of Local Area Networking!
Support the Anti-SPAM Amendment!  Join at http://www.cauce.org/
12/19/98: Partisan Republicans scornfully ignore the voters they "represent".
12/09/00: the date the Republican Party took down democracy in America.

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg