Subject: Re: [Asrg] Re: Asrg Digest, DNSBL BCP v.2.0

From: <[email-masked]>
Date: Sun, 04 Mar 2007 12:46:51 -0600

>> I think you can do FAR BETTER from a content standpoint
> (content analysis, such as Spam Assassin, following "a
> priori" blocking of mail from unknown/untrusted senders
> containing HTML or attachments) than you can using any
> kind of IP-based blacklisting or other "reputation"
> scheme.

> SpamAssassin (and other content filters) don't actually work the way you think they do, on many levels.

> The major measurable component of spam is whether or not the sender
has permission to contact the recipient.

I disagree.  I have no objection at all to being contacted 
by someone I've never met before.  I hand out business 
cards at trade shows and elsewhere.  I have my E-mail 
address on my (well-indexed) personal Web site.  Many 
companies put their E-mail address on Yellow Pages ads and 
other public places.  The fact that I've not previously 
authorized contact isn't the problem.  The problem is the 
delivery of unwanted, highly repetitive and annoying, 
scams and garbage.

> Content filters in particular
have no view into consent; no way to measure consent.

In the absence of a fine-grained whitelist (on a 
per-sender basis), I agree that they don't have any way to 
measure consent.  But adding that component changes that 
situation quite dramatically.  Not only does the system 
then know WHO has established "consent", but also WHAT has 
been "consented" to.

The trick then is deciding which of the NEW, first-time 
contacts is likely to be unwanted.  Certainly, there are 
various clues... including the presence of content 
commonly used to evade filtering (decryption scripting, 
obscured URLs, URL redirection, etc etc).

> There is no HTML
code or X-Header that reliably provides proof of opt-in.

If there were, and if it could be relatively easily 
spoofed, it would be less than terribly useful.

I think the solution involves (among other components) a 
tacit understanding at both ends of the communication 
between what the sender is sending, and what the recipient 
expects them to send.  The bar should be higher for 
previously unknown, therefore unestablished or untrusted, 
senders.

> They do some
good things based on modeling of what looks like spam; but 
it's also
true that things that look like spam are not always spam.

That's true.  And that's where the recognition of a 
familiar (to the recipient) sender enters into things.

> False positive issues you rant about occur just as often with content filters. Some would claim, even more so!

I am likely to be far less upset about getting 
questionable mail if there is at least SOME arguable 
reason why the filter ought to have delivered it.  Users 
ought to be able to tweak their filters so that they can 
change the rules whenever they desire, especially for 
particular cases that occur with some frequency for them.

> With a blacklisting, I get a bounce back and can find somebody to argue with. With the common method of implementing a content filter,
my mail is quietly eaten and I get no information back 
regarding the
failure to deliver the mail to end recipient. This is 
worse than IP
blacklisting; less transparent; less obvious; less 
opportunity for
feedback and investigative recourse.

The big problem with blacklisting bouncebacks is that in 
the general case, you cannot be sure WHO to send the 
bounceback TO.  Once spam has gone through one or more 
levels of forwarding, the only way to go further back is 
via the Received: headers, but those are commonly 
counterfeit.  Sending bouncebacks multiplies the wasted 
bandwidth due to spam.

Worse, "intentional bounceback" can be used by spammer as 
one way to get their spam delivered to a third party... 
they send mail in a way that they are confident will be 
bounced back, but arrange things so that the bounceback 
will go to the actually intended recipient... but this 
time, the (bounceback) message is originating from a 
not-blacklisted MTA.

Ultimately, I believe that the best way to deal with such 
spam is to at least OFFER recipients a chance to review 
blocked messages (and hopefully via rules that they can 
use to eliminate the necessity of their reviewing 
repetitively familiar spam), or the choice of accepting 
the system's determination and just junking it.

But again, I'm far more willing to accept mail from 
someone if (1) I recognize the name of the sender, and (2) 
the mail "looks like" the sort of mail I would expect to 
receive from that sender.

> The fact that you think they're better is likely based on an incomplete view on your part. 

I doubt it, but I'm certainly willing to learn.

> You actually probably have no idea how much of your mail has ever been redirected to a bulk or trash folder
by a content filter.

Actually, I tend to monitor that rather closely, in part 
because I use that knowledge to refine my ruleset.

> And of course, not to mention that SpamAssassin, which you hold up as
the better model,

I consider it a 'respectable' example of the genre.  I 
generally make it a point to include "like" or "-type" in 
references to that product.

> has lovingly crafted hooks into it to allow direct
support of IP-based blacklist and other IP-based 
reputation
mechanisms.

Hopefully they use that as an INPUT into the rating 
process;  I don't have a problem with that, as long as 
mail coming from such "blacklisted" IP addresses is not 
BLINDLY trashed regardless of any other considerations.

> Note to rest of world: I'm not anti-SpamAssassin. I've run it myself
before and likely will again. I'm just pointing out that 
like just
about every other kind of spam filtering or blocking 
mechanism, a
content filter is imperfect.

Certainly they have limitations, including some which are 
so severe as to be essentially crippling.  HTML, embedded 
images, attachments, and the like make it nearly 
impossible for content-based spam filters to do a good and 
effective job.  Even if, (IF!) for example, a content 
filter had OCR abilities to try to analyze 
text-as-image... an embedded image could change the 
referenced image (say) an hour after sending the E-mail, 
such that it was actually read AFTER the analysis had 
passed the (previously linked) image.

> It's a bit mind-blowing to see content
filtering held up as this panacea to address the ills of 
IP-based
blocking, since they're both approximate models of what 
somebody
thinks is spam,

The only opinion that MATTERS is that of the recipient... 
which is why they should be able to control the ruleset 
and the sender-by-sender whitelist, as well as what to do 
with spam (e.g. putting it into a spam folder that they 
can examine as they wish to confirm the accuracy of the 
filtering).

> ...and have flaws inherent to both technology and policy
limitations.

Again, what the USERS want is the ability to have the mail 
that makes it into their inboxes bear some approximation 
to the mail they expect and want to see.  And only those 
users are able to make that judgement call, in the end 
analysis.  What we need is an effective, practical tool 
(or toolset) to allow them to express that set of 
criteria.

It's clearly not enough to look just at the e-mail 
headers; but within that limitation (for example) I was 
getting relatively useful filtering using the web-based 
ruleset offered by my domain provider, until I ran into 
their limit of 200 rules...!

> Regards,
Al Iverson

Gordon Peterson
http://personal.terabites.com
1977-2007  Thirty year anniversary of local area 
networking

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: "Al Iverson" <[email-masked]>
Date: Sun, 4 Mar 2007 20:43:19 -0600

On 3/4/07, [email-masked] <[email-masked]> wrote:
> by someone I've never met before.

It seems like you're perfectly able to run with that by not choosing
to use any sort of blacklists or filtering.

> obscured URLs, URL redirection, etc etc).

It's hard to take somebody seriously when they lean on things other
than consent as the primary measure as to whether or not something is
wanted or unwanted.

Good luck with that, as your disagreement with ASRG members, blacklist
operators, anti-spam activists, and most other folks dealing with
email sending or receiving runs quite a bit deeper than whether or not
IP-based reputation mechanisms are appropriate.

Let us know when you update your website to let people know that it's
okay to spam you as long as they don't obscure URLs, utilize
redirects, or encrypt content with scripting.

Regards,
Al Iverson
-- 
Al Iverson on Spam and Deliverabilty, see http://www.aliverson.com
Message copyright 2007 by Al Iverson. For posts to SPAM-L, permission
is granted only to this lists's owners to redistribute to their sub-
scribers and to archive this message on site(s) under their control.

---

From: <[email-masked]>
Date: Mon, 05 Mar 2007 00:27:34 -0600

It's hard to carry on a meaningful discussion with someone 
who deletes the relevant portions of my message, 
selectively quotes the less specific parts, and then 
complains about what I said (and didn't say).

On Sun, 4 Mar 2007 20:43:19 -0600
  "Al Iverson" <[email-masked]> 
wrote:
>> > SpamAssassin (and other content filters) don't 
>>actually work the way you think they do, on many levels.
>>
  
[Here, Al deleted the following continuing point, where I 
commented what I *do* object to.  I don't have a problem 
getting meaningful, relevant, first-contact mail intended 
specifically for me.  I don't want scams, worms, and 
repetitive come-ons and garbage shoveled out at random by 
the dumpsterful.]

> to use any sort of blacklists or filtering.

I never said I didn't want filtering.  In fact, I have 
stressed CONSTANTLY that I *do* want filtering.  What I 
want, again, is:

    1)  A fine-grained whitelist which allows me to 
specify what sorts of mail individual senders are expected 
and allowed to send to me;

    2)  A default rule prohibiting outright more dangerous 
content and/or tricks commonly used to evade content 
filtering;  subject to my ability to grant less 
restrictions on content from known, trusted senders;

    3)  A good content filter (which hopefully I can 
adjust to my wishes) for non-whitelisted mail which gets 
through the previous two rules.

> than consent as the primary measure as to whether or not 

And it's very hard to carry on a meaningful discussion 
with someone who thinks that nobody needs to get 
(legitimate) "first contact" e-mails from people, and 
can't understand that spam is NOT the same thing as 
"legitimate first-contact E-mails".

> IP-based reputation mechanisms are appropriate.

I think it's very obvious that the existing methods of 
controlling spam are not adequate.  Had existing methods 
actually SOLVED THE PROBLEM, none of us would need to be 
here carrying on this discussion.  Likewise, the control 
of viruses, worms, and other malware by an e-mail 
distribution vector is another area where major work needs 
to be done.

> redirects, or encrypt content with scripting.

Please don't put words in my mouth, especially when you 
know full well that they misrepresent what I have said.

Prohibiting outright such 'prima facie' evidence of trying 
to evade filters is primarily intended to ALLOW subsequent 
anti-spam content filtering to be much more effective and 
practical than it would be otherwise... and to help block 
wanted (presumptuous) first-contact E-mails which don't 
look the way first-contact E-mails reasonably ought to 
look.

Perhaps you actually DO understand my points, and are just 
posting junk like you're posting with the intent to harass 
or intimidate.  In case you haven't figured it out yet, I 
don't get initimidated.  :-)

Gordon Peterson

---

From: der Mouse <[email-masked]>
Date: Mon, 5 Mar 2007 01:40:25 -0500 (EST)

> I never said I didn't want filtering.  In fact, I have stressed
> CONSTANTLY that I *do* want filtering.  What I want, again, is:
> [...]

Fine.  Go to.  You're welcome to it.

What I think we (TINW) object to is your attempt to deny to *others*
filtering methods *they* want to use, such as IP-based DNSBLs.

Or at least I have trouble putting any other construction on what you
wrote, though I don't recall your ever actually coming right out and
saying that you want to deny receivers the use of such tools.

/~\ The ASCII				der Mouse
\ / Ribbon Campaign
 X  Against HTML	       [email-masked]
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

---

From: Seth Breidbart <[email-masked]>
Date: Mon, 5 Mar 2007 02:21:10 -0500 (EST)

> filtering methods *they* want to use, such as IP-based DNSBLs.

I think he's trying to persuade rather than prevent.

The problem with his FUSSP is that it only protect me (from spam
generated by zombies that were infected by email) if *everybody else*
on the Internet uses it.  Given the current state of the world, I need
to protect myself.  (And note that lots of bot infestations don't
spread via email, but by direct connections, bad web pages, etc.)

Seth

---

From: "william(at)elan.net" <[email-masked]>
Date: Sun, 4 Mar 2007 23:33:21 -0800 (PST)

On Mon, 5 Mar 2007 [email-masked] wrote:

> And it's very hard to carry on a meaningful discussion with someone who 
> thinks that nobody needs to get (legitimate) "first contact" e-mails from 
> people, and can't understand that spam is NOT the same thing as "legitimate 
> first-contact E-mails".

I think its the spammers who don't understand what is "legitimate 
first-contact E-mails" or rather they do and they are taking
advantage of it quite well. But let me give you a few hypothetical
examples and you can tell me which of these are and are not spam:

Email example 1:

"Hi, Let me introduce myself - I'm William. I've read your comment
on ASRG mail list and I'm doing research on anti-spam technologies
and would like to talk to you about spam-related issues you have 
experienced"

Email example 2:

"Hi, Let me introduce myself - I'm William. I've read your comment
on ASRG mail list and would like to talk to you about spam-related
issues you have experienced"

Email example 3:

"Hi, Let me introduce myself - I'm William. I've read your comment
on ASRG mail list and would like to talk to you about your business
anti-spam protection and email deliverability issues"

Email example 4:

"Hi, Let me introduce myself - I'm William. I've read your comment
on a mail list. I work for company XYZ and we have a great product
called 'XYZ super server' that will take care of all your anti-spam
and email deliverability needs.

Email example 5:

"Hi, Let me introduce myself - I'm William. I work for company XYZ
and we have a great product called 'XYZ gold server' that will take
care of all your anti-spam and email deliverability needs.

Email example 6:

"Hi, Let me introduce myself - I'm Monica. I've noticed that you
spend lots of time online making comments on various mail lists.
I'm currently doing psychology research on the issues faced by
internet addicts and would like to talk to you"

Email example 7:

"Hi, Let me introduce myself - I'm Monica. I've noticed that you
spend lots of time online making comments on various mail lists
and believe you maybe lonely. I'm currently doing psychology
research sponsored by XYZ consoling services on the issues faced
internet addicts and would like to talk to you."

Email example 8:

"Hi, Let me introduce myself - I'm Monica. I've noticed that you
spend lots of time online and believe you maybe lonely. My psychology 
research sponsored by XYZ consoling services found that people like
you would be interested in visiting XYZ match making services site
at xyz.example"

Email example 9:

"Hi, Let me introduce myself - I'm Monica. I've noticed that you
spend lots of time online and believe you maybe lonely. I'm lonely
too and looking for people to talk to and have s_x online. Please
come and visit my site monica.example

-- 
William Leibzon
Elan Networks
[email-masked]

---

From: <[email-masked]>
Date: Mon, 05 Mar 2007 02:03:24 -0600

On Sun, 4 Mar 2007 23:33:21 -0800 (PST)
  "william(at)elan.net" <[email-masked]> wrote:
>>with someone who 
> I think its the spammers who don't understand what is 
>"legitimate first-contact E-mails" or rather they do and 
>they are taking
> advantage of it quite well. 

Certainly they have gotten very adept at obscuring the 
purpose of their E-mail, and at evading filters designed 
specifically to keep their garbage out.

> But let me give you a few 
>hypothetical

Before we get into your examples (and yes, they're 
interesting) let me point out that I probably wouldn't 
object TOO strenuously to recieving most any of them, THE 
FIRST TIME.  Obviously the ones toward the top of the list 
are less objectionable than those at the bottom.

But also, you're falling into the trap of believing that 
you can make a determination based only on content (or 
only on the header).  Specifically, I'd be less annoyed by 
ANY of these if they came from (say) someone whose name I 
recognized as a frequent poster to one of the lists I'm 
on.

I really believe that matching up mail content with 
senders you recognize is a major key to identifying 
legitimate from illegitimate mail (i.e. mail you've seen 
coming from someone, even if not previously directed 
specifically to you (an example, mail to a mailing list 
you subscribe to)).

Once you have seen the same come-on several times, it 
becomes tiresome and annoying.

> and would like to talk to you about spam-related issues 
>you have experienced"

I'd definitely not consider this one spam.  It's a good 
intro message, since it identifies where he found my 
E-mail address, indicates a common interest, and thus 
establishes a reason for contacting me, and indicates what 
he wishes to discuss further.

> issues you have experienced"

Same.

I will note that it is not at all guaranteed that an 
automatic content scanner will make these subtle 
distinctions.

Generally, what one will do will be to add new rules to 
the content scanner as one sees more and more spam mail 
fitting a given and (too-)familiar pattern.

> anti-spam protection and email deliverability issues"

Again, I wouldn't judge this as spam.  He identified the 
ASRG list (which I subscribe to) and asks a question 
relevant to THAT list.

> and email deliverability needs.

I'd probably put this in a "questionable" category, 
presuming that I don't have a rule (yet?) regarding 
e-mails about "XYZ super server".

> care of all your anti-spam and email deliverability 

I might have a rule that indicates I'm interested in 
anti-spam and mail deliverabilty issues.

> internet addicts and would like to talk to you"

On this one, I probably wouldn't consider it spam, the 
first time.  If I started getting 2-3 like this per week, 
coming from a variety of random names, I'd then tag it as 
spam.

> internet addicts and would like to talk to you."

Again, a questionable case but one that wouldn't piss me 
off, THE FIRST TIME.

> spend lots of time online and believe you maybe lonely. 
>My psychology research sponsored by XYZ consoling 
>services found that people like

If I got a lot of mail promoting the site in question, I'd 
set up a rule blocking E-mails mentioning that site, 
unless they came from someone I expected such stuff to 
come from.

> come and visit my site monica.example

Likewise.  If I had listed "monica.example" as a 
spam-promoted site, I'd probably end up telling my filter 
to sequester or T-can such messages, depending on how 
predictably and frequently I received those.

> [email-masked]

Again, content itself isn't really sufficient.  I also 
want to consider who the mail is from, so I could more 
likely pay attention if it's someone I already know.

Gordon Peterson