Subject: [Asrg] FC: Will new "spam reduction" service result in... more spam?

From: Steve Schear <[email-masked]>
Date: Mon, 24 Mar 2003 16:16:13 -0800

From NYTimes.com http://www.nytimes.com/2003/03/24/technology/24PHIL.html

Start-Up Aims to End Spam
March 24, 2003
By JOHN MARKOFF

>In addition to legislative proposals before Congress and
>state legislatures, there are efforts under way within the
>direct marketing industry to try to deal with spam. And
>last week, the Internet Engineering Taskforce, a committee
>of technology experts that sets Internet standards, met in
>San Francisco to listen to proposals for technical
>solutions to spam.

I've been monitoring and contributing to the mail list associated with this 
IETF function now for about two weeks.  Most of the people are looking for 
a magic bullet to cure spam but I think it will not be that simple.  If it 
were it would have already been done.  It seems all the seemingly good long 
term spam elimination approaches either require notable changes to the 
Internet's email or other infrastructure, make it difficult for some 
classes of current email users, or the require the establishment of new 
services (e.g., financial infrastructure to support real value e-stamps).

>The Mailblocks antispam service is based on a so-called
>challenge-response mechanism to block bulk mail sent
>automatically to e-mail accounts. When a customer receives
>a new message from an unknown correspondent, the system
>will intercept the message and automatically return to the
>sender a digital image of a seven-digit number and a form
>to fill out. Once a human being views that number and types
>it into the form - demonstrating that he or she is a person
>and not an automated mass-mailing machine - the system will
>forward the e-mail to the intended recipient.

E-gold uses this approach. They call it a Turing number (after the British 
mathematician, Alan Turing) 
https://www.e-gold.com/acct/login.html  Challenge responses may eliminate 
spam from bogus addresses but it almost sure to set of an image recognition 
arms race between other spammers and Turing number technologists as the try 
to fashion ever more cleaver images that supposedly can be easily read by 
humans but not machines.  There are already programs to "read" earlier (and 
maybe current versions of E-gold's Turing number images.  I would be 
surprised if these measures proved effective

steve

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Brad Templeton <[email-masked]>
Date: Mon, 24 Mar 2003 17:32:56 -0800

> surprised if these measures proved effective

Actually, it is an interesting question of what arms races spammers would
wish to engage in.

From a purely rational standpoint (bear with me on this!) the spammer
simply wants to send as many messages to the best prospects per unit of
time and bandwidth.

This means that if the spammer gets a challenge (or even something as simple
as a temporarily unavailable status) they can do one of two things:
    a) Try to respond to the challenge
    b) Simply move on to delivering the next message in the list.

As long as B is easier than A, the rational thing to do is to just do B.

This changes in two cases.  If most people start issuing challenges or
other such barriers, B is no longer productive, and so you now start the
arms race -- but only until you have enough people to send to again.

Secondly, if you have some idea as to the "quality" of an address, in terms
of probability of making a sale (direct marketers try to measure this all the
time) then you are motiviated to do extra work on the higher "quality"
targets.

Finally, spammers will not be rational, and may wish to get in an arms race
for the spite or challenge of it.   (There's a lot of spite in both directions
in this field.)

Nonetheless, I think people overestimate the arms race.  I have seen challenge
response systems that try to do natural language questions, or embed images
only the human eye can see in graphics.

I wrote a challenge/response system six years ago that simply asks for any
reply at all -- it doesn't put any burden on the other party, and would be
easy to defeat with something as simple as an autoresponder.   Yet it works,
the spammers have not attempted to use this simple defeat.  Once they start,
I will easily enough move to something else, but it is telling that in six
years they have not, even though others have also built a number of
challenge/response systems since then.    Sometimes spammers have autoresponders
for other reasons, but they have been easy for me to eliminate.

---

From: Kee Hinckley <[email-masked]>
Date: Tue, 25 Mar 2003 09:13:46 -0500

>the spammers have not attempted to use this simple defeat.  Once they start,

If a challenge response system puts messages in the "look at me 
later" queue if you don't respond, then I don't think spammers will 
care.  (And it's not clear that you'll be that much happier as a user 
of the system.  You will have to scan the queue.)

Why is not clear to me is a) how anyone expects your typical user to 
whitelist commercial addresses and mailing lists in advance and b) 
how a challenge response system (which had *better* respond to 
envelope from) avoids getting them removed from said list, or not 
receiving notification about their purchase or what not.

Just consider the following.

1 User sends email to [email-masked]?subject=subscribe
2 Think quick.  What address should you whitelist?  [email-masked]? 
[email-masked]?  Nope.  [email-masked].  And you knew 
that because...?
3 asrg sends back a confirmation request.  Now as it happens, it does 
this from [email-masked] (envelope) and asrg-request (from). 
But some mailers use a custom address for this.  But let's assume 
we're dealing with the average user here.  They either didn't do 
anything at all (forgot they had to) or their software whitelisted 
based on the To: address (asrg-request).
4.1 A challenge gets sent back to the asrg list.  The result depends 
on a combination of how the list software works and how the challenge 
software constructed its reply.
4.1.1 It's treated as a bounce and the user is not added
4.1.2 It's treated as a confirmation and the user is added
4.1.3 It goes to the admin, who says something I can't repeat and 
throws it in the trash.
4.2 It makes it through because we whitelisted the right thing.
5 The first list message comes through.  If you had whitelisted 
asrg-admin, you're fine.  If you whitelisted asrg-request, we 
challenge it.  If the list software uses a different envelope from 
each time, you got problems.

Now, let's take amazon.com.

I've received automated email from [email-masked], 
[email-masked], [email-masked], [email-masked], 
[email-masked], [email-masked], 
[email-masked]... and they actually send mail from 
their domain--never mind what happens if they higher m0.net or 
someone to deliver it.

And if you start sending challenges to those--Amazon's going to see 
them as bounces and dump me.

Of course we could just whitelist all of amazon.com.  But I rather 
suspect the spammers might figure that one out.

If you want challenge/response to work, the first thing you should do 
has nothing to do with challenge/response.  The first thing is to 
come up with an RFC for a standard format for challenges so that 
automated mail systems can recognize that they aren't the same as 
bounces.  And come up with a protocol whereby they can reply and say 
"Yo! I'm an automated system you idiot."  Where you go from there I 
don't know.

However, see my next message on "Protocols".
-- 
Kee Hinckley
http://www.puremessaging.com/        Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

I'm not sure which upsets me more: that people are so unwilling to accept
responsibility for their own actions, or that they are so eager to regulate
everyone else's.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Chuq Von Rospach <[email-masked]>
Date: Tue, 25 Mar 2003 08:36:12 -0800

On Tuesday, March 25, 2003, at 06:13  AM, Kee Hinckley wrote:

> whitelist commercial addresses and mailing lists in advance

I think there has to be a responsibility here for the commercial sender 
to help the user figure this out. In fact, it's one of the issues I'm 
mulling over in revamping system documentation on my lists and other 
things. we're now seeing enough challenges that we have to find a way 
to help users figure this out. (FWIW, we don't respond to challenges. 
We've talked it over and decided if the user hasn't whitelisted us, we 
shouldn't validate from the outside. we ring the bell, we don't turn 
the knob. To me, the risks of validating a whitelist and upsetting 
someone are a lot worse than the risks of someone under a whitelist 
expecting to get a subscription and no realizing why it's not 
happening.)

We're probably going to add language explaining whitelisting issues to 
our stuff down the road, since t seems like whitelists are starting to 
be used fairly widely and I expect that trend to continue.

> [email-masked]?  Nope.  [email-masked].  And you knew that 
> because...?

Because I read the FAQ, and it told me.

> suspect the spammers might figure that one out.

So amazon has to figure out whitelists, too, and help people understand 
what addresses things will come from. With a foot on both sides of this 
cashm, I really feel the sender of this mail shouldn't put the burden 
of responsibility on the user here. They need to help them out.

---

From: Brad Templeton <[email-masked]>
Date: Tue, 25 Mar 2003 10:59:55 -0800

> of the system.  You will have to scan the queue.)

Any challenge/response system has to put the unresponded messages into a
queue to look at later.  However mine simply mails me a summary at the end
of the day of messages not responded to for over 2 days.  This summary is
just headers and a few lines of body.  You pick any messages you want from
it and they are delivered.  The summary is sorted by spam-score.  I could
use a fancy spam scorer but a very simple one works pretty well.

So on the occasions when somebody does not respond, or mailing list mail comes
in, it shows in the summary and always in the top.  If I were to improve the
UI it would just mean a quick click to correct the mailing list.

It works very well, and as noted I have been running it with success for
six years, and almost no spam gets through it.   I get 240 spam/day
delivered to it.   I don't find it much of a burden at all to browse the summary
once a day.  In fact, it's somewhat amusing since I am interested in spam
trends.

There is a psychological reason for this.  As a heavy email user, I read
mail all day long, so spam is a constant interruption.   When it's once a day,
the attitude is very different.  It's like my paper mailbox, which also gets
lots of junk mail.   Once a day I take it in, hold it over the garbage pail,
and drop the junk mail and take my real mail.   Like most people, I never
felt it was enough of an annoyance worth reworking the postal service over.

> receiving notification about their purchase or what not.

No autoresponder should respond to a bulk message at all.  No, users will
not whitelist their mailing lists, but my system works reasonably well.

Of course, there is another alternative which also works well at least in
newsgroups, which is to use a filtered address to post to newsgroups and
an unfiltered for mail you send out to everything but mailing lists.  Mailing
lists are a problem.  One would like to use the "public" address in mailing
them, but it's a pain to configure most lists to take mail from an address
other than the one you subscribe under.  Because, the irony not being lost, of
spam.
> [email-masked]?  Nope.  [email-masked].  And you knew 

I think this is a hard problem. Some try to solve it but you can't do it
entirely by looking at outgoing subscribes.  Many people subscribe via
the web.

Another good thing you can do is notice patterns to spot possible mailing
lists you have joined on incoming mail.

> software constructed its reply.

No, only broken autoresponders, which should be taken off the list, would
challenge list mail with a proper Precedence header.

(Oddly, most spammers don't put on proper precedence headers, and if you want
to, you can temporarily use that as an indication of non-spam!)

You won't convince me a challenge/response system won't work, because I have
been using one with great success for 6 years.  It's not a matter of
opinion.  However, they do have flaws, and handling legit mailing lists is one
of them -- they are the killer in almost all spam systems.  They also
have trouble with anonymous mail, which goes into the review queue.

However, I would strongly oppose a challenge/response system without a
regular summary of the blocked mail.  In fact, all spam systems should offer
one.  They can sort it by spam score, and make it infrequent, but it should
always be there.   If your mail is to be blocked, you should know it, and
there should be a way to detect bugs and errors in the blocking software.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Kee Hinckley <[email-masked]>
Date: Tue, 25 Mar 2003 17:21:38 -0500

At 8:36 AM -0800 3/25/03, Chuq Von Rospach wrote:
>I think there has to be a responsibility here for the commercial 
>sender to help the user figure this out. In fact, it's one of the 
>issues I'm mulling

I would go further.  I think we should have a standard mechanism 
(akin to mailto:, but probably using a new scheme name) specifically 
for whitelisting.  If your email client supports whitelisting, it 
should support a url that you can click on to automatically whitelist 
an address.  (And of course, it had better confirm this with the 
user, or the spammers will just embed it in an image tag.)  Then web 
sites can easily provide the user with a way to whitelist the 
necessary address or addresses.
-- 
Kee Hinckley
http://www.puremessaging.com/        Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

---

From: Daniel Feenberg <[email-masked]>
Date: Tue, 25 Mar 2003 18:17:43 -0500 (EST)

> necessary address or addresses.

Is there something in this proposal to prevent spammers from using
Amazon.com as their return address? Do we assume that mail is securely
signed? Or does the whitelist include the IP address of the connecting
system?

Or if not, would amazon.com eventually get to using a unique from: address
for each receipient, that the spammer couldn't guess? If they didn't, they
would have troubles communicating with customers, because noone would want
to put them in their whitelist. But that means that there can't be any
cooperative whitelists, or sharing of whitelists, which is quite a shame.

---

From: Kee Hinckley <[email-masked]>
Date: Tue, 25 Mar 2003 19:54:06 -0500

>signed? Or does the whitelist include the IP address of the connecting

Independent issue.  The direction I see a number of people moving in 
this group (and the first productive move I've seen so far :-) is to 
see if we can specify certain standards that would provide a 
framework for anti-spam software.  The goal is (in my mind at least) 
that even if people try a number of different solutions, at least 
there will be some degree of interoperability.  I'm not proposing any 
standard for how whitelisting works.  Just that there should be a 
standard mechanism for a mailing list operator to easily provide the 
necessary information (via a URL) to your whitelisting software.

That was as far as I got.  You've pointed out that of course there 
are different ways whitelisting could work.  Address, domain, IP 
address, certificate, unique token all come to mind.

Clearly which ones a given vendor provides depends on what they feel 
like supporting (may the most popular solution win).

So what probably makes sense is to define a scheme (in the technical 
sense--that thing to the left of the colon) and a set of standard 
arguments, as well as a mechanism for adding items to the standard.

Completely off the top of my head:

email address (possibly wildcarded--although how you do that without 
using legal email characters I don't know off hand--need some 
encoding)
IP range
special token
public key

Those all seem like things that a whitelist system might want to 
know.  And if the client does use one of them, it can ignore it.

Back to your original question.  Won't the spammers just start using 
Amazon's address if they think a substantial number of people have 
whitelisted it.  Yes.  That's a fundamental flaw with whitelists.  On 
the other hand, spammers who do this run far more serious legal 
risks.  But the goal of this proposal wasn't to solve the problem 
with whitelists.  I don't even *like* whitelists :-). The goal is 
simply to make it easier for recipients and senders to use them.
-- 
Kee Hinckley
http://www.puremessaging.com/        Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

---

From: [email-masked]
Date: Wed, 26 Mar 2003 00:06:07 -0800 (PST)

Standard way to report things is good. URLs are good, but they are added 
for specific protocols, so do you have a new protocol in mind for 
whitelisting?

---

From: Kee Hinckley <[email-masked]>
Date: Wed, 26 Mar 2003 11:31:59 -0500

>for specific protocols, so do you have a new protocol in mind for

Not for implementing whitelisting.  And not a protocol per-se 
(leastwise I don't tend to think of mailto: as a protocol--it's more 
like a single-call API with an optional set of arguments).  (Then 
again, I've already been chastised once off-list for my loose use of 
the word "protocol" :-).

It seems to me that whitelists have two major drawbacks.

1. User doesn't whitelist in advance, or know what *to* whitelist in advance.
2. Spammers can take advantage of them.

I have no intention of trying to solve #2.  But there seem to be a be 
a lot different solutions out there that depend on whitelisting.  I 
think that they would benefit from a mechanism whereby a 
list-maintainer could provide a simple clickable URL that would 
provide the necessary whitelist information to the whitelisting 
software.  E.g.  "whitelist:[email-masked]".  Presumably a set of 
optional arguments would be specified that would allow use by more 
sophisticated whitelisting systems. 
"whitelist:[email-masked]?pgpkey=xxxx&iprange=yyy-xxx".

I don't think I'm the right person to do this.  To me whitelisting is 
what someone does when our software has a bug :-).  But if nobody 
else is willing to pick it up then I guess I can try see if I can't 
come up with a list of information used by current whitelisting 
systems.

Again (since I've received a couple messages about this offlist), I 
have no interest in discussing pros and cons of different 
whitelisting systems.  I just want to make life easier for someone 
who maintains a mailing list and is getting sick of the growing 
number of subscriber bounces.
-- 
Kee Hinckley
http://www.puremessaging.com/        Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

---

From: Chuq Von Rospach <[email-masked]>
Date: Wed, 26 Mar 2003 08:52:39 -0800

On Wednesday, March 26, 2003, at 08:31  AM, Kee Hinckley wrote:

> have no interest in discussing pros and cons of different whitelisting 
> systems.

I can say from looking at what we're seeing come back, whether you like 
whitelists or not is irrelevant. They're here, and they're being 
adopted in with increasing speed. So coming to grips with them as a 
service creator is necessary.

And personally, I've gone from being "not a fan" of whitelists to 
believing it's the way to protect role accounts from the growing abuse 
we're seeing (personal accounts I'm on the fence over, but that's a big 
shift as it is). I've had (or watched) discussions about how whitelists 
suck with people who instead come up with increasingly baroque 
validation schemes for web pages, which, when you boil it down to 
essenticals, is still a challenge/response system, just moved to a 
different point in the process, and whcih does nothing to stop mail 
directly into role accounts. and maybe I'm crazy, but I still have a 
real person who reads postmaster and abuse, and as someone who hates 
getting lost in "press 3 to get lost in our phone tree" voicemail 
systems, I don't want that to change -- when technology bollixes, the 
last thing you want is another piece of technology making sure nobody 
can tell you (and being pissed at you because of it).

All IMHO, of course. But I think it's beyond time to argue about 
whitelists, it's time to figure out how to deal with them and get used 
to responding to them.