Subject: Re: [Asrg] Washington Post: Earthlink to Deploy a Challenge-Response System for

From: "John Fenley" <[email-masked]>
Date: Fri, 09 May 2003 12:15:34 -0600

>From: Vernon Schryver <[email-masked]>
> > From: "John Fenley" <[email-masked]>
>
> > Someone mentioned writing some guidelines for responsible 
>Challenge/Response
> > systems. Any chance we could officially ask Earthlink to hold off 
>deployment
> > for a month so we can get something together to guide them?
>
>Who is we, Kemo Sabe?  As I keep saying, the IETF is about increasing
>the likelihood of interoperation, not policing the network.  What
>consenting parties do with IP packets is their private business.
>Earthlink and Earthlink's customers are consenting parties minding
>their private business.
>
>But set that asside and also the fact that this is an IRTF instead of
>IETF group.  If you were in charge of Earthlink and so probably had
>a few minions who are familiar with the way the IETF works and know
>that BCPs and standards track RFCs require at least months and sometimes
>years to to get past the final last call, what would you do?  If you
>were an Earthlink stockholder, wouldn't you expect your board of
>directories to fire CEO that delays offering something the market
>wants because the IETF might someday say something about the best way
>to implement it?
>
>Don't you suppose that Earthlink had surely been testing their scheme
>in fairly large scale trials for months before the public announcment?
>Have you ever tried to tell people that what they've built and tested
>for months must be delayed because some self-appointed experts like
>us haven't commented on it?

Ok, good point, but I was mostly thinking about copycats. Who knows how many 
other ISP's will copy what they do, but not put as much thought into it.

A list created by the ASRG to help all prospective copycats from making the 
simple mistakes could save a lot of pain for everyone.

I see Challenge/Response as the final threshhold of Spam Blocking. If it is 
done wrong on a large scale, it could sour the idea for everyone. Then when 
a truly good way to impliment it comes, everyone will dismiss it as having 
the same old problems.

>
>
> > Perhaps even a list of fatal flaws and suggested best practices would 
>help.
> >
> > These are my suggestions for a beginning list of problems:
> > 1. Avoid infinite loops.
>
>Let's assume until contrary evidence arrives that Earthlink's designers
>and implementors are not complete idiots.

Some groups of people are complete idiots. The people at Earthlink may have 
a very good plan, but i havn't seen it, and I prefer not waiting for a 
catastrophy.

> > 2. Allow all Opt-in mail.
>
>That is an impossible goal.

Not realy, just requires infrastructure that isn't available yet.
I think Choicelist could handle this problem. I created that system 
specifically to solve this problem with C/R.

>
> > 3. take steps to prevent spammer evolution.
>
>That sounds impossible.

You just need to provide an incentive to behave in a certain way.
Let people bypass the filters by using ADV, then all the ADV mail is easy 
for a user to deal with.
Personaly it isn't the volume so much as the effort required to deal with 
it.
If I could delete all the ADV mail with one click, I realy wouldn't care how 
many I got. 1000 is just as easy as 1, and the spammer has gotten through 
the filters without needing to innovate.

They get lazy, we just delete their junk. simple.

> > 4. Provide proof that the challenge is legitimate.
>
>Why except as eyewash?   And what would it be?

Spammers have already started spoofing challenges to try to get through. I 
didn't expect this, but it needs to be addressed immediately.

I don't know what form it would take yet.

> > 5. Avoid simple reply style challenges that are easy to autorespond to.
>
>That is a design trade-off.  If you do that, you also prevent some
>legitimate responses.  Not all mail involves HTML.  If it's text,
>it' at least practical to parse.

I meant require a turing test of some sort. Earthlink already has that, but 
copycats my not realize that this is important.

> > 6. Use generous auto whitelisting.
>
>That also involves design trade-offs that can argue to the contrary.

Then argue them if you feel inclined, but don't make me guess what they are.

> > 7. support all types of  identity proofs including new ones.
>
>We don't want much, do we?

I feel it is important that any fledgling system at least have the ability 
to adapt to later standards.
if your gonna change the whole thing anyway, you might as well at least 
aknowledge that spoofing will be a problem and leave an avenue to deal with 
it. so i'll change that last one to:

7. Leave room for authentication mechanisms.

> > 8. Don't just delete everything that doesn't respond correctly, perhaps
> > reward correct challenges by placing them at the top of the inbox.
>
>Spam that you see is not really filtered.  Which of your time is wasted
>on spam matters less than than the fact that your time is stolen.

This was from a suggestion by someone else, it helps limit false positives.
Use the C/R as a helper, not a blocker. I thought that was a great idea.

>I hope I've at least demonstrated that a challenge-response BCP is
>not as simple as it sounds and that it would be unwise for Earthlink
>to wait for an Internet ***RESEARCH*** Task Force to speak before
>responding to the marketplace.

I guess your right, but that doesn't mean we shouldn't make the list to 
guide others. there will be stupid copycats. Earthlink is just going to be 
testing now anyway, they are in beta, they would probably like a little free 
help seeing problems.

John Fenley

_________________________________________________________________
Help STOP SPAM with the new MSN 8 and get 2 months FREE*  
http://join.msn.com/?page=features/junkmail

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Vernon Schryver <[email-masked]>
Date: Fri, 9 May 2003 13:05:37 -0600 (MDT)

> ...

I agree that a BCP for challenge-response spam defenses is a good idea,
for future implementors.

> ...

I don't understand how anyone might hope that Challenge/Response won't
be done wrong on a large scale even with completely clear standards
track RFC.  Perhaps my point of view is colored by old scars inflicted
by large outfits.

I've not responded to you counters to my counters to your points about
Challenge/Response systems, because unless you're working on an ID,
I fear it would be a waste of effort.

Vernon Schryver    [email-masked]
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: "Eric Dean" <[email-masked]>
Date: Fri, 9 May 2003 15:30:21 -0400

We have built a very good challenge response system (and I'll save the
marketing plug).  It works real well...except that I find that the concept
doesn't particularly work real well.  By this, I mean, many people simply
don't respond to the challenge verification...I'm not sure this will
change..doesn't matter if it does.  Therefore it's an option that users can
disable.

However, what is of value is that we use the C/R as a probe to see if an
actual email address really exists.  When a spammer spoofs an aol or yahoo
address..and use a bogus sender..we get an immediate unknown-user reject..as
opposed to other systems that ingest the message and then bounce later
(bounce messages are of a type variety).  We actually drop a non-trivial
percentage of spam just by probing the sender's email address.

---

From: Vernon Schryver <[email-masked]>
Date: Fri, 9 May 2003 13:36:09 -0600 (MDT)

> From: "Eric Dean" <[email-masked]>

> percentage of spam just by probing the sender's email address.

By this measure, how many of the addreses at free providers are invalid
when you test them?

Depending on how you treat failures to respond to C/R probes, you should
also be able to guage how quickly drop boxes are terminated.

Have you sampled the system to see how many of the messages dropped
are spam instead of legitimate mail with "munged" or otherwise bogus
sender addresses?

---

From: "Eric Dean" <[email-masked]>
Date: Fri, 9 May 2003 16:41:52 -0400

> By this measure, how many of the addreses at free providers are invalid

I haven't looked at the percentage of valid email from free providers vs the
amount of email dropped as a result of the probe C/R...however, I have
looked at the percentage of spam that is dropped automatically (via the C/R
probe) vs that which is dropped manually by the user.  We drop approximately
half of the spam via the probes...however, it's somewhat trivial for
spammers to double their efforts.  We also see about 95% of senders are
validated by the users rather than via sender self-verification.

A couple of other interesting, yet intuitive, things we see is that
blacklisting spammers are all but useless.  Spammers most always use a
unique sender address to track successful delivery.  The only real people on
my personal blacklist are from legitimate solicitors that somehow think I
opted in for something.  For the most part, dropping the single spam and
blacklisting the spammer produce the same results.

I hesitate to say that we need to be careful not to get into a
anti-bacterial soap type of circumstance whereby we direct spammers to start
spoofing valid freemail accounts.  As with ant-bacterial soaps...only the
mutant strands of bacterial that are impervious to the chemicals survive and
spawn.  If we don't come forward with a bullet-proof method of ending
spam..then we probably shouldn't do anything at all.  If spammers start
spoofing valid sources, then our problems have puppies...until we wind up
with PKI everywhere.

---

From: Vernon Schryver <[email-masked]>
Date: Fri, 9 May 2003 15:22:10 -0600 (MDT)

> amount of email dropped as a result of the probe C/R.

That's not what I meant.  I've long claimed that most of the addresses
used as envelope Mail_From values in spam are not "forged," but the
legitimate property of spammers.  My claim would be supported if a
substantial number of the probes you send to Hotmail are not rejected
as invalid addresses.

>                                                     ...however, I have
> half of the spam via the probes

That's interesting, but it seems to concern the effectiveness of C/R
as a spam filter.  What do you mean by "dropped manually by the user"?
How can the target of mail drop spam before the C/R probing is complete?

>                                ...however, it's somewhat trivial for
> spammers to double their efforts.  

What efforts by spammers might affect the effectiveness of your C/R system

>                                    We also see about 95% of senders are

Does that mean that your users white-list mail senders before the C/R
system gets a chance to try?  Or perhaps after the C/R system has
tried and failed?  Are you saying that that the C/R system in some
sense fails 95% of the time?  That's much worse than even I would
guess, and I think C/R systems are hopeless.

> spoofing valid sources, then our problems have puppies...until we wind up

That horse has left barn a long time ago.  From my observations, spam
is less likely to involve spoofing valid sources today that it was
half a dozen years ago before the Flowers.com case and the many laws
against header forgery.  The frequence of spoofed valid sources remains
signifcant, but ebs and flows as spammers come and go, the moon waxes
and wanes, and no one can know what else.

---

From: "Eric Dean" <[email-masked]>
Date: Fri, 9 May 2003 21:37:27 -0400

> substantial number of the probes you send to Hotmail are not rejected

I apologize for making general comments.  All of our mail gets stored in a
database as well as all relevant statistics.  Let me run some number and get
back to you.  In general though, it's not difficult for a spammer to send a
million messages with a valid freemail account and a million with an invalid
freemail account.  Any assertion we make today and design as an assumption
will certainly be thwarted by less than 3 lines of additional code for
shareware tomorrow.

> How can the target of mail drop spam before the C/R probing is complete?

Ah, yes.  I'l try not to tout may warez but rather share our approach.  When
we "quarantine" email, we send a C/R for users that have the function
enabled.  In addition, we periodically send a digest of all email within the
quarantine database to the user.  Often, we find that the user takes a
whitelist, blacklist, forward, drop policy decision before the sender
responds to the C/R.

> system gets a chance to try?

yes, or blacklist

> Or perhaps after the C/R system has
> tried and failed?

Also possible.  If the C/R gets an SMTP reject, then we'll deep-six the
message automatically..assuming the reject isn't due to a valid reason.

> Are you saying that that the C/R system in some
> sense fails 95% of the time?

Depends on what you define as a failure.  Our observation is that C/R is not
a panacea.  People are not gravitating to it in droves.  It's a nice tool
that has some nice capabilities...most of what I like is dropping bogus
senders moreso than dynamically adding self-verified senders to a whitelist.

> That's much worse than even I would

I would say that alone, they have valueable but limited functionality and
only help with spam prevention.

> and wanes, and no one can know what else.

If there's a name for it..it means that someone else has done it...and a
spammer has done it a million billion times.

---

From: J C Lawrence <[email-masked]>
Date: Fri, 09 May 2003 18:39:27 -0700

On Fri, 9 May 2003 13:05:37 -0600 (MDT) 
Vernon Schryver <[email-masked]> wrote:

>> I see Challenge/Response as the final threshhold of Spam Blocking. If
>> it is done wrong on a large scale, it could sour the idea for
>> everyone. Then when a truly good way to impliment it comes, everyone
>> will dismiss it as having the same old problems.  ...

> track RFC.  Perhaps my point of view is colored by old scars inflicted

Certainly the clearly written standards on vacation bots have shown us
what behaviour we can expect on a regular basis.

> Challenge/Response systems, because unless you're working on an ID, I
> fear it would be a waste of effort.

Would it be worthwhile spending the time to wrap the consent token or
forward chained received headers stuff into an ID?

-- 
J C Lawrence                
---------(*)                Satan, oscillate my metallic sonatas. 
[email-masked]               He lived as a devil, eh?		  
http://www.kanga.nu/~claw/  Evil is a name of a foeman, as I live.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Vernon Schryver <[email-masked]>
Date: Fri, 9 May 2003 20:21:49 -0600 (MDT)

> From: J C Lawrence <[email-masked]>

> what behaviour we can expect on a regular basis.

Yes, although I was feeling PPP scars.

> forward chained received headers stuff into an ID?

On the off chance that was addressed to me, I'd say that whoever
invests the editorial work must decide.  It seems likely that only a
support would want to invest the work.  Everyone who has commented on
those or other mechanisms is duty bound to at least try to read such
documents if they appear.

We currently have 
  - Dave Crocker's ID that he has promised will have a new version
      realsoonnow.
  - Paul Vixie's draft ID for an MX checking DNS mechanism.  Paul has
      left the building, but perhaps he could be convinced to return
      long enough consider suggestions and forward the document.  If
      not, he might be willing to pass the document to someone friendly.
  - Gordon Fecyk's ID.
  - others I can't seem to remember.
  - still other documents that as far as I know have not been layered
      with boilerplate and generally hammered into draft IDs.

I don't know, but suspect all but Dave Crocker's draft ID would have
to be "Informational" "individual submissions" or come from an IETF
WG and not from here in either case.  The three not yet draft IDs are
official products of this group, but it's early to think about their
fate until they become draft IDs.

Those not familiar with the IETF/IRTF or standards committees in
general might not realize that the IETF has strong fences separating
committee turfs.  Standards committeee fences are honored to keep the
inter-committee battles from getting too distracting.  IEEE PARs and
the ANSI equivalents are higher and defended more ferociously than
the IETF fences, but the IETF equivalents can't be violiated by
newcomers without lots of political capital.  Those who have enough
capital are unlikely to spend it on minor spam IDs.

That's not to say that talk about those IDs is off-topic, but that
getting support here is only a first, small step in a long process
unless you take the shortcut straight to the IESG for the low goal of
"individual" and "informational."

I hope that if I've misstated the rules of the RFC game, someone more
knowledgeable will correct me.  I've tried to use lots of jargon to
give pause to newcomers who might otherwise simply inform me that I'm
all wet about the Standards Process.