Subject: Proposal For Token-Based Authentication In Mail Submission As Anti-Forgery Effort

From: "Sabahattin Gucukoglu" <[email-masked]>
Date: Sat, 06 Mar 2004 05:54:54 -0000

Hi people,

Before I make a gigantic fool of myself, I'm going to lay these ideas 
before the IETF so I can have the usual disperse range of comments before 
pushing this toward the ASRG/draft/similar.  I know of only one distinct 
and important catch in this proposal, that being a modification to all 
MUAs and MTAs to allow the acceptance and carrying of a password token 
which should persist throughout the entire mail delivery, which may or may 
not somehow get itself encoded into the envelope sender (using a hack in 
source routing, perhaps) if persistence can't otherwise be guaranteed due 
to lack of ESMTP implementations and/or other problem with this 
assumption.  In other respects, though, I think you'll find complete 
backward-compatibility and all round niceness.  There may also be this 
issue with new network traffic generation, which we'll try and address 
later.  Note as well that this isn't the full story since we deliberately 
leave out a big part of this spec to your own imaginations - the token 
server, DNS record format and said source routing hacking.  More later.  
It goes without saying that if you have any questions, you should feel 
free to ask!

Okay then, here we go:

My proposal is a scheme for anti-forgery, which makes use of a non-blank 
token, or password, which can be verified by a recipient system as 
matching against a password submitted by a sender and against a list of 
passwords held at a fixed location by the sender domain's designated query 
resource, before accepting mail from the sender.  This means that the 
whole bother about listing domain names and IP addresses should be 
completely alleviated, even when a user makes use of spoofing and 
autoforwarding for good reasons and provided the envelope is properly 
preserved for the true sender; only the password should determine whether 
the recipient gets the mail and/or considers it likely spam, and it's the 
sender's responsibility to ensure it gets sent either directly or by the 
forwarder.  The need for rewrites may or may not be necessary, that's open 
for you lot to discuss.  In short, it's like SPF, but without the 
heartache inherent in the good and sensible among us resulting from such 
strong dependence on the aspects of connecting hosts, but with the added 
inconvenience of additional network traffic (more later) and the token 
transfer.  Here's the outline:

1.  Sender MUA submits message through some host X, indicating token to X 
using the stok extension to be defined in SMTP as an extention to its 
"mail from:" command:
mail from:<[email-masked]> stok=blorb
If the server doesn't support the extension or ESMTP, the client will 
proceed normally, and the authentication simply won't work, with or 
without fatal results.

2.  X stashes away token, and does the usual lookup in DNS to figure out 
where the mail is going.  It may use one or more relays, in which case it 
should submit the mail envelope in precisely the same way as in step 1, 
above, inclusive of token as parameter to stok.  The point here is that 
the password *must* be submitted down the whole chain as in the 
preservation of the envelope, else the whole thing breaks.  Still, it does 
so in a friendly and hopefully short-lived way.

3.  Assuming X uses one of example.org's MXs and delivers straight to it, 
X will listen out for the advertisement of the stok keyword, and make the 
final mail transaction, beginning with the "mail from:" and including the 
stok=blorb as above.  At this point, policy at the MX determines how mail 
from unauthenticated submitters is to be handled; it might refuse outright 
if no token is given, it might flag it with a header - whatever takes its 
fancy.  Anyway, here comes the bad bit.

4.  MX begins a password query.  It must connect to some kind of password 
query resource.  The MX may connect to a designated MTA for a domain and 
use the stok keyword to query for a password (>>"stok blorb" <<"250 Token 
is tasty!") or some other simplified database query.  No publically 
accessible list may be used, since spammers can just as easily use those 
(my original thought before it bombed on me), unless they are accessed by 
verifying that the MX is somehow in authority to do so, provided no other 
host can (I can't think of such a way that can't somehow easily be 
circumvented by someone with a DNS server handy).  Anyway, whatever way 
this query works, we publish the resource type and location in DNS (or 
just the location if only a single type of query is used) for sender 
domain in a TXT record of that sender domain.

5.  Authenticated submitters are welcome, unauthenticated submitters 
aren't, policy-dependent.  All done!  Expectations will naturally be that 
admins start gentle and get more and more aggressive.

Sidenotes:
1.  Implement caching in password requests, similar to DNS, to assist in 
keeping traffic minimal and/or use a very minimal password protocol and 
daemon?
2.  Sender designates stok (Submittal Token) upon creating his/her account 
with his/her ISP.  MUA of user must add a submittal password field in its 
configuration.  User must be permitted to change stok whenever he wants 
with his provider, perhaps using a provided protocol/service as in POP3PW.
3.  Security depends on the protection and unguessability (sophistication) 
of passwords.  Those tokens will contain no whitespace characters but may 
contain legally permitted characters in SMTP transactions.  If a spammer 
gets hold of someone's password, we're back on square one.  Thing is, the 
password tells us, without a doubt, who the abusing (or abused victim of 
identity theft) was.  ISP disables that password, all is good again, and 
we can also trace abused/abuser with much greater ease.  Naturally, we 
don't want to put the password out to the public.
4.  MX policy decides whether stok is sufficient enough both to 
accept/reject/tag/whatever mail, and also whether it is now safe to lower 
open relay or other barriers.  However, I personally advise caution - 
unless you can be sure that spam from other than forged domain envelopes 
will not in big part emerge through your server, perhaps because you 
bother to do the A-RR check and also check the existence of a given 
domain, don't do it.  The short is that this is intended to be forgery 
protection to go hand in hand with other means of spam detection and help 
alleviate the awful blockage of loads of innocent hosts, and maintain your 
freedom to use any relay in your power, not to encourage open relays; I 
just hope one day the usefullness of such a system will make it plausible 
in the future to do so again.
5.  A password query was picked rather than a password list because it 
does away with both too much complexity and the whole issue of how to keep 
the list away from one pair of eyes and available to another.
6.  The insentive to move to this type of standard would be that final MXs 
will start refusing, or marking as spam, any unauthenticated submittals.
7.  The use of the envelope is yet to be dreamed up by enthusiastic minds; 
needless to say we want an audience here and may not be able to depend 
easily on ESMTP (though this seems unlikely).
8.  This won't stop all spam.  If a spammer sets up shop, we're all in 
for, especially since standard domain name checks by most modern MTas 
won't work neither.  Still, tracing these virmin suddenly becomes a bit 
easier.

Right, I think that's everything.  Any questions?  What do you reckon?

Cheers,
Sabahattin
-- 
Thought for the day:
    Bagpipes (n): an octopus wearing a kilt.

Latest PGP Public key blocks?  Send any mail to:
<[email-masked]>

Sabahattin Gucukoglu
Phone: +44 (0)20 7,502-1615
Mobile: +44 (0)7986 053399
http://www.sabahattin-gucukoglu.com/
Email/MSN: <[email-masked]>

---

From: Vernon Schryver <[email-masked]>
Date: Sat, 6 Mar 2004 08:08:38 -0700 (MST)

> From: "Sabahattin Gucukoglu" 

> ...
> which should persist throughout the entire mail delivery, ...

> token, or password, which can be verified by a recipient system ...

How does your scheme prevent forgery better than SMTP-AUTH, SMTP-TLS,
S/MIME, or PGP?  Most MTAs support SMTP-AUTH and SMTP-TLS.

The difficulties in preventing spam sender forgery are not in defining
token protocols but in

 - defining forgery in a way that excludes common, legitimate practices.
    Why isn't sending mail with your Hotmail account as sender while
    sitting at your desk at work or with your work sender address while
    in a customer site, hotel room, or airplane "forgery" that would be
    prevented by the proposed sender-verifying mechanism?

 - key distribution.
    Verisign/Microsoft would be happy become the toll collecter for
    all Internet mail using the current or a variation of the current
    commercial PKI.  Some of us are not keen on that idea.  All other
    key distribution mechansims have their own substantial problems,
    including those that would use the DNS.

 - preventing spammers from buying as many tokens as they need.
    The major spammer that currently calls itself Zhang Jun and Qing
    Zhang has been burning several new domain names per day for the
    last 2 or 3 months.  Why won't it be able to make or buy tokens
    to go with each of its domain names?  Why won't domain2004.com,
    managernic.com, namelite.com, nicsimple.com, sitesadmin.com, and
    the rest of its DNS servers serve its SPF RRs or your password
    tokens as readily as they serve NS, MX, and A RRs?  Why can't it
    replace those DNS servers as they become recognized with new DNS
    servers, as it has been doing for years?

> ...
> is tasty!") or some other simplified database query.  ...

> aren't, policy-dependent.   ...

Have you looked at SMTP-AUTH?
What about SMTP-TLS with verified certs required?

I hope you won't be too offended if someone points out 
http://www.rhyolite.com/anti-spam/you-might-be.html
I wrote it during the first months of the ASRG mailing list.

Vernon Schryver    [email-masked]

---

From: Michael Thomas <[email-masked]>
Date: Sat, 6 Mar 2004 08:19:00 -0800 (PST)

Vernon Schryver writes:
[]

You know, it's quite elucidating seeing the banter
about the subject of spam, especially between you
and Paul but what strikes me more is the
overarching dynamic going on:

for (;;) {
   1) proposal is made
   2) proposal is classified into one of several
      general buckets
   3) proposal is deconstructed by those who've seen
      this many many times
   4) no silver bullet is uncovered
}

In the mean time, the exponential curve of spam
keeps on moving along the X axis for ever greater
values of X.

So... instead of pointing out the obvious that
there is no silver bullet, wouldn't it be a lot
more productive to frame this debate in terms of
what incremental steps could be taken to at least
try to change the overall climate? To perhaps move
things in a direction that might be in our favor?
To perhaps be open to making some mistakes and/or
no-ops?

We know spammers are smart and adaptable. The
problem is that in our paralysis, we are not.

	   Mike

---

From: Vernon Schryver <[email-masked]>
Date: Sat, 6 Mar 2004 10:56:24 -0700 (MST)

> From: Michael Thomas <[email-masked]>

> To perhaps be open to making some mistakes and/or

Am I interfering with incremental debate framing, climate changing, or
designing, implementing, testing, and deploying possible solutions that
might be mistakes and no-ops?  I hope not and I don't think so.  In about
1997 Paul Vixie mentioned the notion of spam checksum clearinghouses.
I pointed out the obvious problems, but 6 or 9 months later hacked a
form of the idea into sendmail.  The DCC is now resisting about
350,000,000 spam/week.  When I heard about greylisting, I pointed out
some obvious problems, but worked hard to add it to the DCC client code.

That a problem seriously wants a solution does not imply that it has
one.  That personal immortality, matter transmission, and communicating
consent to receive mail sound nice does not imply that they are possible
or that they would solve more problems than they would create.  Either
way, lists of problems from wet bankets like me should not stop anyone
from designing, implementing, testing and deploying, unless they need
to sell a lot of stock beforehand.

> problem is that in our paralysis, we are not.

Whose paralysis do you mean, Kemo Sabe?  Outside the mass media, mailing
lists, and usenet, plenty is being done about spam.  Some efforts have
been more effective than others.  Others such as laws have more future
hope than past performance.  Filter effectiveness above 95% is common.
Reasonably spam free mailboxes that are open to mail from perfect strangers
are more readily available today then they were 3 years ago.  Nothing
so far have been or will be a silver bullet.  Unless you believe vague
handwaving or swallow any of several brands of patent medicine, there is
no prospect of a FUSSP (Final Ultimate Solution to the Spam Problem).

By itself, framing debates is not productive unless you're only
interested in debates.  Few of those who do more talking and writing
about spam than administrating anti-spam mechanisms, designing, writing
or deploying code, enforcing laws, or anything else that directly
affects spam in more than their personal mailboxes are contributing
to solutions.

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Sat, 6 Mar 2004 15:41:56 -0800

At 8:19 AM -0800 3/6/04, Michael Thomas wrote:
>try to change the overall climate?

Only if such framing includes the costs of the steps. To date, most 
of the initial proposals we have seen on this (and many other) lists 
have three attributes in common:

- They don't list the obvious problems

- They don't even guess at the costs of those problems

- They don't have an analysis of how hard or easy it will be for 
spammers to adapt to the proposal

We have already seen that every deployed anti-spam solution has 
costs. We have already seen that those costs can be listed with an 
extra hour or two of effort. We have already seen that spammers 
quickly adapt to anti-spam tools.

This is not that much different than doing a security analysis on new 
protocol proposals. "Just authenticate {senders | MTAs | messages}" 
is not that different than "we authenticate by sending a password in 
the clear".

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: "Sabahattin Gucukoglu" <[email-masked]>
Date: Sun, 07 Mar 2004 08:31:57 -0000

> S/MIME, or PGP?  Most MTAs support SMTP-AUTH and SMTP-TLS.

That should, you're right, be "Sender forgery".  SMTP Auth is not carried 
from host to host and only guarantees some privileged user his rights, 
such as relaying or submission, during any given transaction; it doesn't 
permit an end-to-end verification of sender across multiple hosts, 
necessary for normal mail delivery.  It doesn't (or shouldn't) prevent any 
mail from being delivered at all to a domain if the destination host is an 
MX for that particular domain, regardless of any details of the mail.  
There are issues concerning today's worms/viruses that make port 25-
blockage on all but the local smarthost more plausible and reasonable, 
too, and the submission port won't take long to follow when the next gen 
of worms flood the net.  I don't like it, y'know, but I'm ready to *grrrr* 
face it.  It is, in summary, useless as a measure against sender forgery 
in short term, and I'm ready to bet long term as well, despite having its 
real uses as a trusted submitter/relay-permitted client authentication 
method.  Good security practice to keep hackers from hopping your firewall 
and trying to relay through your system from a local vantage.  PGP is 
strictly per-user, and isn't practical for mail delivery.  Use of it for 
quality spam detection to any useful worth is akin to whitelisting, and 
PKI infrastructures and corporation-determined trust are a point of 
weakness (as you say, below).  Same for StartTLS and S/MIME, even if TLS 
*were* encouraged to global use.  If any of these were at all useful, they 
would either need to be updated and/or used and deployed almost 
immediately, which is obviously impractical.  At least this means holds as 
much configuration and balanced (and uncontrolled) distribution as DNS 
itself, not to mention gentle backward compatibility.

>  - defining forgery in a way that excludes common, legitimate practices.

This proposal attempts exactly that!  Other proposals we've seen all 
depend on administrative configuration which predetermines some particular 
aspect of mail delivery, i.e. IP addresses from which any given mail 
should emerge for a particular domain, all of which harm - as you say - 
legitimate transactions.  This scheme leaves that choice to the user.  
Admins simply maintain a list of user-chosen passwords, and it is the user 
who, by providing the correct password to his MUA when submitting mail 
through his work's MTA under his Hotmail address, guarantees its delivery 
when, at the other end, the user's password (not his domain name, not his 
IP address) is verified against the same password at the user's home 
domain - Hotmail - by a query from his mate's MTA a moment later.

>     sitting at your desk at work or with your work sender address while in
>     a customer site, hotel room, or airplane "forgery" that would be

Again: only the password is verified!  Forgery is now dependent only upon 
the misuse of the password.  The administration of the domain referenced 
in the envelope sender is now soley responsible for any of its spamming 
users, case by case, and there is no opportunity for a domain to be 
misrepresented provided those passwords are the cause of the mail's 
acceptance.  That is the whole point!  I *do* hope you read my entire 
message ... but let's suppose you didn't.  If you are suggesting that all 
those things really *are* forgery, then I'm afraid I'm addressing the 
wrong person for comments.  I'm going to suppose - to hope - that you 
don't.  I appreciate your need for good sense and conservatism, all the 
while encouraging good and useful inovations in SMTP, and I'm trying hard 
to strike that balance.  The sacrifice, and there always is one, is an 
additional piece of user information, a token.  Only difference between 
that and the SPF is that the token is supplied by a user, not an 
administrator, and goes wherever the user goes.  I believe in the use of 
TLS for security, in SASL for avoiding problems with IPV4 (spammers 
hopping ACLs to get relaying permission), for guaranteeing relaying when 
you need it in the car on the way to visit Auntie Jane, or whatever.  I 
don't, though, believe in sitting at a local cafe in mexico and using my 
home MTA just to send out a bit of mail to the local hotel, particularly 
when my MTA sits here in the north of London, England!  If you agree that 
this last situation is in fact plausible, then you are not being 
conservative.

>     including those that would use the DNS.

No, I wouldn't be happy with such corporation or government-run trust, 
either.  This is a problem of cryptographic PKIs, yes.  Even now, the 
situation is troubling, with Key Escro possibly being the largest threat.  
As for the substantial problems inherent in other systems of key 
distribution, I'm interested in hearing the problems specific to mine. :-) 
 This "Key distribution" is nothing more than a database or other simple 
client-server query made by a destination system.  It is limited to sender 
forgery only - nothing less, nothing more.  It depends on getting a yes/no 
answer from the domain owner's database in response to a password, all the 
while ensuring that the database itself is not available for public 
(internet) viewing.  Keys are not even distributed, there is no trust 
relationship, no guy in charge.  DNS holds only information about reaching 
such a database, both the DNS record and the database being under the 
jurisdiction - presumably - of the domain holder, if the database even 
exists.

>     servers, as it has been doing for years?

You're right.  This specification does nothing to address such a problem.  
Remember, though, that it was a mark of progress in spammer tactics when 
they stopped using non-existent domain names, all of which could be 
readily identified by a simple DNS query, and started using legitimate 
ones.  This proposal does nothing to stop people from using legitimate 
domains under their own power to spam with, or even domains under someone 
elses' power if that domain chooses to support spammers or be negligent of 
them.  Still, a lot of mail comes from domain names - legitimate, non-
spamming domain names - whose systems were not used to deliver spam 
purporting to come from those domains, and whose administrators are very 
definite about their anti-spam policy - "No spam here".  I'm pretty sure 
the effect of this sort of specification would make our desires felt, and 
a noticeable impact on our inboxes, just as SPF should.  Having said all 
that, this specification ensures that the only people sending from a 
domain are those in authority to do so, whether truely or maliciously, so 
the envelope sender of a spam suddenly becomes music to the ears of a spam 
fighter and, thencefrom, a good solid RHSBL.  The only kind of domain who 
would then suffer would be careless or uninterested domains, whose 
reputations tell their fate.

> > "mail from:" command: mail from:<[email-masked]> stok=blorb 
> 
> > ...

Yes.  See above.  If you are the guy I should be talking to, then you'll 
see at once why this isn't good enough.  If you aren't, then you probably 
do see it as sufficient, almost in self-contradiction.  Are you the guy?

> What about SMTP-TLS with verified certs required?

Yes.  See above.  PKI = bad.  Also, I've seen more than enough bad 
implementations of this specification which, after a failed handshake of 
TLS caused purely by - say - the CA certificate of the sending MTA's cert 
not being in the list of the recipient as in self-signature for local 
benefit, would break and fail to consider a non-secure transaction for 
continuing the session.  Thanks, but no thanks.

> http://www.rhyolite.com/anti-spam/you-might-be.html

Not at all.  Read it already.  Probably responsible for the very first 
line in my original message.  You did read that, didn't you? :-)

> I wrote it during the first months of the ASRG mailing list.

Hmmm.  I can see that now.  It all makes sense.

Cheers,
Sabahattin
-- 
Thought for the day:
    Intuition (n): an uncanny sixth sense which tells people 
    that they are right, whether they are or not.

Sabahattin Gucukoglu

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Sun, 07 Mar 2004 06:09:29 -0800

attempt at positive contribution....

check out draft-ietf-msgtrk-mtqp and draft-ietf-msgtrk-smtpext for a worked 
example of "send a verifiable token out with the email without giving away 
the password".

there are some interesting corner cases in doing this sort of thing, and 
msgtrk ran across some of them.

one note, though.....

--On 7. mars 2004 08:31 +0000 Sabahattin Gucukoglu 
<[email-masked]> wrote:

> I

reality reset:
I spent most of last week sending mail to people staying in the same hotel 
through my MTA in Trondheim, Norway - transmitting to their MTAs scattered 
across the globe, and them picking up their mail from there.
Seems most of the other people present did too.

seems conservative to me.

---

From: Michael Thomas <[email-masked]>
Date: Sun, 7 Mar 2004 15:03:19 -0800 (PST)

> spammers to adapt to the proposal

Fine. Truth in advertising is wonderful. Then
what?  From what I can tell, anything that falls
short of perfection then gets summarily
executed. What metrics do you suggest when the
answer is less than perfect that doesn't result in
paralysis? That seems to be the real breakdown
here.

		Mike

---

From: Dave Crocker <[email-masked]>
Date: Sun, 7 Mar 2004 15:23:49 -0800

MT> So... instead of pointing out the obvious that
MT> there is no silver bullet, wouldn't it be a lot
MT> more productive to frame this debate in terms of
MT> what incremental steps could be taken to at least
MT> try to change the overall climate?

Serious discussions about spam control acknowledge the fact of
limited, incremental benefit, significant deployment costs, potential
impact on basic modes of legitimate email, and the like.

Unfortunately, serious discussion is rather rare. What is missing from
most proposals is any interest in such careful consideration about
ramifications.

Instead, efforts to explore real costs and real efficacy are met with
the usual plea that this is an emergency and we have to do _something_.

Emotional responses that block legitimate review, in favor of premature
action fall under the category of "hysteria".

The IETF MARID BOF showed that serious discussion is, in fact, possible.
One simply needs to insist on it and encourage it when it happens.

d/
--
 Dave Crocker <dcrocker-at-brandenburg-dot-com>
 Brandenburg InternetWorking <www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Sun, 7 Mar 2004 15:54:02 -0800

At 3:03 PM -0800 3/7/04, Michael Thomas wrote:
>From what I can tell, anything that falls
>executed.

The majority of the "anti-spam" proposals being actively discussed 
are variants on the "prove the sender is who he says he is". None of 
these are perfect, yet:

- they are being actively discussed in the ASRG

- they are being actively discussed on the [email-masked] mailing list

- there was a BOF about them last week in Seoul

- some people are creating experimental implementations and looking 
at the results

This seems different than "summarily executed".

--Paul Hoffman, Director

---

From: Vernon Schryver <[email-masked]>
Date: Sun, 7 Mar 2004 17:43:33 -0700 (MST)

> From: Dave Crocker 

> most proposals is any interest in such careful consideration about

No, let's be honest no matter how impolitic.  What's out of order
from most anti-spam discussions is anything that might squelch the
urgent, exciting, and positive talk.  That certainly includes
consideration of inconvenient ramifications and obvious technical
issues.  The taboos also cover any sentiment like "Ok, I'll implement
this and report back soon with results."

(Recent example technical issues:
  SMTP-TLS does not imply commericial PKI, except in the sense that
   commercial PKI is the only working(?) model of large scale key
   distribution.
 No law, standard, or anything else prohibits an SMTP relay from using
   the same authenticator on output that it used on input for a message.)

> the usual plea that this is an emergency and we have to do _something_.

That's true only in the sense of urgent pleas that _other_ people to
do something.  Every month or so, I check the ASRG archives.  If there
has been a change in the last year, I can't see it.  It's all urgent,
and devoid of anything like reports of actions.  Even survey and BCP
documents start and then fade into the mist.  I just now checked
https://www1.ietf.org/mail-archive/working-groups/asrg/current/maillist.html
to see if I'm being unfair.

Of course this problem is endemic to the Standards Process.  It's worse
with spam because the problem hard verging on unsolvable and few if
any of the participants are trying to ship a product before market
window closes, graduate students trying to complete a thesis, others
trying to publish papers before the grant runs out, or mail system
operators trying to avoid drowning.

There are vendors and so forth, but they see that it might make sense
to ship, install, or test a white box with Linux and SA but it is silly
to spend any salaries or time on "proposals" that can't have any effects
before the spam problem is finished by other effects.

> One simply needs to insist on it and encourage it when it happens.

If http://www.imc.org/ietf-mxcomp/mail-archive/msg00067.html is
reasonably accurate, then I beg to differ.  As far as I can see, it
could be a summary of the most useful content of ASRG mailing list
from March and April, 2003.

  =============================

] From: Paul Hoffman / IMC 

] ...
] The majority of the "anti-spam" proposals being actively discussed
] are variants on the "prove the sender is who he says he is". None of
] these are perfect, yet:

Given the shift of many major spammers from forging domain names to 
using their own throw-aways like xxcdfm1.com, pointlesstomovehere.com,
and attractiveinternetnews.com, "not perfect" is an understatement.

] - they are being actively discussed in the ASRG

Somehow "actively discussed" is doesn't quite convey "continually
discussed round and round without any change."

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Sun, 07 Mar 2004 19:24:12 -0800

> paralysis? That seems to be the real breakdown

acknowledge the non-perfection, say that in your estimation, the benefit is 
still greater than the cost, and march on.
((we're starting to get there (I think) with the proposals presented at the 
MARID BOF - while they don't "solve" anything, IMHO, they have a couple of 
significant benefits, which seems likely to be larger than their cost.))

the idea that you should stop trying because you've been publicly ridiculed 
is one that has always struck me as somewhat strange. Either you believe in 
your ideas even after you've carefully considered the objections raised, or 
you don't.
If you don't, you should give up.
If you still believe in them, you're still alive.

---

From: "Robert G. Brown" <[email-masked]>
Date: Mon, 8 Mar 2004 12:49:18 -0500 (EST)

> paralysis? That seems to be the real breakdown

There is no real breakdown here, and perfection isn't the issue.  A
proposal doesn't have to be perfect; it has to be realistic and not
obviously flawed.

It seems fairly obvious that any serious proposal for anything, let
alone a complex problem such as spam abatement, should include a
feasibility and cost/benefit analysis.  This is SOP throughout business,
government, academe, engineering -- why should IETF proposals and
discussions be exempted from this?

Vernon is pointing out that most of the discussion on this topic on this
list in the recent past has omitted these components, and propose
solutions over and over again that have either been proposed in the past
but rejected as infeasible or expensive or that have been TRIED in the
past, are implemented now, and that are not provING (now, in real time)
to be tremedously effective in preventing spam.  In a previous reply his
remark about some of the proposals being "innumerate" was dead on the
money -- in most cases a very simple analysis of the actual numbers
demonstrates that a proposed measure, after being implemented at great
expense and inconvenience, will only affect a tiny fraction of the
problem (for example) or will not have any effect at all.

There are several things one should accept in any discussion of spam
abatement.  The first and foremost (one that might well go at the very
head of the "principles" statement we were discussing last week) is that
there MAY BE NOTHING THAT THE IETF CAN DO at the protocol level to
control spam, at least not directly.  If you prefer this phrased in a
prettier way, it may be that any measures that WOULD result in an
abatement of spam are all cures that are worse than the disease, either
because of astronomical costs or because they would necessitate removing
some desired/fundamental property from email (such as the ability to
receive mail from strangers without a complex dance that would be even
more annoying and stultifying to electronic communication than spam is).

My memory isn't what it used to be (and it was never very good) but here
is a short list of what I have heard proposed recently as ways of
abating spam (and in some cases, other forms of network abuse such as
viruses as well):

  a) Add a "cost" per message.  Bill Gates himself came out in public
favor of this in the newspaper over the weekend.  (A cynical public is
invited to wonder why.)

  Pros:  "Some people estimate" that a cost of as little as
$0.01/message would deter spammers.  [Who these people are and why their
guess is any better than mine remains unsaid.  I personally note that
costs of anywhere from a dime to a dollar plus the hassle of having to
physically handle paper, envelopes, postage do not seem to have the
slightest effect on the direct advertising fraction in my real mailbox
on a daily basis, with a persistent noise (advertising) to signal (all
other forms of communication combined) ratio that easily exceeds 2:1.]
"It is believed" (by these same people) that everyday users won't mind
paying the cost in time or money because they don't send much mail.

  Cons: I don't want to pay any cost per message.  I don't want to solve
a puzzle to send mail.  I don't want to have to solve a puzzle eight
thousand times to send mail via a list.  I don't want to have to manage
a cost-based apparatus.  The freedom of the Internet is far more
valuable to me than spam abatement and this is a cure worse than any
disease.  Note that I'm just giving MY response to this proposal.  I
send twenty or thirty pieces of mail a day and there are other cheaper
methods of controlling spam.  Finally, I strongly suspect that the
"people" who are estimating that cost will deter spammers are at least
in some cases people who stand to make money hand over fist charging it.

The fundamental premise here seems to be that we are more able and
willing to pay a higher cost for mail than spammers, in spite of the
fact that they actually MAKE money from email and I just use it for
communications with no obvious or direct monetary yield and have to pay
for it with real money, not a fraction of the income generated by the
activity itself. I can only conclude that this proposal is promoted by
individuals who don't actually have to try to get money (from a grant
agency, from a corporate budget, from their own pockets) in order to pay
for mail-related infrastructure and/or by individuals who hope to profit
by collecting some of those added costs.

  b) Require all mail to be electronically signed.  In some cases
encrypted as well, imagining encryption to be a "cost" that might deter
spammers.  Signature/encryption schema vary, tools required to enable
actual authentication of said signatures left vague.

  Pros:  A signature permits you to positively identify mail from
friends and people you know that are not spammers, IF you have an
authority or agent capable of managing the large and widely distributed
database of signature keys involved.  Encryption (as a separate issue)
prevents email from being read in transit, and requires a few
thousandths of a second of modern CPU per KB of payload on both ends.
It has similar key database/certification requirements.

  Cons:  I >>already<< can identify mail from friends and people I know
with 99.99% accuracy.  The "From" header from these individuals is
generally completely correct.  I cannot recall receiving a single piece
of spam, ever, with a header forged so that it appears to come from a
friend of mine.  I have received a small but nonzero number of viruses
that way (small because most of my friends use linux and hence are not
susceptible to most current header-forging viruses).  Encryption
obviously doesn't add a sufficient cost to deter spammers, and
elementary arithmetic indicates that it (or associated code-driven
delays) will NEVER add a cost that would deter spammers before it also
deterred all sorts of legitimate uses of mail and cost a fortune in
wasted resources at Internet scale.

Encrypted mail already is available.  People can already sign their mail
digitally if they wish (and many do) and manage keys as best they can.
Neither measure seems likely to impact spam, because spam is sent by
strangers, and strangers are perfectly capable of electronically signing
or encrypting their mail to me - I just won't recognize the signature
because they are strangers.  I'd expect this to have absolutely no
impact on spam at all besides making my internal whitelist whiter (a
difficult concept in a binary decision).

  c) Wait!  Spam comes from strangers, right?  So we'll require all mail
to come from people you know, or people you "consent" to receive mail
from!  And naturally, you won't consent to receive any spam!

  Pros:  Well, hard to argue with this one.  If I only consent to
receive mail from people I know, or mail from strangers that isn't spam,
won't that abate the spam problem?  Kind of a tautology, that...

  Cons:  Yes, and if only the Palestinians and Israelis would lay down
their arms, open all their borders, convert to Zen Buddhism and embrace
one another in a big love-fest, it would abate the problem they have
with killing each other too.  This is a prime example of a proposal
without any sort of internal reality check or CBA.

First off, I consider the abilty to receive mail from strangers an
essential feature of email, not a bug, and will oppose (by ignoring in
any local implementation) any proposal that would remove that feature.
A cure clearly worse than the disease.  I assume all of you agree, after
all NONE of you know me -- I just up and signed up for the list (self
appointed, un-vouched for).  I recognize just one name among the posters
from other lists I'm on (Joel:-).  I could insert a line like "Visit my
website.  Read my books of poetry there.  Send me money." and the most
vigilant of you would have just received spam (or DID you just receive
spam, heh, heh, hard to tell...;-).

Second, any implementation that LEAVES IN this ability has an associated
problem in pure logic.  A stranger sends me mail.  Do I consent to
receive it?  If no, then I'm rejecting mail from strangers (cure worse
than disease).  If yes, then I can be spammed if the stranger happens to
be sending me spam (as my stealth spam above clearly proves).  End of
story.  Note that the fact that I don't "consent to receive spam" is
irrelevant until somebody invents a psychic 100% efficient spam filter,
and if we had one of those we wouldn't be having this discussion.

Note that one of many reasons this is a useless proposal is that email
identities are cheap, easily changed, readily available from multiple
sources, and in constant churn for legitimate reasons (e.g.  changing
ISPs, getting a new account, seeking anonymity).  If they weren't, we
wouldn't NEED the proposal as blacklisting individuals would suffice.
Blacklisting me as [email-masked] is easy because I'm NOT a (real)
spammer -- I've had the same email address for well over a decade now.
However, if I clicked over to yahoo...I could probably spam the list
once a day "forever" in spite of your best efforts to stop me.

  d) OK, so we can't control individuals as there are order of a billion
of them and millions of them change identities on a given day.  What
about controlling networks?  Only accept mail from "clean" networks.

  Pros:  "Consent" applied at the network level (via white and
blacklists) is already in fairly common use.  I personally believe that
tightening up the regulation of networks might well "help" abate the
spam nuisance.  Not a magic bullet, but improving AUPs and enforcement
of same and clearly requiring SPs to police their networks in specific
ways might, actually, help.  This is very much a matter for open debate,
however, as one has to show in any proposal how it will both alter and
improve what we already achieve here.

  Cons:  Consent at the network level IS in common use, and it may be
that we've already gotten all the benefit it can yield.  There is a time
lag problem here as well -- blacklists are often trying to "catch up"
with the rapidly changing spammer identities.  Network identities aren't
a lot more expensive or difficult to change than individual identities,
and some superlarge domains (e.g. yahoo, hotmail) are effectively
impossible to blacklist (much as I'm sure many of us have been tempted
to do so:-) because there are too many friendly strangers mixed in with
the evil spammers that abuse their services.

There are also complex legal issues to resolve.  AUPs tend to be actual
contracts and have to be dickered out by lawyers.  Enforcment is not
cheap, which is why many providers throw up their hands and refuse to
deal with the problem or blame somebody else.  Some SPs may have a
vested interest in NOT controlling the problem, as they profit
(indirectly) from spammers working through their domains.  Still, this
DOES seem to me at least to be a place where the IETF might make some
small contribution, perhaps by working out a clean partitioning of the
responsibility that everybody seems to want to avoid and getting it
written into future AUPs from the top down, possibly by integrating this
process with e).

  e) Ah, so the networks are avoiding the responsibility of dealing with
they spammers advertently or inadvertently provide network access to.
How about if we write some laws and regulations REQUIRING them to deal
with the problem with fines and other penalties for noncompliance?
While we're at it, how about if we whack spammers upside the head with
all sorts of laws and penalties?  THERE'S a way of adding real costs to
those that profit from spam.

  Pros:  In my opinion, this is almost certain to prove the most
effective way to abate spam in the long run.  One proven model is the
national DNC list, which has been near-miraculous in its effectiveness
in abating phone spam (even MORE expensive, recall, than paper mail,
although the $1-2/message cost didn't deter phone spammers from sending
two to ten messages a day to my household, see a) above).  This adds a
very real "cost" to spamming -- fines and the risk of jail time -- and
forces spammers into the same operational zone as virus hackers.  It
permits the actual pursuit of spammers through networking barriers.  It
adds costs to SPs that "enable" spam (or fail to police it) as well.

  Cons:  Spam is as international as the Internet itself, making
enforcement much more difficult than it first appears.  Laws are great
but have to be enforced, which requires individuals to complain, police
to act, DAs to act, juries to act.  Action takes time and is a
significant cost: legal measures are slow and can be very costly.
Finally and perhaps most important, regulatory laws ALSO reduce the
freedom of the Internet itself and may well prove to be a cure worse
than the disease in the long run.  Nibble away on freedom of speech
here, and somebody seeking control over public discourse will work out a
way of taking a bite out there.

This approach seems to be gradually moving forward of its own accord,
driven by considerable public dissatisfaction with spam.  Prosecutions
are occuring.  Little attention has been paid as yet to SP
responsibilities and liability, but it may well be that a few successful
prosecutions of spammers and/or enabling service providers provides the
"spark" that brings about self-regulation by the rest of the SPs if only
to keep the nose of the very smelly government regulatory camel out of
their tents.  "Clean up your act or we'll clean it up for you" may prove
to be the deciding argument in this particular dialogue.

  f) Filters.

  Pros:  In near-universal use already, at least in some venues.  Can be
very, very effective in abating spam (easily 90% and up).  This is
literally as close as one can come to an automated implementation of a
real "consent" model (which requires examination of content one way or
another, regardless of whether or not a document is signed, encrypted,
certified).

  Cons:  Its a war.  There is an absolutely unavoidable (mathematically
grounded) conflict between effectiveness (percent of false negatives,
spam that gets through) and undesireable side effects (percent of false
positives, real mail that gets rejected as spam). Spammers try to fool
the filter and filter writers try to catch the spam.  Incorporating the
filter into the MTA to ameliorate the false positive problem has
advantages (the hope that the message sender learns that the message
didn't get through and can resend or send an out of band message to open
a hole for the filtered message) and disadvantages (the user can filter
some more in their MUA, the spammer can literally probe the MTA filter
looking for holes in the filter algorithm, additional load on mail
servers).  Filtering is all about cost-benefit at many levels.

This is a very robust and dynamic solution, and is unlikely to go away
unless/until things like legal measures and improved AUPs ameliorate the
problem (if they ever do).  It can be implemented by individuals at the
user level.  It can be implemented by sysadmins at the domain level.
Filters and other intelligent agents COULD be implemented by SPs at the
transport level to identify clients that are spamming, and if it ever
WERE implemented at this level and the SPs came down on AUP violators
like a ton of bricks with contractual monetary penalties, the spam
problem might really significantly abate.  I believe it was Vernon
(again) who pointed out that if tier 1 and 2 providers really wanted to
regulate the problem and were willing to "own" it they could do it now.

Most of the detailed solutions thus far proposed fit into one or more of
the categories above.  Some require universal registration of one sort
or another of individuals or additional registration components for
networks.  Some require certification authorities.  Some require
integrated costing agents or challenge/response systems to be inserted
into (I suppose) MTAs.  Little discussion of costs on the part of the
proposers, often wildly optimistic benefit claims.

In summary, in my opinion, for whatever it might be worth (quite
possibly nothing:-):

  a) A silly solution.  There isn't any reason to believe that adding
scaled costs to spammers will deter spam (cost doesn't deter advertising
anywhere else in our lives at costs up to $1/message, so why should it
deter spam at costs less than this).  Adding costs WILL deter us from
using mail at a far lower threshold than it deters spammers.  At
$0.10/message, I'd be spending $3-5/day on mail, depending on how lists
are billed.  If lists are billed per message, I'd be spending hundreds
of dollars a day at $0.01/message, as I'm on some pretty big lists.
Costs in time (puzzle solving) are just as bad and discriminate against
children and stupid people and handicapped people and list
administrators.

  b) A silly solution to continue to promote AS A SOLUTION TO SPAM and
IN PROTOCOL as part of e.g. a new MTA.  That is, by all means sign
messages if you like.  Create new signature aware mail clients or graft
the capability in to old ones.  Use encrypted mail transport.  Create
distributed signature databases and tools. Just don't suggest that it
will have any impact on spam and don't "force" me to use them, make them
so attractive that I WANT to use them, entirely separate from the spam
issue.  After all, I can't see why this would abate spam in the
slightest on purely logical grounds.  I have no more of a way of
assessing mail from a stranger that signed it than I do of assessing
mail from a stranger that didn't sign it -- either way I have to open
the envelope and filter it and/or look at it.

  c) Don't make me laugh.  I don't consent to spam NOW, and insist on
being able to receive mail from real strangers without the slightest
hint of prior introduction or a priori common ground.  I just cannot
tell what is spam and what isn't without looking, and given order of a
billion (several hundred million and growing) strangers and corporate
entities with access to mail and given that I can set up a dozen
effectively "anonymous" mail accounts in the time it has taken me to
write this response I'll NEVER be able to give consent on a person by
person basis without opening the envelope.  If I have to open the
envelope, this adds NOTHING to the anti-spam filtering measures already
represented in f).

Note that a) through c) I think are so ill-conceived that I (at the
moment) cannot see how they could ever be made into proposals worth
taking seriously.  They all fail a cost-benefit analysis (high cost up
front, likely to have little or even no discernable effect on spam).  So
sure, keep bringing them up and I (and others) will keep pointing this
out.  This isn't "being negative".  It is "not wasting time and money on
a complex measure that, in the end, won't have any useful effect on
spam".

Remember, I think that there may not BE a solution to spam, in the sense
that people seem to be looking for.  There aren't any perpetual motion
machines either.  These two problems may well be linked (information
theory is a common foundation).  So to me, saying that a "solution" that
is obviously flawed is obviously flawed is a CONSTRUCTIVE thing.

On to the good news.

  d) seems worth pursuing.  At least in the sense that AUPs are one of
the effective impediments to spam NOW -- one of the things that largely
keeps it from originating within academic networks, for example -- and
thus there is a real possibility that a better schema for spam
regulation at the network level could result in a significant, enduring,
abatement of spam. Here I think that the IETF could be a very positive
force and provide real guidance (in the form of specifications for
software that might be used by SPs to detect patterns of abuse
originating within their boundaries, for example, and possibly with
legal work on contract templates that would permit them to impose
financial penalties).  Note also that "could" does not mean "will" --
this is a reasonable place to TRY to find a better solution, but one may
not exist or may be too costly or infeasible for other reasons.

  e) Time will tell, but again worth pursuing.  Largely outside of the
IETF's purview, though, except as a sort of "amicus curiae" and insofar
as it integrates with work done on d).

  f) The ONLY solution (aside from AUPs in widespread use against spam),
and in many cases a very effective solution.  It has the advantage of
being evolutionary (responsive to changes in the strategy of spammers
and a changing AUP and legal landscape) and of requiring no higher-level
consent or approval (from e.g. the IETF) to implement on any level from
the individual to the domain.  A variety of free filtering agents are
available as are a variety of commercial filtering agents, and there is
a healthy degree of competition and market choice.  I don't see the
preemminance of filtering as an anti-spam measure changing, and don't
REALLY see much of a role for the IETF in its continuing evolution.

Obviously this is bad news, possibly even unacceptably bad news, to
folks disenchanted with filtering as anti-spam measures.  Bad news or
not, it is reality at the moment and quite possibly really is the best
we can do short of legislation or better enforcement at the AUP/SP
level.

Even things such as signatures, white/blacklisting of networks or
individuals, inclusion of various tokens (solutions to puzzles, e.g.)
are likely to become just another component, and not necessarily a very
powerful component, in the multidimensional decisioning process of such
a filter.  Once the mail has left the point of origination, ONLY a
filter (including the human brain used as a filter) can examine content,
and ONLY content determines whether a message, however else you might
flag it, authenticate it, sign it, white or black list it, vouch for it,
is or isn't spam.  AT the point of origination this is very nearly true
as well, but there the controlling agents are SPs or the government as
they are far away from you personally in space and time.

Perhaps statements "urging" the integration of suitable filters with the
MTA where this is possible are reasonable, I don't know -- arguments
presented here were moderately persuasive that while this wouldn't
affect their effectiveness against spam, the improved treatment of false
positives would permit a lowering of the threshold that identifies a
piece of mail as spam and favorably alter the false negative/false
positive numbers.  This could be so, although I'd worry a bit about
exposing the filter so directly to the spammers to be probed for
weaknesses unless this were accompanied by other measures that might
limit such probing (such as "immediate" exposure to detection followed
by rejection from the network and/or prosecution).

In conclusion, I have seen almost nothing in the entire spam abatement
discussion that can be taken seriously BECAUSE the proposals do not
include the steps described by Vernon.  It would be lovely if future
proposals did, in fact, list BOTH the pros (what one hopes to gain from
the proposal to abate spam) AND the cons (the "obvious problems").  It
would be lovely if they were "numerate" (actually used numbers, backed
up by observation, measurement, real data) where relevant.  It would be
smashing if they had a cost-benefit analysis where the benefits (real or
imagined) of the pros were contrasted with the costs of implementation
and of dealing with the cons.  It would be nice if the numerate part of
the cost-benefit analysis included the scaling issues -- for example,
the costs of imposing a user-level solution on a half-billion or so mail
users using a dazzling array of platforms and operating systems as
opposed to the costs YOU experience hacking something together for
yourself on top of an open source operating system where the solution
comes to you prebuilt for free.  To add a requirement to Vernon's list,
it might also be nice to see what in any proposal is actually new --
we've seen at least three or four notes proposing "solutions" that are
not only not magic bullets, they are already implemented to little or no
effect on spam.  Finally, it does seem like an analysis of whether or
not a proposed solution would stand the test of time is in order.

For example, I can go to my reject/spam folder du jour and -- perhaps --
find some key phrase that is present in all spam.  I can then put that
phrase into my filter definition and say "Voila!  I have solved the spam
problem!"  And for a day, that might even be true, and if I kept it a
dark secret it might last a month or more.  However, can you imagine me
proposing this phrase as THE solution for EVERYBODY on this list?  Just
publishing it here would ensure that it wouldn't last the day.  For a
solution to be realistic, one has to be able to at least argue
persuasively that the first hacker who comes along won't be able to
route around your "impervious" obstacle, that your obstacle will have a
real effect on a whole class of spam that cannot easily be overcome by
someone with access to the same data that you have.

   rgb

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:[email-masked]

---

From: John Leslie <[email-masked]>
Date: Mon, 8 Mar 2004 17:48:17 -0500

Robert G. Brown <[email-masked]> wrote:
> ...
> here is a short list of what I have heard proposed recently as ways
> of abating spam (and in some cases, other forms of network abuse
> such as viruses as well):
> 
>   a) Add a "cost" per message...
> 
> willing to pay a higher cost for mail than spammers...

   Nonetheless, this might be useful for some parts of the 'net.
If Bill Gates wants to run MSN.com in such a way that he collects
a penny per email accepted there, I'm happy to let him try. ;^)

>   b) Require all mail to be electronically signed...
> People can already sign their mail digitally if they wish... I'd
> expect this to have absolutely no impact on spam at all besides
> making my internal whitelist whiter...

   Digitally-signed whitelisting would be a very good thing --
rendering forgery of From addresses nearly harmless. But few of
us believe it will be implemented widely.

>   c) ... require all mail to come from people you know, or people
>      you "consent" to receive mail from...
> I consider the abilty to receive mail from strangers an essential
> feature of email...

   Most of us on this list would agree. Others won't -- for example
many parents would want this model for their children.

>   d) ... Only accept mail from "clean" networks.
> I personally believe that tightening up the regulation of networks
> might well "help" abate the spam nuisance...

   Obviously, not all networks _will_ tighten regulation of spam.
For those that don't, a dose of cost-per-message seems appropriate.

> There is a time lag problem here as well -- blacklists are often
> trying to "catch up" with the rapidly changing spammer identities.

   There is definitely room for improvement there.

> some superlarge domains (e.g. yahoo, hotmail) are effectively
> impossible to blacklist... because there are too many friendly
> strangers mixed in with the evil spammers that abuse their services.

   A small cost-per-message might change attitudes here...

> AUPs tend to be actual contracts and have to be dickered out by
> lawyers. Enforcment is not cheap, which is why many providers throw
> up their hands and refuse to deal with the problem or blame somebody
> else.

   A small cost-per-message might change attitudes here, too.

> Some SPs may have a vested interest in NOT controlling the problem,
> as they profit (indirectly) from spammers working through their
> domains. 

   Very important point here! So long as we insist on subsidizing
such SPs, we're going to keep increasing the spam load.

> Still, this DOES seem to me at least to be a place where the IETF
> might make some small contribution, perhaps by working out a clean
> partitioning of the responsibility that everybody seems to want to
> avoid and getting it written into future AUPs from the top down,
> possibly by integrating this process with e).

   I'm not quite sure what you're proposing: if you mean that IETF
should define the responsibilities of each SP, I'd advise against it.
If you mean defining a machine-readable language for encoding AUP
policy, that might be useful. If you mean defining a protocol for
third parties to express opinions about the effectiveness of AUP
enforcement by various domains, I think that _would_ be useful.
   
>   e) How about if we write some laws and regulations REQUIRING
>      them to deal with the problem with fines and other penalties
>      for noncompliance...
> driven by considerable public dissatisfaction with spam.

   But is this anything other than "damage" to be routed around?

>   f) Filters.
> This is a very robust and dynamic solution, and is unlikely to go
> away unless/until things like legal measures and improved AUPs
> ameliorate the problem (if they ever do). It can be implemented by
> individuals at the user level. It can be implemented by sysadmins
> at the domain level.

   Not an IETF concern...

> Filters and other intelligent agents COULD be implemented by SPs
> at the transport level to identify clients that are spamming,

   Tell you what -- why don't _you_ start the SP that does this?

> and if it ever WERE implemented at this level and the SPs came
> down on AUP violators like a ton of bricks with contractual
> monetary penalties, the spam problem might really significantly
> abate. 

   ... when the SPs are put out of business by the lawsuits...

   I'm afraid it's _much_ safer for a SP to publish a policy that
certain IP addresses are assigned to poorly-monitored customers
than to actually interrupt their traffic.

   And there _is_ a role for IETF in defining a protocol for
publishing such policy.

   It's safer still for third-parties to publish such policy, just
less accurate. Third-parties are _now_ publishing IP ranges that
shouldn't be trusted, with the result that SPs that use those
third-party blacklists get a lot of grief for blocking too much.
It would be a substantial improvement if blacklists would accurately
reflect SP policy on monitoring.

> Most of the detailed solutions thus far proposed fit into one or
> more of the categories above. 

   The problem is difficult enough, IMHO, to require a combination
of approaches.

> Some require universal registration of one sort or another...

   Unlikely, and rather contrary to some basic 'net design principles.

> Some require certification authorities. 

   Many, alas, require certification authority -- singular. A bad
thing, IMHO.

> Some require integrated costing agents or challenge/response systems
> to be inserted into (I suppose) MTAs. 

   Unlikely.

> Little discussion of costs on the part of the proposers, often
> wildly optimistic benefit claims.

   Understandable...

> In summary...
>   a) [ePostage] A silly solution. There isn't any reason to believe
> that adding scaled costs to spammers will deter spam...

   Any cost will "deter" some spam -- but very likely not enough.

>   b) [digital-signiture]... by all means sign messages if you
> like... Just don't suggest that it will have any impact on spam
> and don't "force" me to use them, make them so attractive that I
> WANT to use them...

   A worthy goal, but rather long-term.

>   c) [consent to each sender] I don't consent to spam NOW, and
> insist on being able to receive mail from real strangers...
> I'll NEVER be able to give consent on a person by person basis
> without opening the envelope. If I have to open the envelope,
> this adds NOTHING to the anti-spam filtering measures already
> represented in f).

   Opening the email need not signify "consent". Also, it need not
be done daily, and it could wait for third-party advice.

> Remember, I think that there may not BE a solution to spam, in the
> sense that people seem to be looking for. There aren't any
> perpetual motion machines either. 

   We shouldn't be talking of "a solution"; we should talk spam-
abatement -- measures which improve the signal-to-noise.

   People who want to stop all porn spam -- and there are many --
will have to limit themselves to email from people they already
know; and they'll have to limit their circle of friends to those
clueful enough to avoid all viruses.

   Most of us, I hope, will settle for spam-abatement to the point
where we can ignore the problem for a few days at a time without
subjecting ourselves to sorting through hundreds of spam to find
the one false-positive.

> On to the good news.
> 
>   d) [accept from clean networks] seems worth pursuing. At least
> in the sense that AUPs are one of the effective impediments to spam
> NOW -- one of the things that largely keeps it from originating
> within academic networks, for example -- and thus there is a real
> possibility that a better schema for spam regulation at the network
> level could result in a significant, enduring, abatement of spam.

   Yes -- it is important to note which kinds of networks are
relatively free from spam-sending.

> Here I think that the IETF could be a very positive force and
> provide real guidance (in the form of specifications for
> originating within their boundaries, for example...

   I seriously doubt the IETF could hope to stay ahead of spammers
here. I believe the actual tools for detection of spam-sending will
have to be individually developed and constantly tuned.

> and possibly with legal work on contract templates that would
> permit them to impose financial penalties). 

   I doubt this -- there are too many jurisdictions involved, most
of which have their local lawyers writing the laws.

>   e) [laws forcing SPs to react] Time will tell, but again worth
> pursuing. Largely outside of the IETF's purview, though...

   Agreed.

>   f) [filters] ... has the advantage of being evolutionary...
> and of requiring no higher-level consent or approval (from e.g.
> the IETF) to implement on any level...

   In other words, not an IETF concern. Agreed.

> Obviously this is bad news, possibly even unacceptably bad news,
> to folks disenchanted with filtering as anti-spam measures. Bad
> news or not, it is reality at the moment and quite possibly really
> is the best we can do short of legislation or better enforcement
> at the AUP/SP level.

   But we should not rule those out. Especially we should look to
encourage better AUP enforcement by means other than new laws.

> In conclusion, I have seen almost nothing in the entire spam
> abatement discussion that can be taken seriously BECAUSE the
> proposals do not include the steps described by Vernon.

   While I quite agree that such analysis is necessary before
any proposal nears Proposed Standard, at the same time I'd prefer
to see a brief, succinct, statement of each "new" idea -- so that
I can decide whether to expend my own time considering it in
that much detail.

   Granted, this will result in a large number of ideas for which
"no serious objections were raised", we have been living with that
for years. Perhaps a bot-like posting of the rejection form would
help...

--
John Leslie <[email-masked]>

---

From: Dean Anderson <[email-masked]>
Date: Mon, 8 Mar 2004 23:43:19 -0500 (EST)

>    the same authenticator on output that it used on input for a message.)

Sorry to inject specifics into a meta discussion, but the above is
technically wrong. Mathematical reality imposes some prohibitions. All the
realy has is the certificate and the public key.  It will not be able to
reuse that without the private key, and it won't be very likely to compute
the private key.  So the relay can't use the authenticator given by the
client for exactly the same reasons that you can't have a transparent
https proxy.  If such a proxy were transparent, it would be able to pose a
man-in-the middle attack.  SMTP-TLS has a role to prevent snooping the
message during some transfers. The most significant threat to SMTP
snooping comes from the users' shared LAN or shared cable connection. TLS
is especially useful during such initial transfers to the relay, which has
more controlled access to the internet. However, this has no anti-spam
benefit.  The authenticator certificates (TLS can be anonymous, but we'll
assume for sake of argument that they aren't anonymous) would be given out
by the ISPs just like regular user accounts. We already know that
spammers/abusers will use stolen or disposable user accounts.

PKI isn't an anti-spam solution, except for very small groups who leverage
the non-scalability and lack of widespread use of PKI to their benefit.  
But in those cases, it isn't the PKI that is operative, it is the
non-scalability that is operative. Simply by creating a "secret" header,
which only their small group knows, they obtain the same solution, which
has the same scaling properties. That is, if the "secret" is widely known,
then spammers may learn it. Similarly, if every residential ISP gives
certificates to every user, then spam will come signed by those
certificates, either disposable or stolen. 

A comparable solution is to setup a private network that doesn't connect
to the internet. But we don't see online services disconnecting from the
internet, or just blocking all email from the internet.  While
disconnecting from the internet would perhaps block a lot of spam, it
isn't what the users want or purchased.  Indeed, if the online service is
large enough, it will still have a spam/abuse problem.  As I've said
before, the problem is essentially identical to that of covert or sneaky
channels in Information Theory.

But I have some hopes that nearly all spam/abuse is sent by a relatively
few miscreant script kiddies.  I see that the first CAN-SPAM civil case
has been filed.  This will certainly be interesting.  I expect that once
these script kiddies have been caught and properly punished, that the
spam/abuse problem will be reduced to the "norm" of other anti-social
behaviors, and that things will drop back to less than 1% of mail being
abusive.

But the question of punishment is partly upto the technical community. If
we keep giving script kiddies jobs and promotions in spite of abusive
behavior, then we'll keep giving positive reinforcement to the script
kiddie behavior pattern.  The best thing we can do is make sure that such
activity is career limiting. It is no different from the accounting
scandals and insider trading scandals. If the participants are hired back
into responsible positions with pay raises, then there will be no
incentive to have honest executives or auditors.

Nearly all of the spam abusers that I've caught have been IT people or
administrators who work (or worked) for ISPs.  Some have been caught and
fired. Others have been caught (or abuse traced to a ISPs non-customer
network--where a call to said ISP results in a yell to someone to "knock
it off"), and their activities essentially ignored so long as they
stopped.  That is really the wrong message, and the abuse will no doubt
continue.  In most such cases, they just stop temporarilly, and when they
think the "heat is off", they start doing something else abusive.

P.S. I said last week that we get relay abuse whenever I defend the uses
of open relays. Well, on schedule, we had two relay abuse attempts this
weekend from Tiawanese proxies (haven't had a relay abuse attempt for some
time).  Something over 20,000 messages from the first one, and I haven't
looked at the report for the second, but I'd guess its about the same. All
abuse was blocked.  It is always interesting to review the recipient list
of such abuse.  A funny thing about this case is that the very first
recipient was a [email-masked], and nearly all of the rest were to
.tw and .cn addresses.  So, was [email-masked] the abuser? or just
a known target of the abuser? Perhaps it is just a bogus address that
would tell the abuser (if at starnetinc.com) that the relay abuse was
successful.  Admins at starnet would get a bounce or a log entry if we try
to deliver the message and the address isn't valid.  I've also had some
previous conflict with Starnet admins regarding the uses of open relays,
and we've had abuse originating directly from starnet after/during that
conflict, so it is quite possible that it is a starnet admin.

Googling this address turned up nothing. If it is a real user, they don't
post to any indexed mailing lists. Though, I suppose I should also google
"jmahler at starnetinc dot com" (Another stupid anti-spam tactic; like web
robots can't parse that into an email address).  Without logs of the
proxy, we can't tell where it came from.  But we do know that the abusers
are listening to the IETF list. They are probably employed by companies
that participate in the IETF. We also know that such abuse isn't
commercial, as some naively assume.  The solution to a social problem is
basically social: Companies have to fire abusers, and make sure that abuse
is a career-limiting move, just like was for dishonest auditors.

		--Dean